2024-09-05T11:35:16.658 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T11:35:16.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T11:35:16.766 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407 branch: wip-bharath6-testing-2024-09-05-1413-squid description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: hyelloji@redhat.com first_in_suite: false job_id: '7890407' kernel: client: branch: testing flavor: default kdb: 1 sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath6-testing-2024-09-05-1413-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: e856d6b2462509a377ef39149a05475ed006ebac subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: dry_run: true install: ceph: flavor: default sha1: e856d6b2462509a377ef39149a05475ed006ebac kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath6-testing-2024-09-05-1413-squid sha1: e856d6b2462509a377ef39149a05475ed006ebac owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8220 sha1: e856d6b2462509a377ef39149a05475ed006ebac sleep_before_teardown: 0 subset: 990/1024 suite: fs:workload suite_branch: wip-bharath6-testing-2024-09-05-1413-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: e856d6b2462509a377ef39149a05475ed006ebac syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgCImsZlVYItN29jdfyx6IF5zzo3LrCAh2ylfKiOKJQJrUUQAKVIxQPs172dvIQ+FJl7BFWCjrt5AsTAjAFJTs= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF//48oeJYDS9h8mLWKmAdfplHxOTQxaq7Rv4stVqkJWaDi/pqqOzFrbwDBfwKU3xXkjAla/ffjS9ykbR6L+i+U= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNlJvomQtsd7obd2icRUQd7L3X0Whz8gN2k2VRInstCiPKMZBw9UKo95U3ESbvt/T7M5k5b9VQKZh4as+3DBVxM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-05_11:20:21 tube: smithi user: hyelloji verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T11:35:16.767 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa; will attempt to use it 2024-09-05T11:35:16.767 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks 2024-09-05T11:35:16.767 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T11:35:16.769 INFO:teuthology.task.internal:Checking packages... 2024-09-05T11:35:16.791 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e856d6b2462509a377ef39149a05475ed006ebac' 2024-09-05T11:35:16.791 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T11:35:16.791 INFO:teuthology.packaging:ref: None 2024-09-05T11:35:16.792 INFO:teuthology.packaging:tag: None 2024-09-05T11:35:16.792 INFO:teuthology.packaging:branch: wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T11:35:16.792 INFO:teuthology.packaging:sha1: e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:35:16.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T11:35:16.930 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-261.ge856d6b2 2024-09-05T11:35:16.932 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T11:35:16.938 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T11:35:16.938 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T11:35:16.945 INFO:teuthology.task.internal:Saving configuration 2024-09-05T11:35:16.964 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T11:35:16.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T11:35:16.993 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 11:30:16.692292', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgCImsZlVYItN29jdfyx6IF5zzo3LrCAh2ylfKiOKJQJrUUQAKVIxQPs172dvIQ+FJl7BFWCjrt5AsTAjAFJTs='} 2024-09-05T11:35:17.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 11:30:16.693873', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF//48oeJYDS9h8mLWKmAdfplHxOTQxaq7Rv4stVqkJWaDi/pqqOzFrbwDBfwKU3xXkjAla/ffjS9ykbR6L+i+U='} 2024-09-05T11:35:17.027 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 11:30:16.694920', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNlJvomQtsd7obd2icRUQd7L3X0Whz8gN2k2VRInstCiPKMZBw9UKo95U3ESbvt/T7M5k5b9VQKZh4as+3DBVxM='} 2024-09-05T11:35:17.027 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T11:35:17.034 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-05T11:35:17.034 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-05T11:35:17.034 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-05T11:35:17.034 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T11:35:17.106 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3f641c7ac0>, signals=[15]) 2024-09-05T11:35:17.106 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T11:35:17.112 INFO:teuthology.task.internal:Opening connections... 2024-09-05T11:35:17.113 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T11:35:17.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:35:17.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-05T11:35:17.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:35:17.268 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-05T11:35:17.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:35:17.347 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T11:35:17.353 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-09-05T11:35:17.372 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-09-05T11:35:17.372 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-09-05T11:35:17.430 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T11:35:17.431 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-09-05T11:35:17.431 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T11:35:17.431 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T11:35:17.431 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T11:35:17.431 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-09-05T11:35:17.445 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-05T11:35:17.464 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-05T11:35:17.464 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-05T11:35:17.522 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-09-05T11:35:17.522 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T11:35:17.523 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T11:35:17.524 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-05T11:35:17.544 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-05T11:35:17.563 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-05T11:35:17.563 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-05T11:35:17.621 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-05T11:35:17.621 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-05T11:35:17.621 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-05T11:35:17.621 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-05T11:35:17.621 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T11:35:17.622 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T11:35:17.622 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-05T11:35:17.644 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T11:35:17.654 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T11:35:17.660 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T11:35:17.660 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-09-05T11:35:17.663 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-05T11:35:17.665 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-05T11:35:17.681 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T11:35:17.687 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T11:35:17.688 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-09-05T11:35:17.720 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-05T11:35:17.723 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-05T11:35:17.818 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T11:35:17.889 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T11:35:17.906 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '8ad8b8b160987a14eb199bb67cc00e23a08d43f6'} 2024-09-05T11:35:17.907 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-05T11:35:17.907 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-05T11:35:17.907 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T11:35:17.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:35:17.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-05T11:35:18.096 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T11:35:18.096 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8ad8b8b160987a14eb199bb67cc00e23a08d43f6"... 2024-09-05T11:35:18.096 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-09-05T11:35:18.117 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-503.el9.x86_64 2024-09-05T11:35:18.117 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T11:35:18.118 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-05T11:35:18.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum upgrade -y linux-firmware 2024-09-05T11:35:20.390 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-05T11:35:22.374 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-05T11:35:26.842 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-05T11:35:28.326 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-05T11:35:29.105 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-05T11:35:34.720 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-05T11:35:36.871 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-09-05T11:35:36.872 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-09-05T11:35:36.872 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-09-05T11:35:36.990 INFO:teuthology.task.kernel:Downloading kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-09-05T11:35:36.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T11:35:37.140 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ 2024-09-05T11:35:37.141 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-05T11:35:37.143 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi059.front.sepia.ceph.com to complete... 2024-09-05T11:35:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05 11:35:43 URL:https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm [359646365/359646365] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-05T11:35:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:FINISHED --2024-09-05 11:35:43-- 2024-09-05T11:35:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:Total wall clock time: 6.4s 2024-09-05T11:35:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:Downloaded: 1 files, 343M in 6.1s (56.6 MB/s) 2024-09-05T11:35:43.527 INFO:teuthology.task.kernel:Installing kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-09-05T11:35:43.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-05T11:35:45.726 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying... ######################################## 2024-09-05T11:35:45.856 INFO:teuthology.orchestra.run.smithi059.stdout:Preparing... ######################################## 2024-09-05T11:35:45.984 INFO:teuthology.orchestra.run.smithi059.stdout:Updating / installing... 2024-09-05T11:35:54.277 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-6.11.0_rc5_g8ad8b8b16098-1 ######################################## 2024-09-05T11:36:34.085 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-05T11:36:34.086 DEBUG:teuthology.orchestra.run.smithi059:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-05T11:36:34.180 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/System.map-6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.180 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/config-6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.180 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/vmlinuz-6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/System.map 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/config 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-05T11:36:34.181 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-05T11:36:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ansi_cprng.ko 2024-09-05T11:36:34.183 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/anubis.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/arc4.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_memcpy.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_pq.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_tx.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_xor.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/raid6test.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authenc.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authencesn.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blake2b_generic.ko 2024-09-05T11:36:34.184 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_common.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/camellia_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast5_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast6_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast_common.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ccm.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cmac.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crc32_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_engine.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_user.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cts.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/des_generic.ko 2024-09-05T11:36:34.185 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/dh_generic.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecc.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecdh_generic.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/echainiv.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/essiv.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/fcrypt.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/khazad.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4hc.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/md4.ko 2024-09-05T11:36:34.186 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/michael_mic.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcbc.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcrypt.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/rmd160.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/seed.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/serpent_generic.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tcrypt.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tea.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_common.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_generic.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/vmac.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/wp512.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xcbc.ko 2024-09-05T11:36:34.187 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xor.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xxhash_generic.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_pad.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/ec_sys.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/platform_profile.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbs.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbshc.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/video.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/acard-ahci.ko 2024-09-05T11:36:34.188 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ahci_platform.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ata_generic.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/libahci_platform.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_acpi.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ali.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_amd.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_artop.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atiixp.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atp867x.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd640.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd64x.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cypress.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_efar.ko 2024-09-05T11:36:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt366.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt37x.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it8213.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it821x.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_jmicron.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_marvell.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_mpiix.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_netcell.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ninja32.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87410.ko 2024-09-05T11:36:34.190 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87415.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_oldpiix.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_opti.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_optidma.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pcmcia.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_piccolo.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_rdc.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sch.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_serverworks.ko 2024-09-05T11:36:34.192 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sil680.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sis.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sl82c105.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_triflex.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_via.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pdc_adma.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_inic162x.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_mv.ko 2024-09-05T11:36:34.193 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_nv.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_promise.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_qstor.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil24.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sis.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_svw.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sx4.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_uli.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_via.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_vsc.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay/ks0108.ko 2024-09-05T11:36:34.194 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma/bcma.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe/aoe.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/brd.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd/drbd.ko 2024-09-05T11:36:34.195 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/floppy.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/loop.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/nbd.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk/null_blk.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/pktcdvd.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/rbd.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/virtio_blk.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkfront.ko 2024-09-05T11:36:34.196 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram/zram.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hangcheck-timer.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/via-rng.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-05T11:36:34.197 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/lp.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave/mwave.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ppdev.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tlclk.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-05T11:36:34.198 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/uv_mmtimer.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/virtio_console.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto 2024-09-05T11:36:34.199 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-aes.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-sha.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-05T11:36:34.200 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca/dca.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/amd64_edac.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/e752x_edac.ko 2024-09-05T11:36:34.201 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/edac_mce_amd.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3000_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3200_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5100_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5400_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7300_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7core_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i82975x_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/ie31200_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/sb_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/x38_edac.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/edd.ko 2024-09-05T11:36:34.202 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-ich.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-appleir.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-aureal.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-axff.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-dr.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elecom.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elo.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-emsff.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gaff.ko 2024-09-05T11:36:34.203 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gt683r.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gyration.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtekff.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-hyperv.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-icade.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-keytouch.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-kye.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lcpower.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-led.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lenovo.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-05T11:36:34.204 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-multitouch.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-ortek.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-penmount.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-petalynx.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-picolcd.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-pl.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-plantronics.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-primax.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-rmi.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-common.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-05T11:36:34.205 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-saitek.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-samsung.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sjoy.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sony.ko 2024-09-05T11:36:34.206 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-speedlink.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-steelseries.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sunplus.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tivo.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tmff.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-topseed.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-twinhan.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-uclogic.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-waltop.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-wiimote.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-xinmo.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zpff.ko 2024-09-05T11:36:34.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zydacron.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/uhid.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/wacom.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_balloon.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_utils.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_vmbus.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru3.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7414.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7418.ko 2024-09-05T11:36:34.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adc128d818.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1025.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1026.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1029.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1031.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm9240.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ads7828.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7410.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7411.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7462.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7470.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7475.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7x10.ko 2024-09-05T11:36:34.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/amc6821.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/applesmc.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asb100.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asc7621.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/atxp1.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/coretemp.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dme1737.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds1621.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds620.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc1403.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc6w201.ko 2024-09-05T11:36:34.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71805f.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71882fg.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f75375s.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fam15h_power.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fschmd.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g760a.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g762.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl518sm.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl520sm.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5500_temp.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5k_amb.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmaem.ko 2024-09-05T11:36:34.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmpex.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina209.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina2xx.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/it87.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k10temp.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k8temp.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lineage-pem.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm63.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm73.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm75.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm77.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm78.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm80.ko 2024-09-05T11:36:34.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm83.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm85.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm87.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm90.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm92.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm93.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95234.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95241.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95245.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc2945.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4151.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4215.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4222.ko 2024-09-05T11:36:34.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4245.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4260.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4261.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max16065.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1619.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1668.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max197.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6639.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6650.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6697.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/mcp3021.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6683.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775-core.ko 2024-09-05T11:36:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct7802.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87360.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87427.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pcf8591.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-05T11:36:34.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/powr1220.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5627.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5636.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht15.ko 2024-09-05T11:36:34.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht21.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/shtc1.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sis5595.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47b397.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m1.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m192.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/thmc50.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp102.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp103.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp401.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp421.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via-cputemp.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via686a.ko 2024-09-05T11:36:34.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt1211.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt8231.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627ehf.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627hf.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83781d.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83791d.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83792d.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83793.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83795.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l785ts.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l786ng.ko 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos 2024-09-05T11:36:34.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-05T11:36:34.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-05T11:36:34.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-dev.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-mux.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-smbus.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-stub.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_core.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-05T11:36:34.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-05T11:36:34.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-05T11:36:34.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/gameport.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/joydev.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse 2024-09-05T11:36:34.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/appletouch.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/bcm5974.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/cyapatp.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/sermouse.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/altera_ps2.ko 2024-09-05T11:36:34.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/arc_ps2.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/serio_raw.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/sparse-keymap.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-blinkm.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lm3530.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lp3944.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-ss4200.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-05T11:36:34.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache/bcache.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-bio-prison.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache-smq.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-crypt.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-delay.ko 2024-09-05T11:36:34.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-flakey.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-log-userspace.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-multipath.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-queue-length.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-raid.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-round-robin.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-service-time.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-switch.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-thin-pool.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-verity.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-05T11:36:34.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid0.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid1.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid10.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid456.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptbase.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptctl.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptfc.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptlan.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptsas.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptscsih.ko 2024-09-05T11:36:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptspi.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_ich.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_sch.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/mfd-core.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/sm501.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/viperboard.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/vx855.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/wl1273-core.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds9802als.ko 2024-09-05T11:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds990x.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/bh1770glc.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710/cb710.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/at24.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/max6875.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/enclosure.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/hpilo.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29003.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29020.ko 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d 2024-09-05T11:36:34.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-me.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-txe.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-05T11:36:34.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_7xx1.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_core.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tsl2550.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_balloon.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding/bonding.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-05T11:36:34.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dummy.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/eql.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-05T11:36:34.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-05T11:36:34.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-05T11:36:34.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio 2024-09-05T11:36:34.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-05T11:36:34.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dnet.ko 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex 2024-09-05T11:36:34.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ethoc.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/fealnx.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e 2024-09-05T11:36:34.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-05T11:36:34.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/jme.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-05T11:36:34.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia 2024-09-05T11:36:34.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-05T11:36:34.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan 2024-09-05T11:36:34.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-05T11:36:34.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet 2024-09-05T11:36:34.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/geneve.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ifb.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-05T11:36:34.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvlan.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvtap.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mii.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/net_failover.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/netconsole.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/nlmon.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-05T11:36:34.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/amd.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm87xx.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/broadcom.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/cicada.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/davicom.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/dp83640.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/et1011c.ko 2024-09-05T11:36:34.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/icplus.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/lxt.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/marvell.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/micrel.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/national.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/phylink.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/qsemi.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/realtek.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/smsc.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/ste10Xp.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/vitesse.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-05T11:36:34.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_async.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppoe.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppox.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pptp.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slhc.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slip.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/sungem_phy.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tap.ko 2024-09-05T11:36:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_random.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tun.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/veth.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/virtio_net.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-05T11:36:34.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan/vxlan.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netfront.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb/ntb.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme-core.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme.ko 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport 2024-09-05T11:36:34.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_cs.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_pc.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_serial.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/xen-pcifront.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/i82092.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/pd6729.ko 2024-09-05T11:36:34.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acerhdf.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-05T11:36:34.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/hdaps.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel_ips.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-05T11:36:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-05T11:36:34.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps_parport.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/pps_core.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp_kvm.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/fmpm.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-05T11:36:34.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-em3027.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-05T11:36:34.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-max6900.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-05T11:36:34.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-x1205.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-9xxx.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-sas.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-xxxx.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/BusLogic.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/a100u2w.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/advansys.ko 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx 2024-09-05T11:36:34.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/am53c974.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/atp870u.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa/bfa.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i 2024-09-05T11:36:34.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ch.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dc395x.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-05T11:36:34.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dmx3191d.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esp_scsi.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic/fnic.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hpsa.ko 2024-09-05T11:36:34.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hptiop.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hv_storvsc.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/initio.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ipr.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ips.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci/isci.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc/libfc.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-05T11:36:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas/libsas.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-05T11:36:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvumi.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pmcraid.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla1280.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/raid_class.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_debug.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-05T11:36:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ses.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/st.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/stex.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-05T11:36:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/virtio_scsi.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/wd719x.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb/ssb.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_file.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_iblock.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_mod.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_pscsi.ko 2024-09-05T11:36:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_user.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-05T11:36:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty 2024-09-05T11:36:34.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_gsm.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_hdlc.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/nozomi.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/arc_uart.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core 2024-09-05T11:36:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_aec.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_cif.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_pci_generic.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_sercos3.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-acm.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-05T11:36:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usblp.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usbtmc.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/mdc800.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/microtek.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/adutux.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/appledisplay.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi26.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi62.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ezusb.ko 2024-09-05T11:36:34.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/idmouse.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/iowarrior.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ldusb.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/legousbtower.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usb3503.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usblcd.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/uss720.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/yurex.ko 2024-09-05T11:36:34.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy/phy-generic.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/aircable.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ark3116.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ch341.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cp210x.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cyberjack.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/empeg.ko 2024-09-05T11:36:34.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_ti.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipaq.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipw.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ir-usb.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mct_u232.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7720.ko 2024-09-05T11:36:34.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7840.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/navman.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/omninet.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/opticon.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/option.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/oti6858.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/pl2303.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcaux.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcserial.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/quatech2.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/safe_serial.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/sierra.ko 2024-09-05T11:36:34.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ssu100.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/symbolserial.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_debug.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/visor.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/whiteheat.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/uas.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-05T11:36:34.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-karma.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/usb-storage.ko 2024-09-05T11:36:34.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-05T11:36:34.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_net.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_scsi.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/apple_bl.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/lcd.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_balloon.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_mmio.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci.ko 2024-09-05T11:36:34.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_ring.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2482.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2490.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-05T11:36:34.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/wire.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/hpwdt.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/i6300esb.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-05T11:36:34.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ib700wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ibmasr.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it87_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/machzwd.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/nv_tco.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/softdog.ko 2024-09-05T11:36:34.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/via_wdt.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/wdt_pci.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/xen_wdt.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-evtchn.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntalloc.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntdev.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-05T11:36:34.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-privcmd.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p/9p.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs/affs.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs/befs.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/binfmt_misc.ko 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs 2024-09-05T11:36:34.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs/btrfs.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles/cachefiles.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph/ceph.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda/coda.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs/cramfs.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm/dlm.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs 2024-09-05T11:36:34.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs/f2fs.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/fat.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/msdos.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/vfat.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/cuse.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/fuse.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2/gfs2.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs/hfs.ko 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus 2024-09-05T11:36:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus/hfsplus.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs/isofs.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs/jfs.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd/lockd.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix/minix.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs/netfs.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-05T11:36:34.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfs.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv3.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv4.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/grace.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/nfs_acl.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd/nfsd.ko 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2 2024-09-05T11:36:34.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2/nilfs2.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-celtic.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-centeuro.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-croatian.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-cyrillic.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-gaelic.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-greek.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-iceland.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-inuit.ko 2024-09-05T11:36:34.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-roman.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-romanian.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-turkish.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1250.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1251.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1255.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp737.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp775.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp850.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp852.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp855.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp857.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp860.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp861.ko 2024-09-05T11:36:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp862.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp863.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp864.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp865.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp866.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp869.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp874.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp932.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp936.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp949.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp950.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_euc-jp.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-1.ko 2024-09-05T11:36:34.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-13.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-14.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-15.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-2.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-3.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-4.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-5.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-6.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-7.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-9.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-r.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-ru.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-u.ko 2024-09-05T11:36:34.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_utf8.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-05T11:36:34.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs/overlay.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore/ramoops.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs/reiserfs.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs/romfs.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client/cifs.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_arc4.ko 2024-09-05T11:36:34.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_md4.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs/squashfs.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv/sysv.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf/udf.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs/ufs.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs/xfs.ko 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu 2024-09-05T11:36:34.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu/rcutorture.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/torture.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/asn1_encoder.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc-itu-t.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc8.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libaescfb.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libarc4.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libdes.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/libcrc32c.ko 2024-09-05T11:36:34.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lru_cache.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4_compress.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4hc_compress.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math/cordic.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6/raid6_pq.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_bm.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_fsm.ko 2024-09-05T11:36:34.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_kmp.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd/zstd_compress.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm/hwpoison-inject.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/6lowpan.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_dest.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_fragment.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_hop.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_mobility.ko 2024-09-05T11:36:34.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_routing.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_udp.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/garp.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/mrp.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/p8022.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/psnap.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/stp.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q/8021q.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_fd.ko 2024-09-05T11:36:34.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_rdma.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_virtio.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv/batman-adv.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/br_netfilter.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/bridge.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-05T11:36:34.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-05T11:36:34.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtables.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph/libceph.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/failover.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/pktgen.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_diag.ko 2024-09-05T11:36:34.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv4.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv6.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver/dns_resolver.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/dsa_core.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_brcm.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_dsa.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_none.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_trailer.ko 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan 2024-09-05T11:36:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ah4.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/esp4.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/fou.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/gre.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/inet_diag.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_gre.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_tunnel.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_vti.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipcomp.ko 2024-09-05T11:36:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipip.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-05T11:36:34.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_bic.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_dctcp.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_diag.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_highspeed.ko 2024-09-05T11:36:34.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_htcp.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_hybla.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_illinois.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_lp.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_scalable.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_vegas.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_veno.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_westwood.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_yeah.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tunnel4.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_diag.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_tunnel.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-05T11:36:34.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ah6.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/esp6.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/fou6.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_tunnel.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_vti.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ipcomp6.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-05T11:36:34.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-05T11:36:34.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/sit.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/tunnel6.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key/af_key.ko 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp 2024-09-05T11:36:34.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_core.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_eth.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip6.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_netlink.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ppp.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc/llc.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154/mac802154.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls/mpls_gso.ko 2024-09-05T11:36:34.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-05T11:36:34.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-05T11:36:34.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conncount.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-05T11:36:34.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_log_syslog.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat.ko 2024-09-05T11:36:34.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_irc.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_sip.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_tables.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_log.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_chain_nat.ko 2024-09-05T11:36:34.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_compat.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_ct.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fib.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_hash.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_limit.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_log.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_masq.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_nat.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_numgen.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_queue.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_quota.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_redir.ko 2024-09-05T11:36:34.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject_inet.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_socket.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_tunnel.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_AUDIT.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CT.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_DSCP.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HL.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HMARK.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-05T11:36:34.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LED.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LOG.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NETMAP.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFLOG.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_RATEEST.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_SECMARK.ko 2024-09-05T11:36:34.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TEE.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TPROXY.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TRACE.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_addrtype.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_bpf.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cgroup.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cluster.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_comment.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connbytes.ko 2024-09-05T11:36:34.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlabel.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlimit.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connmark.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_conntrack.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cpu.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dccp.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_devgroup.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dscp.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ecn.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_esp.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hashlimit.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_helper.ko 2024-09-05T11:36:34.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hl.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipcomp.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_iprange.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipvs.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_l2tp.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_length.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_limit.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mac.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mark.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_multiport.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nat.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nfacct.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_osf.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_owner.ko 2024-09-05T11:36:34.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_physdev.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_pkttype.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_policy.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_quota.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_rateest.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_realm.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_recent.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_sctp.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_set.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_socket.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_state.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_statistic.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_string.ko 2024-09-05T11:36:34.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_tcpmss.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_time.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_u32.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink/netlink_diag.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh/nsh.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/openvswitch.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-geneve.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-gre.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-vxlan.ko 2024-09-05T11:36:34.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet/af_packet_diag.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_rdma.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_tcp.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_csum.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_gact.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_mirred.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_nat.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_pedit.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_police.ko 2024-09-05T11:36:34.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_simple.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_skbedit.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_vlan.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_basic.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_bpf.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_flow.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_fw.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_route.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_u32.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_cmp.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_ipset.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_meta.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_nbyte.ko 2024-09-05T11:36:34.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_text.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_u32.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_choke.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_codel.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_drr.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_fq.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_gred.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hfsc.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hhf.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_htb.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_ingress.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio_lib.ko 2024-09-05T11:36:34.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_multiq.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_netem.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_pie.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_plug.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_prio.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_qfq.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_red.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfb.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfq.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_tbf.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_teql.ko 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp 2024-09-05T11:36:34.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp_diag.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/sunrpc.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc_diag.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix/unix_diag.ko 2024-09-05T11:36:34.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys 2024-09-05T11:36:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys/trusted.ko 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias.bin 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.alias.bin 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.bin 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.modinfo 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep.bin 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.devname 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.order 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.softdep 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols 2024-09-05T11:36:34.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols.bin 2024-09-05T11:36:34.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.weakdep 2024-09-05T11:36:34.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-09-05T11:36:34.340 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.340 DEBUG:teuthology.orchestra.run.smithi059:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-05T11:36:34.365 INFO:teuthology.orchestra.run.smithi059.stdout:postinstall scriptlet (using /bin/sh): 2024-09-05T11:36:34.365 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g8ad8b8b16098 /lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout:for file in vmlinuz System.map config; do 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098"; then 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: cp "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: fi 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -d "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" ] && \ 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" >/dev/null 2>&1; then 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout: cp -r "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T11:36:34.366 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep" ]; then 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/sbin/depmod 6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-install remove 6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.367 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-09-05T11:36:34.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-09-05T11:36:34.443 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 82.el9 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8274670 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-09-05T11:36:34.444 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-09-05T11:36:34.445 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:36:34.446 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-09-05T11:36:34.447 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-05T11:36:34.447 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:34.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-05T11:36:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-09-05T11:36:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-05T11:36:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-09-05T11:36:36.557 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-09-05T11:36:36.589 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.3ERbZzFxVR 2024-09-05T11:36:36.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3ERbZzFxVR 2024-09-05T11:36:36.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.3ERbZzFxVR 2024-09-05T11:36:36.807 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.3ERbZzFxVR is 6KB 2024-09-05T11:36:36.827 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.3ERbZzFxVR 2024-09-05T11:36:36.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-09-05T11:36:36.905 INFO:teuthology.orchestra.run.smithi059.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-05T11:36:36.905 INFO:teuthology.orchestra.run.smithi059.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-05T11:36:36.905 INFO:teuthology.orchestra.run.smithi059.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-05T11:36:36.905 INFO:teuthology.orchestra.run.smithi059.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098.conf 2024-09-05T11:36:36.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:36:37.110 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-09-05T11:37:07.113 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-05T11:37:07.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:37:07.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:37:25.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-09-05T11:37:34.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:37:34.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:38:34.569 DEBUG:teuthology.orchestra.remote:timed out 2024-09-05T11:38:46.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:38:46.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:38:46.878 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-09-05T11:38:47.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:38:47.393 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g8ad8b8b16098"... 2024-09-05T11:38:47.393 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-09-05T11:38:47.428 INFO:teuthology.orchestra.run.smithi059.stdout:6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:38:47.428 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g8ad8b8b16098 vs 6.11.0-rc5-g8ad8b8b16098 2024-09-05T11:38:47.428 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-05T11:38:47.428 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-05T11:38:48.429 DEBUG:teuthology.parallel:result is None 2024-09-05T11:38:48.430 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T11:38:48.440 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T11:38:48.441 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T11:38:48.445 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T11:38:48.447 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T11:38:48.479 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T11:38:48.485 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T11:38:48.494 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T11:38:48.495 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T11:38:48.524 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T11:38:48.527 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T11:38:48.582 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T11:38:48.588 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T11:38:48.588 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T11:38:48.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:38:48.606 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T11:38:48.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:38:48.625 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T11:38:48.630 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T11:38:48.648 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T11:38:48.686 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.704 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.715 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.733 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.733 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.759 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T11:38:48.763 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T11:38:48.769 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T11:38:48.769 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T11:38:48.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T11:38:48.775 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T11:38:48.826 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T11:38:48.835 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T11:38:48.835 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T11:38:48.839 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T11:38:48.843 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T11:38:48.870 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:48.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:49.046 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.104 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.181 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:38:49.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T11:38:49.235 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:49.274 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:49.339 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.466 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:38:49.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T11:38:49.533 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:49.574 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T11:38:49.634 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.713 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T11:38:49.761 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:38:49.762 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T11:38:49.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-09-05T11:38:49.833 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-05T11:38:49.835 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-05T11:38:49.873 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T11:38:49.903 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T11:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T11:38:50.371 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T11:38:50.378 INFO:teuthology.task.internal:Starting timer... 2024-09-05T11:38:50.378 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T11:38:50.405 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T11:38:50.414 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-05T11:38:50.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-09-05T11:38:50.496 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-09-05T11:38:50.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-09-05T11:38:50.544 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-09-05T11:38:50.546 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-05T11:38:50.586 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-05T11:38:50.588 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T11:38:50.588 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-09-05T11:38:50.647 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-09-05T11:38:50.647 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-09-05T11:38:50.673 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-09-05T11:38:50.674 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-05T11:38:50.702 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-05T11:38:50.702 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-05T11:38:50.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T11:38:50.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:38:50.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T11:38:50.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:38:50.788 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T11:38:50.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T11:38:50.816 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T11:38:50.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-09-05T11:38:50.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-09-05T11:38:50.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-05T11:38:50.946 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T11:38:50.955 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-05T11:38:50.958 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-05T11:38:51.070 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T11:38:51.082 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T11:38:51.083 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T11:43:30.948 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-05T11:43:30.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:43:30.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:43:31.041 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-09-05T11:43:31.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-05T11:43:31.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-05T11:43:31.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:43:31.229 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-05T11:43:31.305 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-05T11:43:31.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-05T11:43:31.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T11:43:31.384 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-05T11:43:31.461 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-05T11:43:31.462 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T11:43:31.472 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T11:43:31.472 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T11:43:31.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T11:43:31.477 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T11:43:31.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T11:43:31.479 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T11:43:31.479 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T11:43:31.507 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T11:43:31.522 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T11:43:31.533 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T11:43:31.535 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T11:43:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-09-05T11:43:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T11:43:31.558 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-09-05T11:43:31.565 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T11:43:31.573 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T11:43:31.574 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-05T11:43:31.586 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-05T11:43:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T11:43:31.600 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T11:43:31.614 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T11:43:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-09-05T11:43:31.641 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-09-05T11:43:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.653 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.670 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T11:43:31.671 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-05T11:43:31.674 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T11:43:31.674 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-05T11:43:31.674 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.674 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.674 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.675 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T11:43:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-09-05T11:43:31.799 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T11:43:31.799 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-09-05T11:43:31.799 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.799 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.800 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.800 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T11:43:31.800 INFO:teuthology.run_tasks:Running task install... 2024-09-05T11:43:31.809 DEBUG:teuthology.task.install:project ceph 2024-09-05T11:43:31.809 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac'}} 2024-09-05T11:43:31.809 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac'} 2024-09-05T11:43:31.809 INFO:teuthology.task.install:Using flavor: default 2024-09-05T11:43:31.814 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T11:43:31.814 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-05T11:43:31.815 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T11:43:31.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:43:31.817 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T11:43:31.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:43:31.818 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T11:43:31.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:43:31.970 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T11:43:31.970 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T11:43:31.975 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T11:43:31.975 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T11:43:31.987 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T11:43:31.987 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T11:43:32.081 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T11:43:32.081 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:43:32.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T11:43:32.091 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T11:43:32.091 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:43:32.091 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T11:43:32.097 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T11:43:32.097 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:43:32.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T11:43:32.119 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T11:43:32.119 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T11:43:32.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T11:43:32.125 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T11:43:32.138 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T11:43:32.139 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T11:43:32.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T11:43:32.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T11:43:32.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T11:43:32.260 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T11:43:32.269 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T11:43:32.314 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T11:43:32.325 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-05T11:43:32.326 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-05T11:43:32.333 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-09-05T11:43:32.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-09-05T11:43:32.369 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-09-05T11:43:32.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-09-05T11:43:32.553 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-05T11:43:32.557 INFO:teuthology.orchestra.run.smithi089.stdout:45 files removed 2024-09-05T11:43:32.581 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T11:43:32.584 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T11:43:32.646 INFO:teuthology.orchestra.run.smithi059.stdout:45 files removed 2024-09-05T11:43:32.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T11:43:33.131 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-05T11:43:33.132 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-05T11:43:33.417 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 275 kB/s | 84 kB 00:00 2024-09-05T11:43:33.424 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-05T11:43:33.498 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-09-05T11:43:33.700 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T11:43:33.709 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-05T11:43:33.774 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T11:43:33.993 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-05T11:43:34.399 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-05T11:43:34.751 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-05T11:43:35.292 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-05T11:43:37.401 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-05T11:43:39.361 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-05T11:43:39.558 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 20 MB 00:04 2024-09-05T11:43:42.044 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-05T11:43:43.661 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-05T11:43:44.507 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-05T11:43:45.420 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-05T11:43:45.775 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-05T11:43:47.229 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-05T11:43:47.567 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-05T11:43:48.300 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-05T11:43:48.455 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-05T11:43:49.947 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-05T11:43:52.182 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:52.183 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:52.194 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T11:43:52.195 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T11:43:52.196 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T11:43:52.197 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T11:43:52.198 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T11:43:52.200 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T11:43:52.201 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T11:43:52.263 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-05T11:43:52.288 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-05T11:43:52.289 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-09-05T11:43:52.289 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-05T11:43:52.289 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-05T11:43:52.289 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T11:43:52.289 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T11:43:52.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T11:43:52.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T11:43:52.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T11:43:52.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T11:43:52.290 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T11:43:52.291 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T11:43:52.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T11:43:52.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T11:43:52.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T11:43:52.292 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T11:43:52.292 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T11:43:52.293 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T11:43:52.294 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T11:43:52.295 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T11:43:52.295 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T11:43:52.295 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T11:43:52.295 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T11:43:52.295 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T11:43:52.296 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T11:43:52.297 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T11:43:52.298 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T11:43:52.299 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T11:43:52.300 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T11:43:52.301 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T11:43:52.302 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T11:43:52.303 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T11:43:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T11:43:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:43:52.306 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout:Install 148 Packages 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 6 Packages 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 222 M 2024-09-05T11:43:52.307 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-05T11:43:52.999 INFO:teuthology.orchestra.run.smithi089.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 45 kB/s | 6.6 kB 00:00 2024-09-05T11:43:53.274 INFO:teuthology.orchestra.run.smithi089.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 3.2 MB/s | 893 kB 00:00 2024-09-05T11:43:53.349 INFO:teuthology.orchestra.run.smithi089.stdout:(3/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-05T11:43:53.383 INFO:teuthology.orchestra.run.smithi089.stdout:(4/154): ceph-immutable-object-cache-19.1.1-261 1.3 MB/s | 145 kB 00:00 2024-09-05T11:43:53.508 INFO:teuthology.orchestra.run.smithi089.stdout:(5/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-05T11:43:53.709 INFO:teuthology.orchestra.run.smithi089.stdout:(6/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 4.7 MB/s | 1.5 MB 00:00 2024-09-05T11:43:53.784 INFO:teuthology.orchestra.run.smithi089.stdout:(7/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-05T11:43:53.986 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-05T11:43:54.017 INFO:teuthology.orchestra.run.smithi089.stdout:(8/154): ceph-common-19.1.1-261.ge856d6b2.el9.x 18 MB/s | 21 MB 00:01 2024-09-05T11:43:54.059 INFO:teuthology.orchestra.run.smithi089.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 606 kB/s | 25 kB 00:00 2024-09-05T11:43:54.327 INFO:teuthology.orchestra.run.smithi089.stdout:(10/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 20 MB/s | 11 MB 00:00 2024-09-05T11:43:54.385 INFO:teuthology.orchestra.run.smithi089.stdout:(11/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 3.7 MB/s | 222 kB 00:00 2024-09-05T11:43:54.460 INFO:teuthology.orchestra.run.smithi089.stdout:(12/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 361 kB/s | 27 kB 00:00 2024-09-05T11:43:54.536 INFO:teuthology.orchestra.run.smithi089.stdout:(13/154): libcephfs-devel-19.1.1-261.ge856d6b2. 423 kB/s | 32 kB 00:00 2024-09-05T11:43:54.636 INFO:teuthology.orchestra.run.smithi089.stdout:(14/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 7.3 MB/s | 749 kB 00:00 2024-09-05T11:43:54.720 INFO:teuthology.orchestra.run.smithi089.stdout:(15/154): libcephsqlite-19.1.1-261.ge856d6b2.el 1.9 MB/s | 163 kB 00:00 2024-09-05T11:43:54.812 INFO:teuthology.orchestra.run.smithi089.stdout:(16/154): librados-devel-19.1.1-261.ge856d6b2.e 1.3 MB/s | 127 kB 00:00 2024-09-05T11:43:54.920 INFO:teuthology.orchestra.run.smithi089.stdout:(17/154): libradosstriper1-19.1.1-261.ge856d6b2 4.5 MB/s | 503 kB 00:00 2024-09-05T11:43:54.938 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-09-05T11:43:55.254 INFO:teuthology.orchestra.run.smithi089.stdout:(18/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86 11 MB/s | 17 MB 00:01 2024-09-05T11:43:55.288 INFO:teuthology.orchestra.run.smithi089.stdout:(19/154): python3-ceph-argparse-19.1.1-261.ge85 1.3 MB/s | 45 kB 00:00 2024-09-05T11:43:55.363 INFO:teuthology.orchestra.run.smithi089.stdout:(20/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-05T11:43:55.388 INFO:teuthology.orchestra.run.smithi089.stdout:(21/154): python3-ceph-common-19.1.1-261.ge856d 1.3 MB/s | 136 kB 00:00 2024-09-05T11:43:55.414 INFO:teuthology.orchestra.run.smithi089.stdout:(22/154): python3-cephfs-19.1.1-261.ge856d6b2.e 3.1 MB/s | 162 kB 00:00 2024-09-05T11:43:55.447 INFO:teuthology.orchestra.run.smithi089.stdout:(23/154): python3-rados-19.1.1-261.ge856d6b2.el 5.3 MB/s | 320 kB 00:00 2024-09-05T11:43:55.481 INFO:teuthology.orchestra.run.smithi089.stdout:(24/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 4.4 MB/s | 300 kB 00:00 2024-09-05T11:43:55.506 INFO:teuthology.orchestra.run.smithi089.stdout:(25/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T11:43:55.532 INFO:teuthology.orchestra.run.smithi089.stdout:(26/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T11:43:55.584 INFO:teuthology.orchestra.run.smithi089.stdout:(27/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-05T11:43:55.649 INFO:teuthology.orchestra.run.smithi089.stdout:(28/154): ceph-grafana-dashboards-19.1.1-261.ge 477 kB/s | 32 kB 00:00 2024-09-05T11:43:55.700 INFO:teuthology.orchestra.run.smithi089.stdout:(29/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-05T11:43:55.725 INFO:teuthology.orchestra.run.smithi089.stdout:(30/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 1.9 MB/s | 146 kB 00:00 2024-09-05T11:43:56.184 INFO:teuthology.orchestra.run.smithi089.stdout:(31/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 23 MB/s | 49 MB 00:02 2024-09-05T11:43:56.249 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:56.250 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:56.259 INFO:teuthology.orchestra.run.smithi089.stdout:(32/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 6.4 MB/s | 3.6 MB 00:00 2024-09-05T11:43:56.264 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T11:43:56.265 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T11:43:56.265 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T11:43:56.267 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T11:43:56.269 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T11:43:56.272 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T11:43:56.273 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T11:43:56.284 INFO:teuthology.orchestra.run.smithi089.stdout:(33/154): ceph-mgr-modules-core-19.1.1-261.ge85 2.4 MB/s | 249 kB 00:00 2024-09-05T11:43:56.310 INFO:teuthology.orchestra.run.smithi089.stdout:(34/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 986 kB/s | 49 kB 00:00 2024-09-05T11:43:56.335 INFO:teuthology.orchestra.run.smithi089.stdout:(35/154): ceph-prometheus-alerts-19.1.1-261.ge8 328 kB/s | 16 kB 00:00 2024-09-05T11:43:56.337 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-05T11:43:56.356 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T11:43:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T11:43:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T11:43:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T11:43:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T11:43:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T11:43:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T11:43:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T11:43:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T11:43:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T11:43:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T11:43:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T11:43:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-05T11:43:56.369 INFO:teuthology.orchestra.run.smithi189.stdout:Install 148 Packages 2024-09-05T11:43:56.370 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 6 Packages 2024-09-05T11:43:56.370 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:43:56.370 INFO:teuthology.orchestra.run.smithi089.stdout:(36/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 4.8 MB/s | 285 kB 00:00 2024-09-05T11:43:56.370 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 222 M 2024-09-05T11:43:56.371 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-05T11:43:56.477 INFO:teuthology.orchestra.run.smithi089.stdout:(37/154): ceph-mgr-diskprediction-local-19.1.1- 9.8 MB/s | 7.4 MB 00:00 2024-09-05T11:43:56.503 INFO:teuthology.orchestra.run.smithi089.stdout:(38/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 136 kB/s | 18 kB 00:00 2024-09-05T11:43:56.528 INFO:teuthology.orchestra.run.smithi089.stdout:(39/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-05T11:43:57.145 INFO:teuthology.orchestra.run.smithi089.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 120 kB/s | 80 kB 00:00 2024-09-05T11:43:57.262 INFO:teuthology.orchestra.run.smithi089.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 260 kB/s | 191 kB 00:00 2024-09-05T11:43:57.395 INFO:teuthology.orchestra.run.smithi089.stdout:(42/154): libgfortran-11.5.0-2.el9.x86_64.rpm 898 kB/s | 801 kB 00:00 2024-09-05T11:43:57.501 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:57.502 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T11:43:57.515 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T11:43:57.516 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T11:43:57.517 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T11:43:57.518 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T11:43:57.519 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T11:43:57.522 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T11:43:57.523 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T11:43:57.595 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T11:43:57.613 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T11:43:57.614 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T11:43:57.615 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-09-05T11:43:57.616 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T11:43:57.617 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T11:43:57.618 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T11:43:57.619 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T11:43:57.620 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T11:43:57.621 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T11:43:57.622 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T11:43:57.623 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T11:43:57.624 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T11:43:57.625 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-09-05T11:43:57.626 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 6 Packages 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:43:57.627 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 222 M 2024-09-05T11:43:57.628 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-09-05T11:43:57.996 INFO:teuthology.orchestra.run.smithi089.stdout:(43/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 298 kB/s | 253 kB 00:00 2024-09-05T11:43:58.071 INFO:teuthology.orchestra.run.smithi089.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 159 kB/s | 106 kB 00:00 2024-09-05T11:43:58.148 INFO:teuthology.orchestra.run.smithi059.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 44 kB/s | 6.6 kB 00:00 2024-09-05T11:43:58.222 INFO:teuthology.orchestra.run.smithi089.stdout:(45/154): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-05T11:43:58.316 INFO:teuthology.orchestra.run.smithi059.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 5.2 MB/s | 893 kB 00:00 2024-09-05T11:43:58.382 INFO:teuthology.orchestra.run.smithi059.stdout:(3/154): ceph-immutable-object-cache-19.1.1-261 2.1 MB/s | 145 kB 00:00 2024-09-05T11:43:58.445 INFO:teuthology.orchestra.run.smithi189.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 33 kB/s | 6.6 kB 00:00 2024-09-05T11:43:58.516 INFO:teuthology.orchestra.run.smithi059.stdout:(4/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-05T11:43:58.675 INFO:teuthology.orchestra.run.smithi059.stdout:(5/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 8.2 MB/s | 2.4 MB 00:00 2024-09-05T11:43:58.714 INFO:teuthology.orchestra.run.smithi089.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 188 kB/s | 135 kB 00:00 2024-09-05T11:43:58.716 INFO:teuthology.orchestra.run.smithi059.stdout:(6/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-05T11:43:58.781 INFO:teuthology.orchestra.run.smithi089.stdout:(47/154): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-05T11:43:58.795 INFO:teuthology.orchestra.run.smithi189.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 2.5 MB/s | 893 kB 00:00 2024-09-05T11:43:58.856 INFO:teuthology.orchestra.run.smithi089.stdout:(48/154): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-09-05T11:43:58.896 INFO:teuthology.orchestra.run.smithi189.stdout:(3/154): ceph-immutable-object-cache-19.1.1-261 1.4 MB/s | 145 kB 00:00 2024-09-05T11:43:58.921 INFO:teuthology.orchestra.run.smithi089.stdout:(49/154): bison-3.7.4-5.el9.x86_64.rpm 6.6 MB/s | 943 kB 00:00 2024-09-05T11:43:58.955 INFO:teuthology.orchestra.run.smithi089.stdout:(50/154): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-09-05T11:43:58.980 INFO:teuthology.orchestra.run.smithi089.stdout:(51/154): python3-urllib3-1.26.5-6.el9.noarch.r 286 kB/s | 215 kB 00:00 2024-09-05T11:43:58.997 INFO:teuthology.orchestra.run.smithi089.stdout:(52/154): elfutils-libelf-devel-0.191-4.el9.x86 287 kB/s | 22 kB 00:00 2024-09-05T11:43:59.031 INFO:teuthology.orchestra.run.smithi089.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-05T11:43:59.056 INFO:teuthology.orchestra.run.smithi089.stdout:(54/154): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-05T11:43:59.098 INFO:teuthology.orchestra.run.smithi089.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 271 kB/s | 18 kB 00:00 2024-09-05T11:43:59.123 INFO:teuthology.orchestra.run.smithi089.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-05T11:43:59.175 INFO:teuthology.orchestra.run.smithi059.stdout:(7/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 9.4 MB/s | 4.7 MB 00:00 2024-09-05T11:43:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:(4/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-09-05T11:43:59.182 INFO:teuthology.orchestra.run.smithi089.stdout:(57/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-09-05T11:43:59.216 INFO:teuthology.orchestra.run.smithi089.stdout:(58/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-05T11:43:59.291 INFO:teuthology.orchestra.run.smithi089.stdout:(59/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-09-05T11:43:59.350 INFO:teuthology.orchestra.run.smithi089.stdout:(60/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 421 kB/s | 24 kB 00:00 2024-09-05T11:43:59.400 INFO:teuthology.orchestra.run.smithi089.stdout:(61/154): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-05T11:43:59.433 INFO:teuthology.orchestra.run.smithi089.stdout:(62/154): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-05T11:43:59.459 INFO:teuthology.orchestra.run.smithi089.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-05T11:43:59.496 INFO:teuthology.orchestra.run.smithi189.stdout:(5/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-05T11:43:59.509 INFO:teuthology.orchestra.run.smithi089.stdout:(64/154): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-05T11:43:59.534 INFO:teuthology.orchestra.run.smithi089.stdout:(65/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-05T11:43:59.568 INFO:teuthology.orchestra.run.smithi089.stdout:(66/154): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-05T11:43:59.710 INFO:teuthology.orchestra.run.smithi089.stdout:(67/154): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-05T11:43:59.750 INFO:teuthology.orchestra.run.smithi059.stdout:(8/154): ceph-common-19.1.1-261.ge856d6b2.el9.x 12 MB/s | 21 MB 00:01 2024-09-05T11:43:59.810 INFO:teuthology.orchestra.run.smithi089.stdout:(68/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-05T11:43:59.842 INFO:teuthology.orchestra.run.smithi059.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 275 kB/s | 25 kB 00:00 2024-09-05T11:43:59.888 INFO:teuthology.orchestra.run.smithi089.stdout:(69/154): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-05T11:43:59.905 INFO:teuthology.orchestra.run.smithi189.stdout:(6/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 2.4 MB/s | 2.4 MB 00:01 2024-09-05T11:43:59.919 INFO:teuthology.orchestra.run.smithi089.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-05T11:43:59.976 INFO:teuthology.orchestra.run.smithi059.stdout:(10/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 13 MB/s | 11 MB 00:00 2024-09-05T11:44:00.005 INFO:teuthology.orchestra.run.smithi189.stdout:(7/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 9.3 MB/s | 4.7 MB 00:00 2024-09-05T11:44:00.086 INFO:teuthology.orchestra.run.smithi089.stdout:(71/154): openblas-openmp-0.3.26-2.el9.x86_64.r 8.6 MB/s | 4.9 MB 00:00 2024-09-05T11:44:00.145 INFO:teuthology.orchestra.run.smithi089.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-09-05T11:44:00.193 INFO:teuthology.orchestra.run.smithi059.stdout:(11/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86 11 MB/s | 17 MB 00:01 2024-09-05T11:44:00.218 INFO:teuthology.orchestra.run.smithi059.stdout:(12/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 918 kB/s | 222 kB 00:00 2024-09-05T11:44:00.268 INFO:teuthology.orchestra.run.smithi059.stdout:(13/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 360 kB/s | 27 kB 00:00 2024-09-05T11:44:00.318 INFO:teuthology.orchestra.run.smithi059.stdout:(14/154): libcephfs-devel-19.1.1-261.ge856d6b2. 317 kB/s | 32 kB 00:00 2024-09-05T11:44:00.322 INFO:teuthology.orchestra.run.smithi189.stdout:(8/154): ceph-common-19.1.1-261.ge856d6b2.el9.x 10 MB/s | 21 MB 00:02 2024-09-05T11:44:00.360 INFO:teuthology.orchestra.run.smithi059.stdout:(15/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 8.0 MB/s | 749 kB 00:00 2024-09-05T11:44:00.386 INFO:teuthology.orchestra.run.smithi059.stdout:(16/154): libcephsqlite-19.1.1-261.ge856d6b2.el 2.4 MB/s | 163 kB 00:00 2024-09-05T11:44:00.389 INFO:teuthology.orchestra.run.smithi189.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 379 kB/s | 25 kB 00:00 2024-09-05T11:44:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:(17/154): librados-devel-19.1.1-261.ge856d6b2.e 2.5 MB/s | 127 kB 00:00 2024-09-05T11:44:00.453 INFO:teuthology.orchestra.run.smithi059.stdout:(18/154): libradosstriper1-19.1.1-261.ge856d6b2 7.3 MB/s | 503 kB 00:00 2024-09-05T11:44:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:(19/154): python3-ceph-argparse-19.1.1-261.ge85 453 kB/s | 45 kB 00:00 2024-09-05T11:44:00.637 INFO:teuthology.orchestra.run.smithi059.stdout:(20/154): python3-ceph-common-19.1.1-261.ge856d 1.6 MB/s | 136 kB 00:00 2024-09-05T11:44:00.729 INFO:teuthology.orchestra.run.smithi089.stdout:(73/154): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-05T11:44:00.729 INFO:teuthology.orchestra.run.smithi059.stdout:(21/154): python3-cephfs-19.1.1-261.ge856d6b2.e 1.7 MB/s | 162 kB 00:00 2024-09-05T11:44:00.754 INFO:teuthology.orchestra.run.smithi089.stdout:(74/154): python3-attrs-20.3.0-7.el9.noarch.rpm 131 kB/s | 87 kB 00:00 2024-09-05T11:44:00.829 INFO:teuthology.orchestra.run.smithi089.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 611 kB/s | 61 kB 00:00 2024-09-05T11:44:00.862 INFO:teuthology.orchestra.run.smithi059.stdout:(22/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-05T11:44:00.887 INFO:teuthology.orchestra.run.smithi059.stdout:(23/154): python3-rados-19.1.1-261.ge856d6b2.el 2.0 MB/s | 320 kB 00:00 2024-09-05T11:44:00.913 INFO:teuthology.orchestra.run.smithi089.stdout:(76/154): postgresql-server-13.16-1.el9.x86_64. 5.6 MB/s | 5.8 MB 00:01 2024-09-05T11:44:00.938 INFO:teuthology.orchestra.run.smithi059.stdout:(24/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 3.9 MB/s | 300 kB 00:00 2024-09-05T11:44:00.938 INFO:teuthology.orchestra.run.smithi089.stdout:(77/154): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-05T11:44:00.963 INFO:teuthology.orchestra.run.smithi059.stdout:(25/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 1.3 MB/s | 99 kB 00:00 2024-09-05T11:44:00.989 INFO:teuthology.orchestra.run.smithi089.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-09-05T11:44:01.005 INFO:teuthology.orchestra.run.smithi059.stdout:(26/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-05T11:44:01.014 INFO:teuthology.orchestra.run.smithi089.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-09-05T11:44:01.048 INFO:teuthology.orchestra.run.smithi089.stdout:(80/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-09-05T11:44:01.065 INFO:teuthology.orchestra.run.smithi089.stdout:(81/154): python3-markupsafe-1.1.1-12.el9.x86_6 686 kB/s | 35 kB 00:00 2024-09-05T11:44:01.080 INFO:teuthology.orchestra.run.smithi059.stdout:(27/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-05T11:44:01.081 INFO:teuthology.orchestra.run.smithi189.stdout:(10/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 9.9 MB/s | 11 MB 00:01 2024-09-05T11:44:01.123 INFO:teuthology.orchestra.run.smithi089.stdout:(82/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-05T11:44:01.156 INFO:teuthology.orchestra.run.smithi189.stdout:(11/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 2.9 MB/s | 222 kB 00:00 2024-09-05T11:44:01.165 INFO:teuthology.orchestra.run.smithi089.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-05T11:44:01.172 INFO:teuthology.orchestra.run.smithi059.stdout:(28/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-05T11:44:01.206 INFO:teuthology.orchestra.run.smithi189.stdout:(12/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 542 kB/s | 27 kB 00:00 2024-09-05T11:44:01.214 INFO:teuthology.orchestra.run.smithi059.stdout:(29/154): ceph-grafana-dashboards-19.1.1-261.ge 238 kB/s | 32 kB 00:00 2024-09-05T11:44:01.248 INFO:teuthology.orchestra.run.smithi189.stdout:(13/154): libcephfs-devel-19.1.1-261.ge856d6b2. 760 kB/s | 32 kB 00:00 2024-09-05T11:44:01.248 INFO:teuthology.orchestra.run.smithi059.stdout:(30/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 1.9 MB/s | 146 kB 00:00 2024-09-05T11:44:01.249 INFO:teuthology.orchestra.run.smithi089.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-09-05T11:44:01.274 INFO:teuthology.orchestra.run.smithi089.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 711 kB/s | 77 kB 00:00 2024-09-05T11:44:01.341 INFO:teuthology.orchestra.run.smithi089.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-05T11:44:01.406 INFO:teuthology.orchestra.run.smithi189.stdout:(14/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 4.6 MB/s | 749 kB 00:00 2024-09-05T11:44:01.408 INFO:teuthology.orchestra.run.smithi089.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-05T11:44:01.450 INFO:teuthology.orchestra.run.smithi089.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-05T11:44:01.490 INFO:teuthology.orchestra.run.smithi189.stdout:(15/154): libcephsqlite-19.1.1-261.ge856d6b2.el 1.9 MB/s | 163 kB 00:00 2024-09-05T11:44:01.509 INFO:teuthology.orchestra.run.smithi089.stdout:(89/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-05T11:44:01.534 INFO:teuthology.orchestra.run.smithi089.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-09-05T11:44:01.565 INFO:teuthology.orchestra.run.smithi189.stdout:(16/154): librados-devel-19.1.1-261.ge856d6b2.e 1.7 MB/s | 127 kB 00:00 2024-09-05T11:44:01.593 INFO:teuthology.orchestra.run.smithi089.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 54 kB 00:00 2024-09-05T11:44:01.607 INFO:teuthology.orchestra.run.smithi059.stdout:(31/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 9.1 MB/s | 3.6 MB 00:00 2024-09-05T11:44:01.632 INFO:teuthology.orchestra.run.smithi189.stdout:(17/154): libradosstriper1-19.1.1-261.ge856d6b2 7.4 MB/s | 503 kB 00:00 2024-09-05T11:44:01.685 INFO:teuthology.orchestra.run.smithi089.stdout:(92/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.0 MB/s | 5.1 MB 00:00 2024-09-05T11:44:01.710 INFO:teuthology.orchestra.run.smithi089.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-05T11:44:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:(32/154): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-05T11:44:01.760 INFO:teuthology.orchestra.run.smithi089.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-09-05T11:44:01.774 INFO:teuthology.orchestra.run.smithi059.stdout:(33/154): ceph-mgr-modules-core-19.1.1-261.ge85 1.5 MB/s | 249 kB 00:00 2024-09-05T11:44:01.811 INFO:teuthology.orchestra.run.smithi089.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-05T11:44:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:(34/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 658 kB/s | 49 kB 00:00 2024-09-05T11:44:01.836 INFO:teuthology.orchestra.run.smithi089.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 477 kB/s | 36 kB 00:00 2024-09-05T11:44:01.866 INFO:teuthology.orchestra.run.smithi059.stdout:(35/154): ceph-prometheus-alerts-19.1.1-261.ge8 179 kB/s | 16 kB 00:00 2024-09-05T11:44:01.891 INFO:teuthology.orchestra.run.smithi059.stdout:(36/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 4.2 MB/s | 285 kB 00:00 2024-09-05T11:44:01.895 INFO:teuthology.orchestra.run.smithi089.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-09-05T11:44:01.945 INFO:teuthology.orchestra.run.smithi089.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-05T11:44:01.970 INFO:teuthology.orchestra.run.smithi089.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-09-05T11:44:01.975 INFO:teuthology.orchestra.run.smithi059.stdout:(37/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 219 kB/s | 18 kB 00:00 2024-09-05T11:44:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:(38/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-05T11:44:02.037 INFO:teuthology.orchestra.run.smithi089.stdout:(100/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-05T11:44:02.058 INFO:teuthology.orchestra.run.smithi189.stdout:(18/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-05T11:44:02.071 INFO:teuthology.orchestra.run.smithi089.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 764 kB/s | 96 kB 00:00 2024-09-05T11:44:02.116 INFO:teuthology.orchestra.run.smithi189.stdout:(19/154): python3-ceph-argparse-19.1.1-261.ge85 777 kB/s | 45 kB 00:00 2024-09-05T11:44:02.129 INFO:teuthology.orchestra.run.smithi089.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-09-05T11:44:02.151 INFO:teuthology.orchestra.run.smithi059.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-05T11:44:02.154 INFO:teuthology.orchestra.run.smithi089.stdout:(103/154): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-05T11:44:02.175 INFO:teuthology.orchestra.run.smithi189.stdout:(20/154): python3-ceph-common-19.1.1-261.ge856d 2.3 MB/s | 136 kB 00:00 2024-09-05T11:44:02.205 INFO:teuthology.orchestra.run.smithi089.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T11:44:02.233 INFO:teuthology.orchestra.run.smithi189.stdout:(21/154): python3-cephfs-19.1.1-261.ge856d6b2.e 2.7 MB/s | 162 kB 00:00 2024-09-05T11:44:02.239 INFO:teuthology.orchestra.run.smithi089.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-05T11:44:02.259 INFO:teuthology.orchestra.run.smithi059.stdout:(40/154): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-05T11:44:02.289 INFO:teuthology.orchestra.run.smithi089.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-09-05T11:44:02.292 INFO:teuthology.orchestra.run.smithi189.stdout:(22/154): python3-rados-19.1.1-261.ge856d6b2.el 5.4 MB/s | 320 kB 00:00 2024-09-05T11:44:02.335 INFO:teuthology.orchestra.run.smithi059.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-05T11:44:02.351 INFO:teuthology.orchestra.run.smithi189.stdout:(23/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 5.0 MB/s | 300 kB 00:00 2024-09-05T11:44:02.360 INFO:teuthology.orchestra.run.smithi059.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-05T11:44:02.364 INFO:teuthology.orchestra.run.smithi089.stdout:(107/154): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-09-05T11:44:02.401 INFO:teuthology.orchestra.run.smithi189.stdout:(24/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 1.9 MB/s | 99 kB 00:00 2024-09-05T11:44:02.418 INFO:teuthology.orchestra.run.smithi059.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-05T11:44:02.451 INFO:teuthology.orchestra.run.smithi189.stdout:(25/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T11:44:02.477 INFO:teuthology.orchestra.run.smithi059.stdout:(44/154): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-05T11:44:02.536 INFO:teuthology.orchestra.run.smithi059.stdout:(45/154): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-05T11:44:02.557 INFO:teuthology.orchestra.run.smithi089.stdout:(108/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-09-05T11:44:02.582 INFO:teuthology.orchestra.run.smithi089.stdout:(109/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 120 kB/s | 26 kB 00:00 2024-09-05T11:44:02.616 INFO:teuthology.orchestra.run.smithi089.stdout:(110/154): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-05T11:44:02.619 INFO:teuthology.orchestra.run.smithi059.stdout:(46/154): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-05T11:44:02.649 INFO:teuthology.orchestra.run.smithi089.stdout:(111/154): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-09-05T11:44:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-05T11:44:02.695 INFO:teuthology.orchestra.run.smithi059.stdout:(47/154): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-09-05T11:44:02.701 INFO:teuthology.orchestra.run.smithi189.stdout:(26/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-05T11:44:02.733 INFO:teuthology.orchestra.run.smithi089.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-05T11:44:02.743 INFO:teuthology.orchestra.run.smithi189.stdout:(27/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 4.0 MB/s | 170 kB 00:00 2024-09-05T11:44:02.794 INFO:teuthology.orchestra.run.smithi189.stdout:(28/154): ceph-grafana-dashboards-19.1.1-261.ge 636 kB/s | 32 kB 00:00 2024-09-05T11:44:02.817 INFO:teuthology.orchestra.run.smithi089.stdout:(114/154): libarrow-9.0.0-11.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-09-05T11:44:02.842 INFO:teuthology.orchestra.run.smithi089.stdout:(115/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-09-05T11:44:02.844 INFO:teuthology.orchestra.run.smithi189.stdout:(29/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 2.9 MB/s | 146 kB 00:00 2024-09-05T11:44:02.867 INFO:teuthology.orchestra.run.smithi089.stdout:(116/154): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-05T11:44:02.893 INFO:teuthology.orchestra.run.smithi089.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-05T11:44:02.926 INFO:teuthology.orchestra.run.smithi089.stdout:(118/154): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-09-05T11:44:02.952 INFO:teuthology.orchestra.run.smithi089.stdout:(119/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-05T11:44:02.985 INFO:teuthology.orchestra.run.smithi089.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-05T11:44:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-05T11:44:03.036 INFO:teuthology.orchestra.run.smithi089.stdout:(122/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-05T11:44:03.053 INFO:teuthology.orchestra.run.smithi089.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-09-05T11:44:03.069 INFO:teuthology.orchestra.run.smithi189.stdout:(30/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 16 MB/s | 3.6 MB 00:00 2024-09-05T11:44:03.070 INFO:teuthology.orchestra.run.smithi089.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-05T11:44:03.095 INFO:teuthology.orchestra.run.smithi089.stdout:(125/154): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-05T11:44:03.120 INFO:teuthology.orchestra.run.smithi089.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-05T11:44:03.146 INFO:teuthology.orchestra.run.smithi089.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-05T11:44:03.162 INFO:teuthology.orchestra.run.smithi059.stdout:(48/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 15 MB/s | 49 MB 00:03 2024-09-05T11:44:03.171 INFO:teuthology.orchestra.run.smithi089.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-05T11:44:03.197 INFO:teuthology.orchestra.run.smithi089.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T11:44:03.230 INFO:teuthology.orchestra.run.smithi089.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-05T11:44:03.254 INFO:teuthology.orchestra.run.smithi059.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-05T11:44:03.255 INFO:teuthology.orchestra.run.smithi089.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T11:44:03.289 INFO:teuthology.orchestra.run.smithi089.stdout:(132/154): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-05T11:44:03.314 INFO:teuthology.orchestra.run.smithi089.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-05T11:44:03.328 INFO:teuthology.orchestra.run.smithi189.stdout:(31/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86 4.9 MB/s | 17 MB 00:03 2024-09-05T11:44:03.329 INFO:teuthology.orchestra.run.smithi059.stdout:(50/154): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-05T11:44:03.348 INFO:teuthology.orchestra.run.smithi089.stdout:(134/154): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-05T11:44:03.374 INFO:teuthology.orchestra.run.smithi089.stdout:(135/154): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-05T11:44:03.379 INFO:teuthology.orchestra.run.smithi059.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 173 kB/s | 22 kB 00:00 2024-09-05T11:44:03.399 INFO:teuthology.orchestra.run.smithi089.stdout:(136/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-05T11:44:03.413 INFO:teuthology.orchestra.run.smithi059.stdout:(52/154): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-05T11:44:03.424 INFO:teuthology.orchestra.run.smithi089.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-05T11:44:03.450 INFO:teuthology.orchestra.run.smithi089.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-05T11:44:03.475 INFO:teuthology.orchestra.run.smithi089.stdout:(139/154): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-05T11:44:03.480 INFO:teuthology.orchestra.run.smithi059.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-05T11:44:03.500 INFO:teuthology.orchestra.run.smithi089.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-05T11:44:03.505 INFO:teuthology.orchestra.run.smithi059.stdout:(54/154): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-05T11:44:03.526 INFO:teuthology.orchestra.run.smithi089.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-05T11:44:03.551 INFO:teuthology.orchestra.run.smithi089.stdout:(142/154): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-05T11:44:03.576 INFO:teuthology.orchestra.run.smithi089.stdout:(143/154): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-05T11:44:03.580 INFO:teuthology.orchestra.run.smithi059.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-05T11:44:03.602 INFO:teuthology.orchestra.run.smithi089.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-05T11:44:03.622 INFO:teuthology.orchestra.run.smithi059.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-05T11:44:03.644 INFO:teuthology.orchestra.run.smithi089.stdout:(145/154): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-05T11:44:03.664 INFO:teuthology.orchestra.run.smithi059.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-05T11:44:03.669 INFO:teuthology.orchestra.run.smithi089.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-05T11:44:03.694 INFO:teuthology.orchestra.run.smithi089.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-05T11:44:03.723 INFO:teuthology.orchestra.run.smithi059.stdout:(58/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-09-05T11:44:03.753 INFO:teuthology.orchestra.run.smithi089.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T11:44:03.753 INFO:teuthology.orchestra.run.smithi189.stdout:(32/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 14 MB/s | 49 MB 00:03 2024-09-05T11:44:03.765 INFO:teuthology.orchestra.run.smithi059.stdout:(59/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 318 kB/s | 45 kB 00:00 2024-09-05T11:44:03.778 INFO:teuthology.orchestra.run.smithi189.stdout:(33/154): ceph-mgr-modules-core-19.1.1-261.ge85 554 kB/s | 249 kB 00:00 2024-09-05T11:44:03.799 INFO:teuthology.orchestra.run.smithi059.stdout:(60/154): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-05T11:44:03.804 INFO:teuthology.orchestra.run.smithi189.stdout:(34/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 986 kB/s | 49 kB 00:00 2024-09-05T11:44:03.829 INFO:teuthology.orchestra.run.smithi189.stdout:(35/154): ceph-prometheus-alerts-19.1.1-261.ge8 328 kB/s | 16 kB 00:00 2024-09-05T11:44:03.832 INFO:teuthology.orchestra.run.smithi059.stdout:(61/154): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-05T11:44:03.874 INFO:teuthology.orchestra.run.smithi059.stdout:(62/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-05T11:44:03.916 INFO:teuthology.orchestra.run.smithi059.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-05T11:44:03.929 INFO:teuthology.orchestra.run.smithi189.stdout:(36/154): ceph-mgr-diskprediction-local-19.1.1- 8.6 MB/s | 7.4 MB 00:00 2024-09-05T11:44:03.941 INFO:teuthology.orchestra.run.smithi059.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-05T11:44:03.953 INFO:teuthology.orchestra.run.smithi089.stdout:(149/154): librados2-19.1.1-261.ge856d6b2.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-05T11:44:03.954 INFO:teuthology.orchestra.run.smithi189.stdout:(37/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 1.9 MB/s | 285 kB 00:00 2024-09-05T11:44:03.967 INFO:teuthology.orchestra.run.smithi059.stdout:(65/154): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-05T11:44:03.980 INFO:teuthology.orchestra.run.smithi189.stdout:(38/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 365 kB/s | 18 kB 00:00 2024-09-05T11:44:04.003 INFO:teuthology.orchestra.run.smithi089.stdout:(150/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-05T11:44:04.025 INFO:teuthology.orchestra.run.smithi059.stdout:(66/154): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-05T11:44:04.055 INFO:teuthology.orchestra.run.smithi189.stdout:(39/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 3.3 MB/s | 765 kB 00:00 2024-09-05T11:44:04.243 INFO:teuthology.orchestra.run.smithi059.stdout:(67/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-05T11:44:04.334 INFO:teuthology.orchestra.run.smithi059.stdout:(68/154): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-05T11:44:04.368 INFO:teuthology.orchestra.run.smithi059.stdout:(69/154): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-05T11:44:04.401 INFO:teuthology.orchestra.run.smithi059.stdout:(70/154): postgresql-contrib-13.16-1.el9.x86_64 5.6 MB/s | 910 kB 00:00 2024-09-05T11:44:04.427 INFO:teuthology.orchestra.run.smithi059.stdout:(71/154): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-05T11:44:04.493 INFO:teuthology.orchestra.run.smithi059.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-05T11:44:04.527 INFO:teuthology.orchestra.run.smithi059.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-05T11:44:04.629 INFO:teuthology.orchestra.run.smithi089.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 65 kB/s | 40 kB 00:00 2024-09-05T11:44:04.647 INFO:teuthology.orchestra.run.smithi189.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 115 kB/s | 80 kB 00:00 2024-09-05T11:44:04.652 INFO:teuthology.orchestra.run.smithi059.stdout:(74/154): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-05T11:44:04.694 INFO:teuthology.orchestra.run.smithi059.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 367 kB/s | 61 kB 00:00 2024-09-05T11:44:04.745 INFO:teuthology.orchestra.run.smithi059.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-05T11:44:04.778 INFO:teuthology.orchestra.run.smithi059.stdout:(77/154): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-05T11:44:04.789 INFO:teuthology.orchestra.run.smithi189.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 260 kB/s | 191 kB 00:00 2024-09-05T11:44:04.829 INFO:teuthology.orchestra.run.smithi089.stdout:(152/154): NetworkManager-1.48.10-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-05T11:44:04.870 INFO:teuthology.orchestra.run.smithi059.stdout:(78/154): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-05T11:44:04.906 INFO:teuthology.orchestra.run.smithi189.stdout:(42/154): libgfortran-11.5.0-2.el9.x86_64.rpm 866 kB/s | 801 kB 00:00 2024-09-05T11:44:04.912 INFO:teuthology.orchestra.run.smithi059.stdout:(79/154): python3-jsonpatch-1.21-16.el9.noarch. 153 kB/s | 26 kB 00:00 2024-09-05T11:44:04.954 INFO:teuthology.orchestra.run.smithi059.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 19 kB 00:00 2024-09-05T11:44:04.979 INFO:teuthology.orchestra.run.smithi059.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-05T11:44:05.038 INFO:teuthology.orchestra.run.smithi089.stdout:(153/154): NetworkManager-libnm-1.48.10-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-05T11:44:05.062 INFO:teuthology.orchestra.run.smithi059.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-05T11:44:05.104 INFO:teuthology.orchestra.run.smithi059.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-05T11:44:05.205 INFO:teuthology.orchestra.run.smithi059.stdout:(84/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-05T11:44:05.230 INFO:teuthology.orchestra.run.smithi059.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-09-05T11:44:05.255 INFO:teuthology.orchestra.run.smithi059.stdout:(86/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-09-05T11:44:05.297 INFO:teuthology.orchestra.run.smithi059.stdout:(87/154): python3-prettytable-0.7.2-27.el9.noar 459 kB/s | 42 kB 00:00 2024-09-05T11:44:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:(88/154): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 269 kB 00:00 2024-09-05T11:44:05.348 INFO:teuthology.orchestra.run.smithi059.stdout:(89/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-05T11:44:05.382 INFO:teuthology.orchestra.run.smithi059.stdout:(90/154): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-05T11:44:05.406 INFO:teuthology.orchestra.run.smithi189.stdout:(43/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-05T11:44:05.440 INFO:teuthology.orchestra.run.smithi059.stdout:(91/154): python3-pytz-2021.1-5.el9.noarch.rpm 434 kB/s | 51 kB 00:00 2024-09-05T11:44:05.479 INFO:teuthology.orchestra.run.smithi089.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 292 kB/s | 248 kB 00:00 2024-09-05T11:44:05.486 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-05T11:44:05.486 INFO:teuthology.orchestra.run.smithi089.stdout:Total 17 MB/s | 222 MB 00:13 2024-09-05T11:44:05.490 INFO:teuthology.orchestra.run.smithi059.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 376 kB/s | 54 kB 00:00 2024-09-05T11:44:05.549 INFO:teuthology.orchestra.run.smithi059.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-05T11:44:05.573 INFO:teuthology.orchestra.run.smithi189.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-05T11:44:05.600 INFO:teuthology.orchestra.run.smithi059.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-05T11:44:05.625 INFO:teuthology.orchestra.run.smithi059.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-05T11:44:05.692 INFO:teuthology.orchestra.run.smithi059.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-09-05T11:44:05.723 INFO:teuthology.orchestra.run.smithi189.stdout:(45/154): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-05T11:44:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-05T11:44:05.767 INFO:teuthology.orchestra.run.smithi059.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-05T11:44:05.842 INFO:teuthology.orchestra.run.smithi059.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-09-05T11:44:05.868 INFO:teuthology.orchestra.run.smithi059.stdout:(100/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 956 kB/s | 96 kB 00:00 2024-09-05T11:44:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-09-05T11:44:06.018 INFO:teuthology.orchestra.run.smithi059.stdout:(102/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 298 kB/s | 45 kB 00:00 2024-09-05T11:44:06.185 INFO:teuthology.orchestra.run.smithi059.stdout:(103/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-05T11:44:06.190 INFO:teuthology.orchestra.run.smithi189.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 172 kB/s | 135 kB 00:00 2024-09-05T11:44:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:(104/154): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-09-05T11:44:06.327 INFO:teuthology.orchestra.run.smithi059.stdout:(105/154): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-09-05T11:44:06.378 INFO:teuthology.orchestra.run.smithi059.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 404 kB/s | 19 kB 00:00 2024-09-05T11:44:06.381 INFO:teuthology.orchestra.run.smithi189.stdout:(47/154): python3-requests-2.25.1-8.el9.noarch. 154 kB/s | 125 kB 00:00 2024-09-05T11:44:06.411 INFO:teuthology.orchestra.run.smithi059.stdout:(107/154): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-09-05T11:44:06.462 INFO:teuthology.orchestra.run.smithi059.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 525 kB/s | 26 kB 00:00 2024-09-05T11:44:06.512 INFO:teuthology.orchestra.run.smithi059.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 976 kB/s | 49 kB 00:00 2024-09-05T11:44:06.540 INFO:teuthology.orchestra.run.smithi189.stdout:(48/154): python3-urllib3-1.26.5-6.el9.noarch.r 263 kB/s | 215 kB 00:00 2024-09-05T11:44:06.545 INFO:teuthology.orchestra.run.smithi059.stdout:(110/154): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-09-05T11:44:06.588 INFO:teuthology.orchestra.run.smithi059.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2024-09-05T11:44:06.630 INFO:teuthology.orchestra.run.smithi059.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-05T11:44:06.638 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-05T11:44:06.663 INFO:teuthology.orchestra.run.smithi059.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-05T11:44:06.713 INFO:teuthology.orchestra.run.smithi059.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-09-05T11:44:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 650 kB/s | 32 kB 00:00 2024-09-05T11:44:06.775 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-05T11:44:06.775 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-05T11:44:06.814 INFO:teuthology.orchestra.run.smithi059.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-05T11:44:06.839 INFO:teuthology.orchestra.run.smithi059.stdout:(117/154): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-05T11:44:06.893 INFO:teuthology.orchestra.run.smithi059.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.1 MB/s | 172 kB 00:00 2024-09-05T11:44:06.932 INFO:teuthology.orchestra.run.smithi059.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-05T11:44:06.965 INFO:teuthology.orchestra.run.smithi059.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.2 MB/s | 303 kB 00:00 2024-09-05T11:44:06.974 INFO:teuthology.orchestra.run.smithi189.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-09-05T11:44:06.999 INFO:teuthology.orchestra.run.smithi059.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-05T11:44:06.999 INFO:teuthology.orchestra.run.smithi189.stdout:(50/154): automake-1.16.2-8.el9.noarch.rpm 852 kB/s | 689 kB 00:00 2024-09-05T11:44:07.032 INFO:teuthology.orchestra.run.smithi059.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-05T11:44:07.075 INFO:teuthology.orchestra.run.smithi189.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 217 kB/s | 22 kB 00:00 2024-09-05T11:44:07.075 INFO:teuthology.orchestra.run.smithi059.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-09-05T11:44:07.108 INFO:teuthology.orchestra.run.smithi189.stdout:(52/154): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-05T11:44:07.116 INFO:teuthology.orchestra.run.smithi059.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 426 kB/s | 18 kB 00:00 2024-09-05T11:44:07.158 INFO:teuthology.orchestra.run.smithi059.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-09-05T11:44:07.166 INFO:teuthology.orchestra.run.smithi189.stdout:(53/154): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-05T11:44:07.192 INFO:teuthology.orchestra.run.smithi189.stdout:(54/154): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-05T11:44:07.200 INFO:teuthology.orchestra.run.smithi059.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-09-05T11:44:07.242 INFO:teuthology.orchestra.run.smithi059.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 476 kB/s | 20 kB 00:00 2024-09-05T11:44:07.267 INFO:teuthology.orchestra.run.smithi189.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-09-05T11:44:07.309 INFO:teuthology.orchestra.run.smithi189.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-05T11:44:07.309 INFO:teuthology.orchestra.run.smithi059.stdout:(128/154): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-09-05T11:44:07.351 INFO:teuthology.orchestra.run.smithi059.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-09-05T11:44:07.376 INFO:teuthology.orchestra.run.smithi189.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-05T11:44:07.376 INFO:teuthology.orchestra.run.smithi059.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T11:44:07.418 INFO:teuthology.orchestra.run.smithi189.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-05T11:44:07.418 INFO:teuthology.orchestra.run.smithi059.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-09-05T11:44:07.477 INFO:teuthology.orchestra.run.smithi189.stdout:(59/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.1 MB/s | 3.0 MB 00:00 2024-09-05T11:44:07.477 INFO:teuthology.orchestra.run.smithi059.stdout:(132/154): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-05T11:44:07.519 INFO:teuthology.orchestra.run.smithi059.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-09-05T11:44:07.577 INFO:teuthology.orchestra.run.smithi189.stdout:(60/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 245 kB/s | 24 kB 00:00 2024-09-05T11:44:07.594 INFO:teuthology.orchestra.run.smithi059.stdout:(134/154): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-05T11:44:07.602 INFO:teuthology.orchestra.run.smithi189.stdout:(61/154): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-05T11:44:07.620 INFO:teuthology.orchestra.run.smithi059.stdout:(135/154): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-05T11:44:07.661 INFO:teuthology.orchestra.run.smithi059.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-09-05T11:44:07.687 INFO:teuthology.orchestra.run.smithi059.stdout:(137/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 538 kB/s | 90 kB 00:00 2024-09-05T11:44:07.694 INFO:teuthology.orchestra.run.smithi189.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-05T11:44:07.719 INFO:teuthology.orchestra.run.smithi189.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-05T11:44:07.729 INFO:teuthology.orchestra.run.smithi059.stdout:(138/154): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-09-05T11:44:07.796 INFO:teuthology.orchestra.run.smithi059.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-09-05T11:44:07.803 INFO:teuthology.orchestra.run.smithi189.stdout:(64/154): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-05T11:44:07.820 INFO:teuthology.orchestra.run.smithi189.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-05T11:44:07.837 INFO:teuthology.orchestra.run.smithi059.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-09-05T11:44:07.863 INFO:teuthology.orchestra.run.smithi059.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-05T11:44:07.870 INFO:teuthology.orchestra.run.smithi189.stdout:(66/154): libtool-2.4.6-46.el9.x86_64.rpm 1.3 MB/s | 581 kB 00:00 2024-09-05T11:44:07.888 INFO:teuthology.orchestra.run.smithi059.stdout:(142/154): python3-routes-2.5.1-5.el9.noarch.rp 833 kB/s | 188 kB 00:00 2024-09-05T11:44:07.914 INFO:teuthology.orchestra.run.smithi059.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T11:44:07.939 INFO:teuthology.orchestra.run.smithi059.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-05T11:44:07.980 INFO:teuthology.orchestra.run.smithi059.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-09-05T11:44:08.006 INFO:teuthology.orchestra.run.smithi059.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-05T11:44:08.047 INFO:teuthology.orchestra.run.smithi059.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-05T11:44:08.105 INFO:teuthology.orchestra.run.smithi059.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-05T11:44:08.229 INFO:teuthology.orchestra.run.smithi189.stdout:(67/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-05T11:44:08.304 INFO:teuthology.orchestra.run.smithi189.stdout:(68/154): openblas-openmp-0.3.26-2.el9.x86_64.r 9.9 MB/s | 4.9 MB 00:00 2024-09-05T11:44:08.337 INFO:teuthology.orchestra.run.smithi189.stdout:(69/154): postgresql-13.16-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-09-05T11:44:08.347 INFO:teuthology.orchestra.run.smithi059.stdout:(149/154): librados2-19.1.1-261.ge856d6b2.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-05T11:44:08.371 INFO:teuthology.orchestra.run.smithi189.stdout:(70/154): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-05T11:44:08.406 INFO:teuthology.orchestra.run.smithi059.stdout:(150/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-05T11:44:08.413 INFO:teuthology.orchestra.run.smithi189.stdout:(71/154): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-05T11:44:08.490 INFO:teuthology.orchestra.run.smithi059.stdout:(151/154): NetworkManager-1.48.10-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-09-05T11:44:08.496 INFO:teuthology.orchestra.run.smithi189.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-05T11:44:08.522 INFO:teuthology.orchestra.run.smithi189.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-09-05T11:44:08.531 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-05T11:44:08.532 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-05T11:44:08.548 INFO:teuthology.orchestra.run.smithi059.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-09-05T11:44:08.573 INFO:teuthology.orchestra.run.smithi059.stdout:(153/154): NetworkManager-team-1.48.10-1.el9.x8 242 kB/s | 40 kB 00:00 2024-09-05T11:44:08.638 INFO:teuthology.orchestra.run.smithi189.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 527 kB/s | 61 kB 00:00 2024-09-05T11:44:08.723 INFO:teuthology.orchestra.run.smithi059.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 4.8 MB/s | 1.8 MB 00:00 2024-09-05T11:44:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-09-05T11:44:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:Total 20 MB/s | 222 MB 00:11 2024-09-05T11:44:08.747 INFO:teuthology.orchestra.run.smithi189.stdout:(75/154): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-05T11:44:08.855 INFO:teuthology.orchestra.run.smithi189.stdout:(76/154): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-05T11:44:08.955 INFO:teuthology.orchestra.run.smithi189.stdout:(77/154): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-05T11:44:09.031 INFO:teuthology.orchestra.run.smithi189.stdout:(78/154): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-05T11:44:09.056 INFO:teuthology.orchestra.run.smithi189.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-05T11:44:09.140 INFO:teuthology.orchestra.run.smithi189.stdout:(80/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-05T11:44:09.165 INFO:teuthology.orchestra.run.smithi189.stdout:(81/154): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-05T11:44:09.282 INFO:teuthology.orchestra.run.smithi189.stdout:(82/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-05T11:44:09.390 INFO:teuthology.orchestra.run.smithi189.stdout:(83/154): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-05T11:44:09.499 INFO:teuthology.orchestra.run.smithi189.stdout:(84/154): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-09-05T11:44:09.566 INFO:teuthology.orchestra.run.smithi189.stdout:(85/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-05T11:44:09.599 INFO:teuthology.orchestra.run.smithi189.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-09-05T11:44:09.675 INFO:teuthology.orchestra.run.smithi189.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-05T11:44:09.747 INFO:teuthology.orchestra.run.smithi189.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-05T11:44:09.784 INFO:teuthology.orchestra.run.smithi189.stdout:(89/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-05T11:44:09.851 INFO:teuthology.orchestra.run.smithi189.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 494 kB/s | 51 kB 00:00 2024-09-05T11:44:09.893 INFO:teuthology.orchestra.run.smithi189.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-09-05T11:44:10.001 INFO:teuthology.orchestra.run.smithi189.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-05T11:44:10.102 INFO:teuthology.orchestra.run.smithi189.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-05T11:44:10.219 INFO:teuthology.orchestra.run.smithi189.stdout:(94/154): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-09-05T11:44:10.328 INFO:teuthology.orchestra.run.smithi189.stdout:(95/154): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-09-05T11:44:10.427 INFO:teuthology.orchestra.run.smithi189.stdout:(96/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-09-05T11:44:10.544 INFO:teuthology.orchestra.run.smithi189.stdout:(97/154): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-05T11:44:10.653 INFO:teuthology.orchestra.run.smithi189.stdout:(98/154): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-05T11:44:10.733 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-09-05T11:44:10.762 INFO:teuthology.orchestra.run.smithi189.stdout:(99/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 884 kB/s | 96 kB 00:00 2024-09-05T11:44:10.878 INFO:teuthology.orchestra.run.smithi189.stdout:(100/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 546 kB/s | 64 kB 00:00 2024-09-05T11:44:10.910 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-09-05T11:44:10.910 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-09-05T11:44:10.987 INFO:teuthology.orchestra.run.smithi189.stdout:(101/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-09-05T11:44:11.171 INFO:teuthology.orchestra.run.smithi189.stdout:(102/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-05T11:44:11.254 INFO:teuthology.orchestra.run.smithi189.stdout:(103/154): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-05T11:44:11.330 INFO:teuthology.orchestra.run.smithi189.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-05T11:44:11.355 INFO:teuthology.orchestra.run.smithi189.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-05T11:44:11.380 INFO:teuthology.orchestra.run.smithi189.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-05T11:44:11.489 INFO:teuthology.orchestra.run.smithi189.stdout:(107/154): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-05T11:44:11.522 INFO:teuthology.orchestra.run.smithi189.stdout:(108/154): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-05T11:44:11.539 INFO:teuthology.orchestra.run.smithi189.stdout:(109/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-05T11:44:11.573 INFO:teuthology.orchestra.run.smithi189.stdout:(110/154): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-05T11:44:11.598 INFO:teuthology.orchestra.run.smithi189.stdout:(111/154): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-05T11:44:11.615 INFO:teuthology.orchestra.run.smithi189.stdout:(112/154): liboath-2.6.7-2.el9.x86_64.rpm 525 kB/s | 49 kB 00:00 2024-09-05T11:44:11.648 INFO:teuthology.orchestra.run.smithi189.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-05T11:44:11.673 INFO:teuthology.orchestra.run.smithi189.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T11:44:11.699 INFO:teuthology.orchestra.run.smithi189.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T11:44:11.732 INFO:teuthology.orchestra.run.smithi189.stdout:(116/154): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-05T11:44:11.758 INFO:teuthology.orchestra.run.smithi189.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-05T11:44:11.783 INFO:teuthology.orchestra.run.smithi189.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-05T11:44:11.808 INFO:teuthology.orchestra.run.smithi189.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-05T11:44:11.842 INFO:teuthology.orchestra.run.smithi189.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-05T11:44:11.867 INFO:teuthology.orchestra.run.smithi189.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-05T11:44:11.892 INFO:teuthology.orchestra.run.smithi189.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-05T11:44:11.926 INFO:teuthology.orchestra.run.smithi189.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 318 kB/s | 11 kB 00:00 2024-09-05T11:44:11.952 INFO:teuthology.orchestra.run.smithi189.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-05T11:44:11.977 INFO:teuthology.orchestra.run.smithi189.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-05T11:44:12.002 INFO:teuthology.orchestra.run.smithi189.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-05T11:44:12.028 INFO:teuthology.orchestra.run.smithi189.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-05T11:44:12.078 INFO:teuthology.orchestra.run.smithi189.stdout:(128/154): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-05T11:44:12.103 INFO:teuthology.orchestra.run.smithi189.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T11:44:12.128 INFO:teuthology.orchestra.run.smithi189.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T11:44:12.153 INFO:teuthology.orchestra.run.smithi189.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T11:44:12.187 INFO:teuthology.orchestra.run.smithi189.stdout:(132/154): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-05T11:44:12.212 INFO:teuthology.orchestra.run.smithi189.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-05T11:44:12.238 INFO:teuthology.orchestra.run.smithi189.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-05T11:44:12.263 INFO:teuthology.orchestra.run.smithi189.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-05T11:44:12.288 INFO:teuthology.orchestra.run.smithi189.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-05T11:44:12.314 INFO:teuthology.orchestra.run.smithi189.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-05T11:44:12.338 INFO:teuthology.orchestra.run.smithi189.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-05T11:44:12.364 INFO:teuthology.orchestra.run.smithi189.stdout:(139/154): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-05T11:44:12.389 INFO:teuthology.orchestra.run.smithi189.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-05T11:44:12.414 INFO:teuthology.orchestra.run.smithi189.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-05T11:44:12.440 INFO:teuthology.orchestra.run.smithi189.stdout:(142/154): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-05T11:44:12.465 INFO:teuthology.orchestra.run.smithi189.stdout:(143/154): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-05T11:44:12.490 INFO:teuthology.orchestra.run.smithi189.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-05T11:44:12.515 INFO:teuthology.orchestra.run.smithi189.stdout:(145/154): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-05T11:44:12.541 INFO:teuthology.orchestra.run.smithi189.stdout:(146/154): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-09-05T11:44:12.591 INFO:teuthology.orchestra.run.smithi189.stdout:(147/154): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-05T11:44:12.850 INFO:teuthology.orchestra.run.smithi189.stdout:(148/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-05T11:44:13.025 INFO:teuthology.orchestra.run.smithi189.stdout:(149/154): librados2-19.1.1-261.ge856d6b2.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-05T11:44:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-09-05T11:44:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-09-05T11:44:13.892 INFO:teuthology.orchestra.run.smithi189.stdout:(150/154): NetworkManager-1.48.10-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-05T11:44:14.009 INFO:teuthology.orchestra.run.smithi189.stdout:(151/154): NetworkManager-libnm-1.48.10-1.el9.x 1.8 MB/s | 1.8 MB 00:00 2024-09-05T11:44:14.534 INFO:teuthology.orchestra.run.smithi189.stdout:(152/154): postgresql-server-13.16-1.el9.x86_64 954 kB/s | 5.8 MB 00:06 2024-09-05T11:44:14.550 INFO:teuthology.orchestra.run.smithi189.stdout:(153/154): NetworkManager-team-1.48.10-1.el9.x8 61 kB/s | 40 kB 00:00 2024-09-05T11:44:14.878 INFO:teuthology.orchestra.run.smithi189.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 285 kB/s | 248 kB 00:00 2024-09-05T11:44:14.884 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-05T11:44:14.885 INFO:teuthology.orchestra.run.smithi189.stdout:Total 12 MB/s | 222 MB 00:18 2024-09-05T11:44:14.965 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-05T11:44:15.041 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T11:44:15.074 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T11:44:15.117 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T11:44:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T11:44:15.584 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:15.689 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:15.718 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:15.793 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:15.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:44:15.881 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T11:44:15.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T11:44:16.061 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T11:44:16.068 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-05T11:44:16.113 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T11:44:16.150 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:16.204 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-05T11:44:16.204 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-05T11:44:16.262 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:16.284 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:16.347 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:16.368 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:16.392 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:16.537 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:16.682 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:16.808 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:16.868 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:16.965 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:17.045 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:17.276 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:17.388 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T11:44:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T11:44:17.517 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T11:44:17.561 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T11:44:17.636 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T11:44:17.696 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T11:44:17.735 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T11:44:17.779 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T11:44:17.812 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T11:44:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T11:44:17.910 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T11:44:17.953 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-05T11:44:17.954 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-05T11:44:17.962 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T11:44:18.001 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T11:44:18.049 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T11:44:18.128 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T11:44:18.160 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T11:44:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:44:18.248 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T11:44:18.320 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T11:44:18.363 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T11:44:18.465 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T11:44:18.606 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T11:44:18.658 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T11:44:19.189 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T11:44:19.270 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T11:44:19.299 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T11:44:19.342 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T11:44:19.380 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T11:44:19.413 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T11:44:19.477 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T11:44:19.560 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T11:44:19.599 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T11:44:19.635 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T11:44:19.681 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T11:44:19.717 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T11:44:19.761 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T11:44:19.803 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T11:44:19.923 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-09-05T11:44:19.949 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T11:44:19.995 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T11:44:20.005 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T11:44:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T11:44:20.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T11:44:20.114 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T11:44:20.140 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T11:44:20.187 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T11:44:20.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T11:44:20.253 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T11:44:20.278 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T11:44:20.353 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T11:44:20.490 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T11:44:20.510 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T11:44:20.527 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T11:44:20.555 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T11:44:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:20.759 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:20.787 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:20.808 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T11:44:20.902 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:20.947 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T11:44:20.990 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T11:44:21.030 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T11:44:21.067 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T11:44:21.086 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:44:21.118 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T11:44:21.119 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T11:44:21.152 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T11:44:21.185 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T11:44:21.195 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T11:44:21.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T11:44:21.321 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T11:44:21.368 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T11:44:21.401 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T11:44:21.434 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T11:44:21.491 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T11:44:21.507 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T11:44:21.517 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:21.568 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T11:44:21.653 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:21.685 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:21.768 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:21.794 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:21.834 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:22.072 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:22.313 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T11:44:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:22.357 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T11:44:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T11:44:22.469 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:22.507 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T11:44:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T11:44:22.608 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T11:44:22.665 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:22.704 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T11:44:22.744 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T11:44:22.786 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:22.928 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T11:44:22.958 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T11:44:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T11:44:23.077 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T11:44:23.360 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T11:44:23.409 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T11:44:23.454 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T11:44:23.540 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T11:44:23.608 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T11:44:23.646 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T11:44:23.685 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T11:44:23.700 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T11:44:23.738 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T11:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T11:44:23.791 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T11:44:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T11:44:23.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T11:44:24.009 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T11:44:24.056 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T11:44:24.148 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T11:44:24.187 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T11:44:24.222 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-05T11:44:24.232 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:44:24.259 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T11:44:24.303 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T11:44:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T11:44:24.355 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T11:44:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T11:44:24.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T11:44:24.517 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T11:44:24.693 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T11:44:24.701 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T11:44:24.759 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T11:44:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:24.934 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T11:44:24.963 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:25.033 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T11:44:25.055 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:44:25.097 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:44:25.123 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T11:44:25.125 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T11:44:25.151 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T11:44:25.151 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T11:44:25.184 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T11:44:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T11:44:25.357 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T11:44:25.385 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:25.475 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T11:44:25.480 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:44:25.503 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:25.503 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T11:44:25.556 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:25.565 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T11:44:25.577 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T11:44:25.586 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:25.611 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T11:44:25.617 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T11:44:25.647 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T11:44:25.686 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T11:44:25.717 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T11:44:25.722 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T11:44:25.766 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T11:44:25.805 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:25.829 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T11:44:25.916 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T11:44:25.959 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T11:44:25.975 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T11:44:25.996 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T11:44:26.055 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T11:44:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:26.095 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T11:44:26.119 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T11:44:26.141 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T11:44:26.191 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T11:44:26.203 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T11:44:26.227 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:26.234 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:26.303 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:26.305 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:26.336 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T11:44:26.522 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T11:44:26.547 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T11:44:26.617 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T11:44:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T11:44:26.667 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T11:44:26.751 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T11:44:26.783 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T11:44:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T11:44:26.819 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T11:44:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T11:44:26.883 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T11:44:26.894 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T11:44:26.912 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T11:44:26.936 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T11:44:26.945 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T11:44:26.985 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T11:44:27.028 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T11:44:27.059 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T11:44:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T11:44:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T11:44:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T11:44:27.212 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T11:44:27.237 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T11:44:27.241 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T11:44:27.280 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T11:44:27.286 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T11:44:27.318 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T11:44:27.360 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T11:44:27.362 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T11:44:27.395 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T11:44:27.416 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T11:44:27.438 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:44:27.461 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T11:44:27.467 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T11:44:27.516 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T11:44:27.538 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T11:44:27.553 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T11:44:27.582 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T11:44:27.587 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T11:44:27.621 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T11:44:27.658 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T11:44:27.665 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T11:44:27.696 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T11:44:27.795 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T11:44:27.801 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T11:44:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T11:44:27.865 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T11:44:27.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T11:44:28.031 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T11:44:28.090 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T11:44:28.398 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T11:44:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T11:44:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T11:44:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T11:44:28.601 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T11:44:28.634 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T11:44:28.668 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:28.690 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T11:44:28.701 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:28.765 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T11:44:28.810 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T11:44:28.846 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T11:44:28.894 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T11:44:28.929 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T11:44:28.973 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T11:44:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T11:44:29.061 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T11:44:29.116 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T11:44:29.148 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T11:44:29.156 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T11:44:29.199 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T11:44:29.240 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T11:44:29.244 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:29.269 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T11:44:29.280 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:29.333 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T11:44:29.355 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T11:44:29.374 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:29.374 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T11:44:29.374 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T11:44:29.374 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:29.393 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T11:44:29.399 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T11:44:29.415 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:29.439 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T11:44:29.464 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T11:44:29.489 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T11:44:29.502 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T11:44:29.540 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T11:44:29.565 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T11:44:29.693 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T11:44:29.787 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T11:44:29.838 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T11:44:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T11:44:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T11:44:29.935 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T11:44:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T11:44:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T11:44:30.049 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T11:44:30.089 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T11:44:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T11:44:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T11:44:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T11:44:30.236 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T11:44:30.269 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T11:44:30.362 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T11:44:30.417 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T11:44:30.447 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T11:44:30.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T11:44:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T11:44:30.813 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T11:44:30.957 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T11:44:31.358 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T11:44:31.404 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T11:44:31.429 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T11:44:31.533 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T11:44:31.592 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T11:44:31.637 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T11:44:31.727 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T11:44:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T11:44:31.959 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T11:44:31.987 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T11:44:32.064 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T11:44:32.665 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T11:44:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T11:44:32.758 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T11:44:32.795 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T11:44:32.827 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T11:44:32.854 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T11:44:33.206 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T11:44:33.238 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:33.360 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:33.394 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T11:44:33.443 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T11:44:33.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T11:44:34.033 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:34.124 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:34.125 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T11:44:34.151 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T11:44:34.209 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T11:44:34.236 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T11:44:34.273 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T11:44:34.581 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T11:44:34.615 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T11:44:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T11:44:34.781 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T11:44:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T11:44:35.293 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:35.587 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T11:44:35.719 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T11:44:36.761 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:36.806 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:37.497 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:37.542 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:37.675 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:37.675 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T11:44:37.675 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T11:44:37.676 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:37.998 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:38.032 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:38.578 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T11:44:38.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:38.713 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T11:44:38.713 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T11:44:38.713 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T11:44:38.713 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:38.745 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:41.453 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-09-05T11:44:41.454 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:41.622 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:41.638 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:41.638 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T11:44:41.638 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:43.099 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:43.130 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:43.130 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:43.130 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T11:44:43.130 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:43.131 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:43.131 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:43.360 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:43.383 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:43.383 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:43.384 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T11:44:43.384 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:43.384 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:43.384 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:43.852 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:43.959 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T11:44:44.015 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T11:44:44.070 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T11:44:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T11:44:44.313 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T11:44:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T11:44:44.479 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T11:44:44.626 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T11:44:44.664 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T11:44:44.695 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T11:44:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T11:44:45.145 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:45.173 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:45.204 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T11:44:45.266 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T11:44:45.296 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:46.336 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:46.373 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:47.334 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:47.365 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:47.467 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:47.627 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T11:44:47.667 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:47.691 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:47.691 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T11:44:47.692 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:47.692 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:47.692 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:47.752 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:47.763 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:47.861 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T11:44:47.909 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T11:44:48.834 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T11:44:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:48.895 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:48.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:44:48.961 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:44:48.961 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:48.962 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T11:44:48.962 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:49.006 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:44:49.029 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:44:49.030 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:49.030 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T11:44:49.030 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:44:49.030 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:44:49.030 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:49.332 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:44:49.355 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:44:49.355 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:49.355 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T11:44:49.355 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:44:49.356 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:44:49.356 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:44:50.535 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:50.535 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-05T11:44:50.535 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-05T11:44:50.535 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-05T11:44:50.536 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-05T11:44:50.536 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-05T11:44:50.536 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-05T11:44:50.536 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-05T11:44:50.536 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:50.731 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:50.731 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T11:44:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-09-05T11:44:50.985 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-09-05T11:44:50.986 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:51.204 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:51.251 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T11:44:51.251 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:51.252 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T11:44:51.252 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:52.225 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:52.252 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:52.252 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:52.252 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T11:44:52.252 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:52.253 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:52.253 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:52.484 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:52.508 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:52.916 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:52.943 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:52.967 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:52.974 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T11:44:52.974 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:52.975 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T11:44:52.975 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:52.975 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T11:44:52.975 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:53.073 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T11:44:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T11:44:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T11:44:53.242 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T11:44:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:53.372 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T11:44:53.418 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T11:44:53.496 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T11:44:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T11:44:53.732 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T11:44:53.768 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T11:44:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:53.799 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T11:44:53.836 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T11:44:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T11:44:53.844 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:53.844 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T11:44:53.844 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:53.844 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T11:44:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:53.895 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T11:44:53.944 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T11:44:53.985 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T11:44:53.985 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T11:44:54.017 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T11:44:54.045 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T11:44:54.047 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T11:44:54.120 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T11:44:54.226 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T11:44:54.240 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:54.268 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:54.276 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T11:44:54.299 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T11:44:54.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T11:44:54.335 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T11:44:54.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T11:44:54.361 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T11:44:54.382 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T11:44:54.390 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:54.422 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T11:44:54.512 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T11:44:54.631 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T11:44:54.809 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T11:44:54.862 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T11:44:54.902 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T11:44:54.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T11:44:54.988 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:44:55.022 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:44:55.043 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:44:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T11:44:55.476 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T11:44:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T11:44:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:55.880 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T11:44:55.962 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T11:44:56.003 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T11:44:56.085 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T11:44:56.115 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T11:44:56.141 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T11:44:56.142 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:44:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T11:44:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:44:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:44:56.436 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:44:56.436 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:44:56.444 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:56.476 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:56.512 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:44:56.513 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:44:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T11:44:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:56.812 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:56.812 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:56.813 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T11:44:56.813 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:56.813 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:56.813 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:56.873 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T11:44:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:56.971 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T11:44:57.018 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T11:44:57.947 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T11:44:57.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:58.005 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:58.005 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:58.005 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T11:44:58.005 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:58.006 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:58.006 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:58.049 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:44:58.072 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:44:58.072 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:58.072 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T11:44:58.072 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:58.074 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T11:44:58.100 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:58.124 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:44:58.148 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:58.245 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T11:44:58.423 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T11:44:58.449 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:44:58.461 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:58.473 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:44:58.473 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:58.474 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T11:44:58.474 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:44:58.474 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:44:58.474 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:44:58.513 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T11:44:58.513 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:58.513 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T11:44:58.513 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:58.513 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T11:44:58.514 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:44:58.570 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:58.597 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T11:44:58.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T11:44:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T11:44:59.852 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T11:44:59.914 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T11:44:59.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:00.007 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:45:00.057 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T11:45:00.057 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:45:00.057 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T11:45:00.057 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:00.115 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T11:45:00.167 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:00.524 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:45:00.576 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T11:45:00.576 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T11:45:00.576 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T11:45:00.576 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:45:00.576 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T11:45:00.577 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T11:45:02.340 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T11:45:02.341 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T11:45:02.342 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T11:45:02.343 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T11:45:02.343 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T11:45:02.355 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T11:45:02.356 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T11:45:02.357 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T11:45:02.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T11:45:02.359 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T11:45:02.360 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T11:45:02.361 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T11:45:02.362 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T11:45:02.363 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T11:45:02.364 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T11:45:02.365 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T11:45:02.366 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T11:45:02.366 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T11:45:02.366 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T11:45:02.366 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:02.366 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T11:45:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T11:45:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T11:45:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T11:45:03.160 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T11:45:03.186 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T11:45:03.372 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T11:45:03.424 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T11:45:03.454 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T11:45:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T11:45:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T11:45:04.088 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:04.127 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:04.148 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:04.968 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T11:45:04.969 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.970 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.971 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T11:45:04.972 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T11:45:04.973 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T11:45:04.974 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T11:45:04.975 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T11:45:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T11:45:04.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T11:45:04.978 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T11:45:04.979 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T11:45:04.980 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.981 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T11:45:04.982 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T11:45:04.983 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:45:04.984 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-05T11:45:05.004 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T11:45:05.090 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T11:45:05.131 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T11:45:05.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T11:45:05.243 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T11:45:05.269 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T11:45:05.269 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:45:05.353 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:45:05.378 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T11:45:05.384 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:05.384 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:05.388 DEBUG:teuthology.parallel:result is None 2024-09-05T11:45:05.563 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:05.563 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:45:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:45:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:45:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T11:45:05.846 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T11:45:05.890 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T11:45:05.927 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T11:45:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T11:45:06.219 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T11:45:06.277 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T11:45:06.306 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T11:45:06.338 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T11:45:06.381 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T11:45:07.117 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:07.170 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:07.212 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T11:45:08.578 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T11:45:08.683 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T11:45:08.739 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T11:45:08.851 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T11:45:08.893 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T11:45:08.922 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T11:45:08.922 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:45:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T11:45:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T11:45:09.086 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:09.086 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:09.331 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:09.331 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:45:09.440 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T11:45:09.440 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:45:09.776 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:45:09.776 INFO:teuthology.orchestra.run.smithi059.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-05T11:45:09.777 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T11:45:11.792 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T11:45:11.793 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T11:45:11.794 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T11:45:11.795 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T11:45:11.796 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T11:45:11.797 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T11:45:11.798 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T11:45:11.799 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T11:45:11.800 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T11:45:11.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T11:45:11.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T11:45:11.803 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:11.804 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T11:45:14.417 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T11:45:14.417 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:45:14.417 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T11:45:14.418 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.419 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.420 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T11:45:14.421 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.422 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.423 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T11:45:14.424 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.425 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T11:45:14.426 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:14.427 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T11:45:14.428 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T11:45:14.429 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T11:45:14.430 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:45:14.431 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-05T11:45:14.802 DEBUG:teuthology.parallel:result is None 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T11:45:15.250 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T11:45:15.251 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T11:45:15.252 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T11:45:15.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T11:45:15.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T11:45:15.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T11:45:15.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T11:45:15.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T11:45:15.270 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T11:45:15.271 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T11:45:15.272 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T11:45:15.273 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T11:45:15.274 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T11:45:15.275 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T11:45:15.276 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T11:45:15.277 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T11:45:15.278 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T11:45:17.473 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:17.474 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.475 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T11:45:17.477 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T11:45:17.478 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T11:45:17.479 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T11:45:17.480 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T11:45:17.481 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T11:45:17.482 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.483 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T11:45:17.484 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T11:45:17.485 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T11:45:17.486 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T11:45:17.488 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.489 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T11:45:17.490 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T11:45:17.491 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:45:17.491 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-09-05T11:45:18.042 DEBUG:teuthology.parallel:result is None 2024-09-05T11:45:18.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:18.189 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T11:45:18.233 INFO:teuthology.orchestra.run.smithi059.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.234 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.234 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T11:45:18.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:18.379 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T11:45:18.405 INFO:teuthology.orchestra.run.smithi089.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.406 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.406 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T11:45:18.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:18.551 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T11:45:18.577 INFO:teuthology.orchestra.run.smithi189.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.578 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T11:45:18.578 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T11:45:18.580 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T11:45:18.580 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:45:18.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T11:45:18.646 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:45:18.646 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T11:45:18.679 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:45:18.680 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T11:45:18.713 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T11:45:18.713 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:45:18.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T11:45:18.769 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T11:45:18.867 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:45:18.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T11:45:18.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T11:45:18.966 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:45:18.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T11:45:18.997 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T11:45:19.067 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T11:45:19.067 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:45:19.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T11:45:19.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T11:45:19.220 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:45:19.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T11:45:19.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T11:45:19.310 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:45:19.311 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T11:45:19.340 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T11:45:19.408 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T11:45:19.408 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:45:19.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T11:45:19.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T11:45:19.562 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:45:19.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T11:45:19.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T11:45:19.652 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:45:19.653 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T11:45:19.682 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T11:45:19.751 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-05T11:45:19.837 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T11:45:19.838 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:19.838 INFO:tasks.cephadm:Cluster fsid is 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:45:19.838 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-05T11:45:19.838 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.59', 'mon.b': '172.21.15.89', 'mon.c': '172.21.15.189'} 2024-09-05T11:45:19.838 INFO:tasks.cephadm:First mon is mon.a on smithi059 2024-09-05T11:45:19.839 INFO:tasks.cephadm:First mgr is x 2024-09-05T11:45:19.839 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-05T11:45:19.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-09-05T11:45:19.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-09-05T11:45:19.923 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-05T11:45:19.953 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-05T11:45:19.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:20.097 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82800/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-261-ge856d6b2', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-261.ge856d6b2'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-05 10:44:16.424921', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-bharath6-testing-2024-09-05-1413-squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-05T11:45:20.238 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-bharath6-testing-2024-09-05-1413-squid, sha1 e856d6b2462509a377ef39149a05475ed006ebac from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:45:20.239 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm 2024-09-05T11:45:20.239 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm 2024-09-05T11:45:20.240 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:20.676 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 11:45 /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:20.676 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.078 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 11:45 /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.079 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.473 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 11:45 /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.473 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.512 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.533 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T11:45:21.574 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac on all hosts... 2024-09-05T11:45:21.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T11:45:21.578 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T11:45:21.580 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T11:45:23.231 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T11:45:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T11:45:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-05T11:46:06.455 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-05T11:46:06.517 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T11:46:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T11:46:13.344 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:46:13.376 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-09-05T11:46:13.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-09-05T11:46:13.467 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-05T11:46:13.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-09-05T11:46:13.557 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-09-05T11:46:13.587 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-05T11:46:13.615 INFO:tasks.cephadm:Writing seed config... 2024-09-05T11:46:13.616 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-05T11:46:13.616 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-05T11:46:13.617 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-05T11:46:13.618 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-05T11:46:13.619 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-05T11:46:13.620 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-05T11:46:13.620 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-05T11:46:13.620 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:46:13.621 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-05T11:46:13.655 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 53d560d4-6b7c-11ef-bcd6-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-05T11:46:13.656 DEBUG:teuthology.orchestra.run.smithi059:mon.a> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service 2024-09-05T11:46:13.699 DEBUG:teuthology.orchestra.run.smithi059:mgr.x> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x.service 2024-09-05T11:46:13.743 INFO:tasks.cephadm:Bootstrapping... 2024-09-05T11:46:13.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac -v bootstrap --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.59 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-05T11:46:14.007 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-09-05T11:46:14.008 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac', '-v', 'bootstrap', '--fsid', '53d560d4-6b7c-11ef-bcd6-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.59', '--skip-admin-label'] 2024-09-05T11:46:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-05T11:46:14.008 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying podman|docker is present... 2024-09-05T11:46:14.053 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 5.2.2 2024-09-05T11:46:14.053 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying lvm2 is present... 2024-09-05T11:46:14.053 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying time synchronization is in place... 2024-09-05T11:46:14.067 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T11:46:14.067 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T11:46:14.079 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T11:46:14.079 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:46:14.093 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-09-05T11:46:14.105 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-09-05T11:46:14.105 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-09-05T11:46:14.105 INFO:teuthology.orchestra.run.smithi059.stdout:Repeating the final host check... 2024-09-05T11:46:14.150 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 5.2.2 2024-09-05T11:46:14.150 INFO:teuthology.orchestra.run.smithi059.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-05T11:46:14.150 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl is present 2024-09-05T11:46:14.150 INFO:teuthology.orchestra.run.smithi059.stdout:lvcreate is present 2024-09-05T11:46:14.163 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T11:46:14.163 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T11:46:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T11:46:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:46:14.189 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-09-05T11:46:14.201 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Host looks OK 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Cluster fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Acquiring lock 139847664313344 on /run/cephadm/53d560d4-6b7c-11ef-bcd6-c7b262605968.lock 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Lock 139847664313344 acquired on /run/cephadm/53d560d4-6b7c-11ef-bcd6-c7b262605968.lock 2024-09-05T11:46:14.202 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 3300 ... 2024-09-05T11:46:14.203 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 6789 ... 2024-09-05T11:46:14.203 INFO:teuthology.orchestra.run.smithi059.stdout:Base mon IP(s) is [172.21.15.59:3300, 172.21.15.59:6789], mon addrv is [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-09-05T11:46:14.210 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.59 metric 100 2024-09-05T11:46:14.210 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.59 metric 100 2024-09-05T11:46:14.216 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-05T11:46:14.216 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-05T11:46:14.216 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1775sec hoplimit 64 pref medium 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-09-05T11:46:14.222 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T11:46:14.223 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-09-05T11:46:14.223 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-09-05T11:46:14.223 INFO:teuthology.orchestra.run.smithi059.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-05T11:46:14.224 INFO:teuthology.orchestra.run.smithi059.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-05T11:46:14.224 INFO:teuthology.orchestra.run.smithi059.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T11:46:14.778 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 2024-09-05T11:46:14.778 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T11:46:14.779 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Getting image source signatures 2024-09-05T11:46:14.779 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:57827424943d9f7b8fbbdf71efc0609494faf4d42198e71777a8add52eb57a7e 2024-09-05T11:46:14.779 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-05T11:46:14.779 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying config sha256:1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 2024-09-05T11:46:14.779 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-05T11:46:15.741 INFO:teuthology.orchestra.run.smithi059.stdout:ceph: stdout ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable) 2024-09-05T11:46:15.741 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph version: ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable) 2024-09-05T11:46:15.741 INFO:teuthology.orchestra.run.smithi059.stdout:Extracting ceph user uid/gid from container image... 2024-09-05T11:46:16.679 INFO:teuthology.orchestra.run.smithi059.stdout:stat: stdout 167 167 2024-09-05T11:46:16.679 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial keys... 2024-09-05T11:46:17.881 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQAJmtlmEtSzAhAApIfQgF5cNfEt4IIpaXp6/A== 2024-09-05T11:46:18.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQAKmtlmjyD1EBAA9jX7y0py+FWG55kpNsG6CQ== 2024-09-05T11:46:19.795 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQALmtlmpHDDDBAAQuTYEDpxSt5kbQvXKWw0cQ== 2024-09-05T11:46:19.796 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial monmap... 2024-09-05T11:46:21.019 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool for a [v2:172.21.15.59:3300,v1:172.21.15.59:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: set fsid to 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:46:21.020 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mon... 2024-09-05T11:46:22.049 INFO:teuthology.orchestra.run.smithi059.stdout:create mon.a on 2024-09-05T11:46:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-05T11:46:22.580 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-05T11:46:22.831 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target. 2024-09-05T11:46:22.832 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target. 2024-09-05T11:46:23.171 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a 2024-09-05T11:46:23.171 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service: Unit ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service not loaded. 2024-09-05T11:46:23.433 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target.wants/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service → /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@.service. 2024-09-05T11:46:24.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:24 smithi059 podman[29645]: 2024-09-05 11:46:24.003323256 +0000 UTC m=+0.352541095 container start 0b5a83e8a835356722890dc5069991b5859e36f8ec7e80ba0f26b24153139508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-05T11:46:24.106 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T11:46:24.106 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-09-05T11:46:24.119 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T11:46:24.119 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:46:24.119 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-09-05T11:46:24.120 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T11:46:24.120 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon to start... 2024-09-05T11:46:24.120 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon... 2024-09-05T11:46:24.348 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:24 smithi059 ceph-mon[29663]: mkfs 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:24.348 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:24 smithi059 ceph-mon[29663]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T11:46:24.348 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:24 smithi059 bash[29645]: 0b5a83e8a835356722890dc5069991b5859e36f8ec7e80ba0f26b24153139508 2024-09-05T11:46:24.348 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:24 smithi059 systemd[1]: Started Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:46:25.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: monmap epoch 1 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: last_changed 2024-09-05T11:46:20.317245+0000 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: min_mon_release 19 (squid) 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: election_strategy: 1 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: fsmap 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: mgrmap e1: no daemons active 2024-09-05T11:46:25.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:25 smithi059 ceph-mon[29663]: from='client.? 172.21.15.59:0/3856964633' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-05T11:46:25.184 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout cluster: 2024-09-05T11:46:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout id: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-05T11:46:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout services: 2024-09-05T11:46:25.190 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.573849s) 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout data: 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pgs: 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.557+0000 7febd9200640 1 Processor -- start 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.558+0000 7febd9200640 1 -- start start 2024-09-05T11:46:25.191 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.558+0000 7febd9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.558+0000 7febd9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7febd407bdc0 con 0x7febd4107ff0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.559+0000 7febd1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.559+0000 7febd1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37580/0 (socket says 172.21.15.59:37580) 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.559+0000 7febd1e00640 1 -- 172.21.15.59:0/1209084150 learned_addr learned my addr 172.21.15.59:0/1209084150 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.559+0000 7febd1e00640 1 -- 172.21.15.59:0/1209084150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd410a920 con 0x7febd4107ff0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.562+0000 7febd1e00640 1 --2- 172.21.15.59:0/1209084150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febc4009820 tx=0x7febc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d452736e10ea1213 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.562+0000 7febd1400640 1 -- 172.21.15.59:0/1209084150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc4037070 con 0x7febd4107ff0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.562+0000 7febd1400640 1 -- 172.21.15.59:0/1209084150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc4032440 con 0x7febd4107ff0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.562+0000 7febd1400640 1 -- 172.21.15.59:0/1209084150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc4032740 con 0x7febd4107ff0 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 -- 172.21.15.59:0/1209084150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 msgr2=0x7febd410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:25.192 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 --2- 172.21.15.59:0/1209084150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7febc4009820 tx=0x7febc402a270 comp rx=0 tx=0).stop 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 -- 172.21.15.59:0/1209084150 shutdown_connections 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 --2- 172.21.15.59:0/1209084150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd410a3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 -- 172.21.15.59:0/1209084150 >> 172.21.15.59:0/1209084150 conn(0x7febd4101b90 msgr2=0x7febd4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 -- 172.21.15.59:0/1209084150 shutdown_connections 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.563+0000 7febd9200640 1 -- 172.21.15.59:0/1209084150 wait complete. 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.564+0000 7febd9200640 1 Processor -- start 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.564+0000 7febd9200640 1 -- start start 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.564+0000 7febd9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.564+0000 7febd9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7febd410ada0 con 0x7febd4107ff0 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.565+0000 7febd1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.565+0000 7febd1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37584/0 (socket says 172.21.15.59:37584) 2024-09-05T11:46:25.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.565+0000 7febd1e00640 1 -- 172.21.15.59:0/3856964633 learned_addr learned my addr 172.21.15.59:0/3856964633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.565+0000 7febd1e00640 1 -- 172.21.15.59:0/3856964633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd419d410 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.565+0000 7febd1e00640 1 --2- 172.21.15.59:0/3856964633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7febc40090b0 tx=0x7febc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc4041070 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc403c7a0 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc4037040 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd419d6a0 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41a0390 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febc4044070 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.566+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febc403d3f0 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.567+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd4108450 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.570+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7febc4046450 con 0x7febd4107ff0 2024-09-05T11:46:25.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.635+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7febd402d900 con 0x7febd4107ff0 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.635+0000 7febcaa00640 1 -- 172.21.15.59:0/3856964633 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7febc4046630 con 0x7febd4107ff0 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 msgr2=0x7febd419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 --2- 172.21.15.59:0/3856964633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7febc40090b0 tx=0x7febc40046c0 comp rx=0 tx=0).stop 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 shutdown_connections 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 --2- 172.21.15.59:0/3856964633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febd4107ff0 0x7febd419ced0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 >> 172.21.15.59:0/3856964633 conn(0x7febd4101b90 msgr2=0x7febd4101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 shutdown_connections 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:24.636+0000 7febd9200640 1 -- 172.21.15.59:0/3856964633 wait complete. 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:mon is available 2024-09-05T11:46:25.195 INFO:teuthology.orchestra.run.smithi059.stdout:Assimilating anything we can from ceph.conf... 2024-09-05T11:46:26.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:26 smithi059 ceph-mon[29663]: from='client.? 172.21.15.59:0/3169782427' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T11:46:26.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:26 smithi059 ceph-mon[29663]: from='client.? 172.21.15.59:0/3169782427' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T11:46:26.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.638+0000 7f63ade00640 1 Processor -- start 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.638+0000 7f63ade00640 1 -- start start 2024-09-05T11:46:26.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.639+0000 7f63ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.639+0000 7f63ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63a807c7f0 con 0x7f63a807de00 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.639+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.639+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37594/0 (socket says 172.21.15.59:37594) 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.639+0000 7f63a6a00640 1 -- 172.21.15.59:0/3220767584 learned_addr learned my addr 172.21.15.59:0/3220767584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.640+0000 7f63a6a00640 1 -- 172.21.15.59:0/3220767584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a807c970 con 0x7f63a807de00 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.641+0000 7f63a6a00640 1 --2- 172.21.15.59:0/3220767584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6398009820 tx=0x7f639802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f327babf1e0f83c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.641+0000 7f63a6000640 1 -- 172.21.15.59:0/3220767584 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6398037070 con 0x7f63a807de00 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.641+0000 7f63a6000640 1 -- 172.21.15.59:0/3220767584 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6398032440 con 0x7f63a807de00 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.641+0000 7f63a6000640 1 -- 172.21.15.59:0/3220767584 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6398032740 con 0x7f63a807de00 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 -- 172.21.15.59:0/3220767584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 msgr2=0x7f63a807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 --2- 172.21.15.59:0/3220767584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6398009820 tx=0x7f639802a270 comp rx=0 tx=0).stop 2024-09-05T11:46:26.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 -- 172.21.15.59:0/3220767584 shutdown_connections 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 --2- 172.21.15.59:0/3220767584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a807c2b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 -- 172.21.15.59:0/3220767584 >> 172.21.15.59:0/3220767584 conn(0x7f63a8102ce0 msgr2=0x7f63a8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 -- 172.21.15.59:0/3220767584 shutdown_connections 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.642+0000 7f63ade00640 1 -- 172.21.15.59:0/3220767584 wait complete. 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.643+0000 7f63ade00640 1 Processor -- start 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.643+0000 7f63ade00640 1 -- start start 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.643+0000 7f63ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.643+0000 7f63ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63a807cf10 con 0x7f63a807de00 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.644+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.644+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37606/0 (socket says 172.21.15.59:37606) 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.644+0000 7f63a6a00640 1 -- 172.21.15.59:0/3169782427 learned_addr learned my addr 172.21.15.59:0/3169782427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:26.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.644+0000 7f63a6a00640 1 -- 172.21.15.59:0/3169782427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819d3d0 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.644+0000 7f63a6a00640 1 --2- 172.21.15.59:0/3169782427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f63980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6398041070 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639803c7a0 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6398037040 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819d660 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a0350 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6398044070 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.645+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f639803d3f0 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.646+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a807c330 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.648+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6398046450 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.711+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f63a802d940 con 0x7f63a807de00 2024-09-05T11:46:26.301 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.715+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f6398046630 con 0x7f63a807de00 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.716+0000 7f63aca00640 1 -- 172.21.15.59:0/3169782427 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63980584b0 con 0x7f63a807de00 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 msgr2=0x7f63a819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 --2- 172.21.15.59:0/3169782427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f63980046c0 comp rx=0 tx=0).stop 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 shutdown_connections 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 --2- 172.21.15.59:0/3169782427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63a807de00 0x7f63a819ce90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 >> 172.21.15.59:0/3169782427 conn(0x7f63a8102ce0 msgr2=0x7f63a8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 shutdown_connections 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:25.717+0000 7f63ade00640 1 -- 172.21.15.59:0/3169782427 wait complete. 2024-09-05T11:46:26.302 INFO:teuthology.orchestra.run.smithi059.stdout:Generating new minimal ceph.conf... 2024-09-05T11:46:27.196 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:27 smithi059 ceph-mon[29663]: from='client.? 172.21.15.59:0/2955646251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:46:27.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.752+0000 7fc4f7e00640 1 Processor -- start 2024-09-05T11:46:27.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.752+0000 7fc4f7e00640 1 -- start start 2024-09-05T11:46:27.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f007c810 con 0x7fc4f007de20 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37614/0 (socket says 172.21.15.59:37614) 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f5400640 1 -- 172.21.15.59:0/3684768684 learned_addr learned my addr 172.21.15.59:0/3684768684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.753+0000 7fc4f5400640 1 -- 172.21.15.59:0/3684768684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f007c990 con 0x7fc4f007de20 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.754+0000 7fc4f5400640 1 --2- 172.21.15.59:0/3684768684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009820 tx=0x7fc4e402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f91b45e2f6be772 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4efe00640 1 -- 172.21.15.59:0/3684768684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4037070 con 0x7fc4f007de20 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4efe00640 1 -- 172.21.15.59:0/3684768684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4e4032900 con 0x7fc4f007de20 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4efe00640 1 -- 172.21.15.59:0/3684768684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e40308e0 con 0x7fc4f007de20 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4f7e00640 1 -- 172.21.15.59:0/3684768684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 msgr2=0x7fc4f007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4f7e00640 1 --2- 172.21.15.59:0/3684768684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009820 tx=0x7fc4e402a300 comp rx=0 tx=0).stop 2024-09-05T11:46:27.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4f7e00640 1 -- 172.21.15.59:0/3684768684 shutdown_connections 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4f7e00640 1 --2- 172.21.15.59:0/3684768684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f007c2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.755+0000 7fc4f7e00640 1 -- 172.21.15.59:0/3684768684 >> 172.21.15.59:0/3684768684 conn(0x7fc4f0102ce0 msgr2=0x7fc4f0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.756+0000 7fc4f7e00640 1 -- 172.21.15.59:0/3684768684 shutdown_connections 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.756+0000 7fc4f7e00640 1 -- 172.21.15.59:0/3684768684 wait complete. 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.756+0000 7fc4f7e00640 1 Processor -- start 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f7e00640 1 -- start start 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f010db20 con 0x7fc4f007de20 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37628/0 (socket says 172.21.15.59:37628) 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f5400640 1 -- 172.21.15.59:0/2955646251 learned_addr learned my addr 172.21.15.59:0/2955646251 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.757+0000 7fc4f5400640 1 -- 172.21.15.59:0/2955646251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f019dc00 con 0x7fc4f007de20 2024-09-05T11:46:27.421 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4f5400640 1 --2- 172.21.15.59:0/2955646251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4e402a820 tx=0x7fc4e4030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e4037040 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f019de90 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4e4040070 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4e403a390 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f01a0b80 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4e403a600 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.758+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4e4045810 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.759+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f007c350 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.762+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc4e403b430 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.824+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc4f01a1100 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.824+0000 7fc4ee000640 1 -- 172.21.15.59:0/2955646251 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fc4e403b6a0 con 0x7fc4f007de20 2024-09-05T11:46:27.422 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 msgr2=0x7fc4f019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 --2- 172.21.15.59:0/2955646251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4e402a820 tx=0x7fc4e4030900 comp rx=0 tx=0).stop 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 shutdown_connections 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 --2- 172.21.15.59:0/2955646251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4f007de20 0x7fc4f019d6c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 >> 172.21.15.59:0/2955646251 conn(0x7fc4f0102ce0 msgr2=0x7fc4f0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 shutdown_connections 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:26.826+0000 7fc4f7e00640 1 -- 172.21.15.59:0/2955646251 wait complete. 2024-09-05T11:46:27.423 INFO:teuthology.orchestra.run.smithi059.stdout:Restarting the monitor... 2024-09-05T11:46:27.537 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:27 smithi059 systemd[1]: Stopping Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:46:27.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a[29659]: 2024-09-05T11:46:27.779+0000 7f50b4200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T11:46:27.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a[29659]: 2024-09-05T11:46:27.779+0000 7f50b4200640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-05T11:46:28.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 podman[29901]: 2024-09-05 11:46:28.046367317 +0000 UTC m=+0.384669013 container died 0b5a83e8a835356722890dc5069991b5859e36f8ec7e80ba0f26b24153139508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-05T11:46:28.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 podman[29901]: 2024-09-05 11:46:28.434169952 +0000 UTC m=+0.772471657 container cleanup 0b5a83e8a835356722890dc5069991b5859e36f8ec7e80ba0f26b24153139508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T11:46:28.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 bash[29901]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a 2024-09-05T11:46:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 podman[29913]: 2024-09-05 11:46:28.67704757 +0000 UTC m=+0.622740095 container remove 0b5a83e8a835356722890dc5069991b5859e36f8ec7e80ba0f26b24153139508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T11:46:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-05T11:46:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 systemd[1]: Stopped Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:46:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service: Consumed 1.226s CPU time. 2024-09-05T11:46:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:28 smithi059 systemd[1]: Starting Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 podman[30004]: 2024-09-05 11:46:29.153096131 +0000 UTC m=+0.120024210 container create 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 podman[30004]: 2024-09-05 11:46:29.060494233 +0000 UTC m=+0.027422314 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 podman[30004]: 2024-09-05 11:46:29.355248028 +0000 UTC m=+0.322176118 container init 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 podman[30004]: 2024-09-05 11:46:29.361741574 +0000 UTC m=+0.328669654 container start 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable), process ceph-mon, pid 2 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: pidfile_write: ignore empty --pid-file 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: load: jerasure load: lrc 2024-09-05T11:46:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: RocksDB version: 7.9.2 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Git sha 0 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Compile date 2024-09-05 09:22:07 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: DB SUMMARY 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: DB Session ID: PHVX6BDMFKWY9C4LF3TW 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: CURRENT file: CURRENT 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104242 ; 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.error_if_exists: 0 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.create_if_missing: 0 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.paranoid_checks: 1 2024-09-05T11:46:29.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.env: 0x5604d9a22940 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.info_log: 0x5604f0c7f6a0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.statistics: (nil) 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.use_fsync: 0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_log_file_size: 0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T11:46:29.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_fallocate: 1 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.use_direct_reads: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.db_log_dir: 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.wal_dir: 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T11:46:29.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.write_buffer_manager: 0x5604f0c8edc0 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T11:46:29.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.unordered_write: 0 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.row_cache: None 2024-09-05T11:46:29.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.wal_filter: None 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.two_write_queues: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.wal_compression: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.atomic_flush: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.log_readahead_size: 0 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T11:46:29.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_background_jobs: 2 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_background_compactions: -1 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_subcompactions: 1 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T11:46:29.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_open_files: -1 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_background_flushes: -1 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Compression algorithms supported: 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kZSTD supported: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kXpressCompression supported: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kBZip2Compression supported: 0 2024-09-05T11:46:29.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kLZ4Compression supported: 1 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kZlibCompression supported: 1 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: kSnappyCompression supported: 1 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-05T11:46:29.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T11:46:29.456 INFO:teuthology.orchestra.run.smithi059.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.merge_operator: 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_filter: None 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T11:46:29.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5604f0c7f300) 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks: 1 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_top_level_index_and_filter: 1 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_type: 0 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_index_type: 0 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_shortening: 1 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: checksum: 4 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: no_block_cache: 0 2024-09-05T11:46:29.689 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache: 0x5604f0c731f0 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_name: BinnedLRUCache 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_options: 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: capacity : 536870912 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_shard_bits : 4 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: strict_capacity_limit : 0 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: high_pri_pool_ratio: 0.000 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_compressed: (nil) 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: persistent_cache: (nil) 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size: 4096 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size_deviation: 10 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_restart_interval: 16 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_block_restart_interval: 1 2024-09-05T11:46:29.690 INFO:journalctl@ceph.mon.a.smithi059.stdout: metadata_block_size: 4096 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: partition_filters: 0 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: use_delta_encoding: 1 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: filter_policy: bloomfilter 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: whole_key_filtering: 1 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: verify_compression: 0 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: read_amp_bytes_per_bit: 0 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: format_version: 5 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_index_compression: 1 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_align: 0 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: max_auto_readahead_size: 262144 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: prepopulate_block_cache: 0 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: initial_auto_readahead_size: 8192 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T11:46:29.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T11:46:29.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T11:46:29.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression: NoCompression 2024-09-05T11:46:29.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T11:46:29.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.num_levels: 7 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T11:46:29.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T11:46:29.694 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T11:46:29.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T11:46:29.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.inplace_update_support: 0 2024-09-05T11:46:29.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.bloom_locality: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.max_successive_merges: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.ttl: 2592000 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T11:46:29.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enable_blob_files: false 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.min_blob_size: 0 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-05T11:46:29.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5bebb856-628e-4951-8a11-6eb758bce242 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536789404392, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536789406633, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99467, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97361, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14048, "raw_average_key_size": 49, "raw_value_size": 89689, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725536789, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5bebb856-628e-4951-8a11-6eb758bce242", "db_session_id": "PHVX6BDMFKWY9C4LF3TW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536789406934, "job": 1, "event": "recovery_finished"} 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5604f0ca2e00 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: DB pointer 0x5604f0d9e000 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** DB Stats ** 2024-09-05T11:46:29.700 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: L0 2/0 99.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Sum 2/0 99.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T11:46:29.701 INFO:journalctl@ceph.mon.a.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T11:46:29.702 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative compaction: 0.00 GB write, 12.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval compaction: 0.00 GB write, 12.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache BinnedLRUCache@0x5604f0c731f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: starting mon.a rank 0 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???) e1 preinit fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).mds e1 new map 2024-09-05T11:46:29.703 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).mds e1 print_map 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: e1 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: btime 2024-09-05T11:46:24:063581+0000 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: legacy client fscid: -1 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout: No filesystems configured 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-05T11:46:29.704 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 bash[30004]: 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 systemd[1]: Started Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: monmap epoch 1 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: last_changed 2024-09-05T11:46:20.317245+0000 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: min_mon_release 19 (squid) 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: election_strategy: 1 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: fsmap 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T11:46:29.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:29 smithi059 ceph-mon[30025]: mgrmap e1: no daemons active 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.928+0000 7fe48ca00640 1 Processor -- start 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.928+0000 7fe48ca00640 1 -- start start 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.929+0000 7fe48ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.929+0000 7fe48ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe488081e70 con 0x7fe4880814a0 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.929+0000 7fe485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.929+0000 7fe485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37634/0 (socket says 172.21.15.59:37634) 2024-09-05T11:46:30.513 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.929+0000 7fe485e00640 1 -- 172.21.15.59:0/4275049779 learned_addr learned my addr 172.21.15.59:0/4275049779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.930+0000 7fe485e00640 1 -- 172.21.15.59:0/4275049779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48810ed40 con 0x7fe4880814a0 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.932+0000 7fe485e00640 1 --2- 172.21.15.59:0/4275049779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe478009820 tx=0x7fe47802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04f40be650ca28c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.932+0000 7fe485400640 1 -- 172.21.15.59:0/4275049779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478037070 con 0x7fe4880814a0 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe485400640 1 -- 172.21.15.59:0/4275049779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe478032900 con 0x7fe4880814a0 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe485400640 1 -- 172.21.15.59:0/4275049779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478030850 con 0x7fe4880814a0 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 -- 172.21.15.59:0/4275049779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 msgr2=0x7fe4880818a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 --2- 172.21.15.59:0/4275049779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe478009820 tx=0x7fe47802a270 comp rx=0 tx=0).stop 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 -- 172.21.15.59:0/4275049779 shutdown_connections 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 --2- 172.21.15.59:0/4275049779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4880818a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 -- 172.21.15.59:0/4275049779 >> 172.21.15.59:0/4275049779 conn(0x7fe48807ccc0 msgr2=0x7fe48807f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 -- 172.21.15.59:0/4275049779 shutdown_connections 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.933+0000 7fe48ca00640 1 -- 172.21.15.59:0/4275049779 wait complete. 2024-09-05T11:46:30.514 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.934+0000 7fe48ca00640 1 Processor -- start 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.934+0000 7fe48ca00640 1 -- start start 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe48ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe48ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881120f0 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37642/0 (socket says 172.21.15.59:37642) 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe485e00640 1 -- 172.21.15.59:0/4206991013 learned_addr learned my addr 172.21.15.59:0/4206991013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.935+0000 7fe485e00640 1 -- 172.21.15.59:0/4206991013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a1fe0 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe485e00640 1 --2- 172.21.15.59:0/4206991013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe4780097f0 tx=0x7fe478030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478037040 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe478040070 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a2270 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47803a390 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a4f60 con 0x7fe4880814a0 2024-09-05T11:46:30.515 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.936+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe47803a630 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.937+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe4780459a0 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.937+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488081920 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:29.941+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe478045460 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.004+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe4881a26f0 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.006+0000 7fe47ea00640 1 -- 172.21.15.59:0/4206991013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe47803b430 con 0x7fe4880814a0 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 msgr2=0x7fe4881a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 --2- 172.21.15.59:0/4206991013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe4780097f0 tx=0x7fe478030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 shutdown_connections 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 --2- 172.21.15.59:0/4206991013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4880814a0 0x7fe4881a1aa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 >> 172.21.15.59:0/4206991013 conn(0x7fe48807ccc0 msgr2=0x7fe48807d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 shutdown_connections 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:30.007+0000 7fe48ca00640 1 -- 172.21.15.59:0/4206991013 wait complete. 2024-09-05T11:46:30.516 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-05T11:46:30.536 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-05T11:46:30.536 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mgr... 2024-09-05T11:46:30.537 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-05T11:46:30.537 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-05T11:46:30.878 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T11:46:30.878 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x.service: Unit ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x.service not loaded. 2024-09-05T11:46:31.112 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968.target.wants/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@.service. 2024-09-05T11:46:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:31 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4206991013' entity='client.admin' 2024-09-05T11:46:31.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 systemd[1]: Starting Ceph mgr.x for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:46:31.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 podman[30238]: 2024-09-05 11:46:31.429881068 +0000 UTC m=+0.103192506 container create 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T11:46:31.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 podman[30238]: 2024-09-05 11:46:31.354251467 +0000 UTC m=+0.027562911 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:46:31.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 podman[30238]: 2024-09-05 11:46:31.598796466 +0000 UTC m=+0.272107905 container init 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T11:46:31.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 podman[30238]: 2024-09-05 11:46:31.60538144 +0000 UTC m=+0.278692888 container start 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T11:46:32.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:31.775+0000 7f0c83dda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T11:46:32.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:31 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:31.847+0000 7f0c83dda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T11:46:32.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.043+0000 7f0c83dda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T11:46:32.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.126+0000 7f0c83dda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T11:46:32.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.307+0000 7f0c83dda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:46:32.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.477+0000 7f0c83dda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T11:46:33.163 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.867+0000 7f0c83dda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T11:46:33.163 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:32.941+0000 7f0c83dda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T11:46:33.163 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:33.161+0000 7f0c83dda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T11:46:34.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:34.328+0000 7f0c83dda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T11:46:34.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:34.423+0000 7f0c83dda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T11:46:35.202 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T11:46:35.202 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-09-05T11:46:35.215 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T11:46:35.216 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:46:35.216 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-09-05T11:46:35.216 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T11:46:35.233 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T11:46:35.234 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr to start... 2024-09-05T11:46:35.248 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr... 2024-09-05T11:46:35.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 bash[30238]: 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e 2024-09-05T11:46:35.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 systemd[1]: Started Ceph mgr.x for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:46:35.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:35.419+0000 7f0c83dda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T11:46:35.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:35.566+0000 7f0c83dda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T11:46:35.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:35.736+0000 7f0c83dda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:46:35.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:35 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:35.812+0000 7f0c83dda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T11:46:36.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:35.992+0000 7f0c83dda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T11:46:36.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.076+0000 7f0c83dda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T11:46:36.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.150+0000 7f0c83dda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T11:46:36.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.339+0000 7f0c83dda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T11:46:36.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.419+0000 7f0c83dda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T11:46:36.754 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.502+0000 7f0c83dda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T11:46:36.754 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.603+0000 7f0c83dda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T11:46:36.755 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.677+0000 7f0c83dda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T11:46:36.755 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:36.752+0000 7f0c83dda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:37.717+0000 7f0c83dda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: Activating manager daemon x 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: mgrmap e2: x(active, starting, since 0.00435313s) 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T11:46:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: Manager daemon x is now available 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' 2024-09-05T11:46:38.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:37 smithi059 ceph-mon[30025]: from='mgr.14100 172.21.15.59:0/3372910577' entity='mgr.x' 2024-09-05T11:46:40.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:39 smithi059 ceph-mon[30025]: mgrmap e3: x(active, since 1.00922s) 2024-09-05T11:46:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:40 smithi059 ceph-mon[30025]: mgrmap e4: x(active, since 2s) 2024-09-05T11:46:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:40 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3103876835' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "53d560d4-6b7c-11ef-bcd6-c7b262605968", 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T11:46:41.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T11:46:41.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T11:46:41.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T11:46:41.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T11:46:41.293 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.294 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T11:46:24:063581+0000", 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T11:46:41.295 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T11:46:24.064393+0000", 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.381+0000 7f6003200640 1 Processor -- start 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.381+0000 7f6003200640 1 -- start start 2024-09-05T11:46:41.296 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.381+0000 7f6003200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.381+0000 7f6003200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f600407b7d0 con 0x7f600407cd50 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.382+0000 7f6001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.382+0000 7f6001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35762/0 (socket says 172.21.15.59:35762) 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.382+0000 7f6001e00640 1 -- 172.21.15.59:0/3632079255 learned_addr learned my addr 172.21.15.59:0/3632079255 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.382+0000 7f6001e00640 1 -- 172.21.15.59:0/3632079255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600407b950 con 0x7f600407cd50 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.383+0000 7f6001e00640 1 --2- 172.21.15.59:0/3632079255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009820 tx=0x7f5ff402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df319872e55c5fac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.383+0000 7f6001400640 1 -- 172.21.15.59:0/3632079255 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4037070 con 0x7f600407cd50 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.383+0000 7f6001400640 1 -- 172.21.15.59:0/3632079255 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ff4032900 con 0x7f600407cd50 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.383+0000 7f6001400640 1 -- 172.21.15.59:0/3632079255 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4030850 con 0x7f600407cd50 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 -- 172.21.15.59:0/3632079255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 msgr2=0x7f600407b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 --2- 172.21.15.59:0/3632079255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009820 tx=0x7f5ff402a270 comp rx=0 tx=0).stop 2024-09-05T11:46:41.297 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 -- 172.21.15.59:0/3632079255 shutdown_connections 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 --2- 172.21.15.59:0/3632079255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f600407b200 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 -- 172.21.15.59:0/3632079255 >> 172.21.15.59:0/3632079255 conn(0x7f6004102ae0 msgr2=0x7f6004104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 -- 172.21.15.59:0/3632079255 shutdown_connections 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.384+0000 7f6003200640 1 -- 172.21.15.59:0/3632079255 wait complete. 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.385+0000 7f6003200640 1 Processor -- start 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.385+0000 7f6003200640 1 -- start start 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6003200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6003200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60041094d0 con 0x7f600407cd50 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6001e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35778/0 (socket says 172.21.15.59:35778) 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6001e00640 1 -- 172.21.15.59:0/3103876835 learned_addr learned my addr 172.21.15.59:0/3103876835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:41.298 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.386+0000 7f6001e00640 1 -- 172.21.15.59:0/3103876835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60041a1df0 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f6001e00640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4032ce0 tx=0x7f5ff4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4042070 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ff40308c0 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4037040 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60041a2080 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.387+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60041a4d70 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.388+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5ff4045070 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.388+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600407b280 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.389+0000 7f5ffb400640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 0x7f5fe803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.389+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ff406bf70 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.389+0000 7f5ff9400640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 0x7f5fe803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.390+0000 7f5ff9400640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 0x7f5fe803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098c0 tx=0x7f5fec006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.395+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ff404b170 con 0x7f600407cd50 2024-09-05T11:46:41.299 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.651+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f600419ad20 con 0x7f600407cd50 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.652+0000 7f5ffb400640 1 -- 172.21.15.59:0/3103876835 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f5ff406b990 con 0x7f600407cd50 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 msgr2=0x7f5fe803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 0x7f5fe803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098c0 tx=0x7f5fec006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 msgr2=0x7f60041a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4032ce0 tx=0x7f5ff4032d10 comp rx=0 tx=0).stop 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 shutdown_connections 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5fe8038f70 0x7f5fe803b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.654+0000 7f6003200640 1 --2- 172.21.15.59:0/3103876835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600407cd50 0x7f60041a18b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.655+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 >> 172.21.15.59:0/3103876835 conn(0x7f6004102ae0 msgr2=0x7f6004104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.655+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 shutdown_connections 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:40.655+0000 7f6003200640 1 -- 172.21.15.59:0/3103876835 wait complete. 2024-09-05T11:46:41.300 INFO:teuthology.orchestra.run.smithi059.stdout:mgr is available 2024-09-05T11:46:42.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3511640191' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T11:46:42.474 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.765+0000 7fe191600640 1 Processor -- start 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.765+0000 7fe191600640 1 -- start start 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.766+0000 7fe191600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:42.475 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.766+0000 7fe191600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c07c810 con 0x7fe18c07de20 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.766+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.766+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55678/0 (socket says 172.21.15.59:55678) 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.766+0000 7fe18aa00640 1 -- 172.21.15.59:0/1769805652 learned_addr learned my addr 172.21.15.59:0/1769805652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.767+0000 7fe18aa00640 1 -- 172.21.15.59:0/1769805652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c07c990 con 0x7fe18c07de20 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.767+0000 7fe18aa00640 1 --2- 172.21.15.59:0/1769805652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009820 tx=0x7fe17c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b922851c2fc657 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe18a000640 1 -- 172.21.15.59:0/1769805652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c037070 con 0x7fe18c07de20 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe18a000640 1 -- 172.21.15.59:0/1769805652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe17c032900 con 0x7fe18c07de20 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe18a000640 1 -- 172.21.15.59:0/1769805652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c0308e0 con 0x7fe18c07de20 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe191600640 1 -- 172.21.15.59:0/1769805652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 msgr2=0x7fe18c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe191600640 1 --2- 172.21.15.59:0/1769805652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009820 tx=0x7fe17c02a300 comp rx=0 tx=0).stop 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe191600640 1 -- 172.21.15.59:0/1769805652 shutdown_connections 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.768+0000 7fe191600640 1 --2- 172.21.15.59:0/1769805652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c07c2d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:42.476 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.769+0000 7fe191600640 1 -- 172.21.15.59:0/1769805652 >> 172.21.15.59:0/1769805652 conn(0x7fe18c102ce0 msgr2=0x7fe18c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.769+0000 7fe191600640 1 -- 172.21.15.59:0/1769805652 shutdown_connections 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.769+0000 7fe191600640 1 -- 172.21.15.59:0/1769805652 wait complete. 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.769+0000 7fe191600640 1 Processor -- start 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe191600640 1 -- start start 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe191600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe191600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c10db20 con 0x7fe18c07de20 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55684/0 (socket says 172.21.15.59:55684) 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe18aa00640 1 -- 172.21.15.59:0/3511640191 learned_addr learned my addr 172.21.15.59:0/3511640191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.770+0000 7fe18aa00640 1 -- 172.21.15.59:0/3511640191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19dc00 con 0x7fe18c07de20 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe18aa00640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe17c030610 tx=0x7fe17c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c037040 con 0x7fe18c07de20 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe17c040070 con 0x7fe18c07de20 2024-09-05T11:46:42.477 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17c03a390 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c19de90 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.771+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a0b80 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.772+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fe17c03b650 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.772+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c07c350 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.773+0000 7fe183e00640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 0x7fe17003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.773+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe17c06be30 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.773+0000 7fe182000640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 0x7fe17003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.774+0000 7fe182000640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 0x7fe17003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098c0 tx=0x7fe174006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.778+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe17c070070 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.967+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe18c1a2380 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.969+0000 7fe183e00640 1 -- 172.21.15.59:0/3511640191 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fe17c0329b0 con 0x7fe18c07de20 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 msgr2=0x7fe17003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:42.478 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 0x7fe17003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098c0 tx=0x7fe174006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 msgr2=0x7fe18c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe17c030610 tx=0x7fe17c032670 comp rx=0 tx=0).stop 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 shutdown_connections 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fe170038f70 0x7fe17003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 --2- 172.21.15.59:0/3511640191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe18c07de20 0x7fe18c19d6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.971+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 >> 172.21.15.59:0/3511640191 conn(0x7fe18c102ce0 msgr2=0x7fe18c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.972+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 shutdown_connections 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:41.972+0000 7fe191600640 1 -- 172.21.15.59:0/3511640191 wait complete. 2024-09-05T11:46:42.479 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling cephadm module... 2024-09-05T11:46:43.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2247124733' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-05T11:46:43.509 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ignoring --setuser ceph since I am not root 2024-09-05T11:46:43.509 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ignoring --setgroup ceph since I am not root 2024-09-05T11:46:43.509 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:43.507+0000 7f1ae4a37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T11:46:43.780 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:43.582+0000 7f1ae4a37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T11:46:43.781 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:43.778+0000 7f1ae4a37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.981+0000 7fb780e00640 1 Processor -- start 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.981+0000 7fb780e00640 1 -- start start 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.981+0000 7fb780e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.981+0000 7fb780e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c107ce0 con 0x7fb77c107310 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.982+0000 7fb77b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.982+0000 7fb77b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55696/0 (socket says 172.21.15.59:55696) 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.982+0000 7fb77b400640 1 -- 172.21.15.59:0/1287946719 learned_addr learned my addr 172.21.15.59:0/1287946719 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.982+0000 7fb77b400640 1 -- 172.21.15.59:0/1287946719 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c1084c0 con 0x7fb77c107310 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.983+0000 7fb77b400640 1 --2- 172.21.15.59:0/1287946719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009820 tx=0x7fb76c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92517a74fd39a673 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.983+0000 7fb77aa00640 1 -- 172.21.15.59:0/1287946719 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c037070 con 0x7fb77c107310 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.983+0000 7fb77aa00640 1 -- 172.21.15.59:0/1287946719 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb76c032900 con 0x7fb77c107310 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.983+0000 7fb77aa00640 1 -- 172.21.15.59:0/1287946719 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c030850 con 0x7fb77c107310 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 -- 172.21.15.59:0/1287946719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 msgr2=0x7fb77c107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 --2- 172.21.15.59:0/1287946719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009820 tx=0x7fb76c02a270 comp rx=0 tx=0).stop 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 -- 172.21.15.59:0/1287946719 shutdown_connections 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 --2- 172.21.15.59:0/1287946719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c107710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 -- 172.21.15.59:0/1287946719 >> 172.21.15.59:0/1287946719 conn(0x7fb77c102ae0 msgr2=0x7fb77c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:43.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 -- 172.21.15.59:0/1287946719 shutdown_connections 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.984+0000 7fb780e00640 1 -- 172.21.15.59:0/1287946719 wait complete. 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.985+0000 7fb780e00640 1 Processor -- start 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.985+0000 7fb780e00640 1 -- start start 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.985+0000 7fb780e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.985+0000 7fb780e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c10b870 con 0x7fb77c107310 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.986+0000 7fb77b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.986+0000 7fb77b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55700/0 (socket says 172.21.15.59:55700) 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.986+0000 7fb77b400640 1 -- 172.21.15.59:0/2247124733 learned_addr learned my addr 172.21.15.59:0/2247124733 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.986+0000 7fb77b400640 1 -- 172.21.15.59:0/2247124733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c19b7f0 con 0x7fb77c107310 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.986+0000 7fb77b400640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097f0 tx=0x7fb76c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.987+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c037040 con 0x7fb77c107310 2024-09-05T11:46:43.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.987+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb76c040070 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.987+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c03a390 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.987+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c19ba80 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.987+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c19e770 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.988+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb76c03b430 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.988+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c10b9f0 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.988+0000 7fb778c00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 0x7fb76003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.988+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb76c06c080 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.988+0000 7fb773e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 0x7fb76003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.989+0000 7fb773e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 0x7fb76003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098c0 tx=0x7fb764006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:42.993+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb76c06bde0 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.226+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb77c19e940 con 0x7fb77c107310 2024-09-05T11:46:43.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.278+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb76c032900 con 0x7fb77c107310 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.280+0000 7fb778c00640 1 -- 172.21.15.59:0/2247124733 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb76c043c30 con 0x7fb77c107310 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 msgr2=0x7fb76003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 0x7fb76003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7640098c0 tx=0x7fb764006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 msgr2=0x7fb77c19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097f0 tx=0x7fb76c030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 shutdown_connections 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7fb760038f70 0x7fb76003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 --2- 172.21.15.59:0/2247124733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb77c107310 0x7fb77c19b2b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 >> 172.21.15.59:0/2247124733 conn(0x7fb77c102ae0 msgr2=0x7fb77c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 shutdown_connections 2024-09-05T11:46:44.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:43.282+0000 7fb780e00640 1 -- 172.21.15.59:0/2247124733 wait complete. 2024-09-05T11:46:44.062 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:43 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:43.864+0000 7f1ae4a37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T11:46:44.314 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2247124733' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-05T11:46:44.314 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-mon[30025]: mgrmap e5: x(active, since 5s) 2024-09-05T11:46:44.314 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:44.060+0000 7f1ae4a37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:46:44.314 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:44.236+0000 7f1ae4a37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T11:46:44.919 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:44.654+0000 7f1ae4a37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T11:46:44.919 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:44.732+0000 7f1ae4a37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T11:46:45.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:44.965+0000 7f1ae4a37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T11:46:45.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1192200123' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T11:46:46.282 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.513+0000 7f5971400640 1 Processor -- start 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.513+0000 7f5971400640 1 -- start start 2024-09-05T11:46:46.288 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.513+0000 7f5971400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.513+0000 7f5971400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f596c07c8e0 con 0x7f596c07de60 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.514+0000 7f596a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.514+0000 7f596a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55730/0 (socket says 172.21.15.59:55730) 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.514+0000 7f596a800640 1 -- 172.21.15.59:0/3721202309 learned_addr learned my addr 172.21.15.59:0/3721202309 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.514+0000 7f596a800640 1 -- 172.21.15.59:0/3721202309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c07ca60 con 0x7f596c07de60 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.515+0000 7f596a800640 1 --2- 172.21.15.59:0/3721202309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f595c009820 tx=0x7f595c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97998d1b65c1c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.516+0000 7f5969400640 1 -- 172.21.15.59:0/3721202309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f595c037070 con 0x7f596c07de60 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.516+0000 7f5969400640 1 -- 172.21.15.59:0/3721202309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f595c032900 con 0x7f596c07de60 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.516+0000 7f5969400640 1 -- 172.21.15.59:0/3721202309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f595c030850 con 0x7f596c07de60 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.516+0000 7f5971400640 1 -- 172.21.15.59:0/3721202309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 msgr2=0x7f596c07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.516+0000 7f5971400640 1 --2- 172.21.15.59:0/3721202309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f595c009820 tx=0x7f595c02a270 comp rx=0 tx=0).stop 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.517+0000 7f5971400640 1 -- 172.21.15.59:0/3721202309 shutdown_connections 2024-09-05T11:46:46.289 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.517+0000 7f5971400640 1 --2- 172.21.15.59:0/3721202309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c07c310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.517+0000 7f5971400640 1 -- 172.21.15.59:0/3721202309 >> 172.21.15.59:0/3721202309 conn(0x7f596c102ce0 msgr2=0x7f596c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.517+0000 7f5971400640 1 -- 172.21.15.59:0/3721202309 shutdown_connections 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.517+0000 7f5971400640 1 -- 172.21.15.59:0/3721202309 wait complete. 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.518+0000 7f5971400640 1 Processor -- start 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.518+0000 7f5971400640 1 -- start start 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.518+0000 7f5971400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.518+0000 7f5971400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1096d0 con 0x7f596c07de60 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.518+0000 7f596a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.519+0000 7f596a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55746/0 (socket says 172.21.15.59:55746) 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.519+0000 7f596a800640 1 -- 172.21.15.59:0/1192200123 learned_addr learned my addr 172.21.15.59:0/1192200123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.519+0000 7f596a800640 1 -- 172.21.15.59:0/1192200123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a1fc0 con 0x7f596c07de60 2024-09-05T11:46:46.290 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.519+0000 7f596a800640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f595c0097f0 tx=0x7f595c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.520+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f595c037040 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.520+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a2250 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.520+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f595c040070 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.520+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f595c03a390 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.520+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a4f40 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.521+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f595c032900 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.521+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c07c390 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.522+0000 7f5963400640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 0x7f595003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.522+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f595c06c160 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.522+0000 7f5969e00640 1 -- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 msgr2=0x7f595003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.522+0000 7f5969e00640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 0x7f595003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.527+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f595c06bec0 con 0x7f596c07de60 2024-09-05T11:46:46.291 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.723+0000 7f5969e00640 1 -- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 msgr2=0x7f595003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.723+0000 7f5969e00640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 0x7f595003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.761+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f596c02da90 con 0x7f596c07de60 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.762+0000 7f5963400640 1 -- 172.21.15.59:0/1192200123 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f595c040ce0 con 0x7f596c07de60 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 msgr2=0x7f595003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 0x7f595003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 msgr2=0x7f596c1a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f595c0097f0 tx=0x7f595c030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 shutdown_connections 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f5950038f70 0x7f595003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 --2- 172.21.15.59:0/1192200123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c07de60 0x7f596c1a1a80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.764+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 >> 172.21.15.59:0/1192200123 conn(0x7f596c102ce0 msgr2=0x7f596c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.765+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 shutdown_connections 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:44.765+0000 7f5971400640 1 -- 172.21.15.59:0/1192200123 wait complete. 2024-09-05T11:46:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-09-05T11:46:46.293 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 5... 2024-09-05T11:46:46.549 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:46.239+0000 7f1ae4a37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T11:46:46.550 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:46.326+0000 7f1ae4a37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T11:46:47.605 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.292+0000 7f1ae4a37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T11:46:47.605 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.433+0000 7f1ae4a37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T11:46:47.606 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.603+0000 7f1ae4a37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:46:47.859 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.678+0000 7f1ae4a37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T11:46:47.859 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.857+0000 7f1ae4a37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T11:46:48.184 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:47.941+0000 7f1ae4a37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T11:46:48.184 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.016+0000 7f1ae4a37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T11:46:48.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.204+0000 7f1ae4a37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T11:46:48.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.284+0000 7f1ae4a37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T11:46:48.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.366+0000 7f1ae4a37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T11:46:48.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.466+0000 7f1ae4a37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T11:46:48.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.540+0000 7f1ae4a37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T11:46:48.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:48.613+0000 7f1ae4a37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T11:46:49.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:49.564+0000 7f1ae4a37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T11:46:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: Active manager daemon x restarted 2024-09-05T11:46:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: Activating manager daemon x 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: osdmap e2: 0 total, 0 up, 0 in 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: mgrmap e6: x(active, starting, since 0.00427004s) 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: Manager daemon x is now available 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:46:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:49 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:46:50.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:50 smithi059 ceph-mon[30025]: Found migration_current of "None". Setting to last migration. 2024-09-05T11:46:50.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:50 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T11:46:50.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:50 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T11:46:50.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:50 smithi059 ceph-mon[30025]: mgrmap e7: x(active, since 1.00648s) 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a5e00640 1 Processor -- start 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a5e00640 1 -- start start 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85a007b7d0 con 0x7f85a007cd50 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55762/0 (socket says 172.21.15.59:55762) 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.771+0000 7f85a4a00640 1 -- 172.21.15.59:0/48433240 learned_addr learned my addr 172.21.15.59:0/48433240 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.772+0000 7f85a4a00640 1 -- 172.21.15.59:0/48433240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a007b950 con 0x7f85a007cd50 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.773+0000 7f85a4a00640 1 --2- 172.21.15.59:0/48433240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8594009820 tx=0x7f859402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7866a60d54fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.773+0000 7f859f400640 1 -- 172.21.15.59:0/48433240 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8594037070 con 0x7f85a007cd50 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.773+0000 7f859f400640 1 -- 172.21.15.59:0/48433240 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8594032900 con 0x7f85a007cd50 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.773+0000 7f859f400640 1 -- 172.21.15.59:0/48433240 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8594030850 con 0x7f85a007cd50 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 -- 172.21.15.59:0/48433240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 msgr2=0x7f85a007b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 --2- 172.21.15.59:0/48433240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8594009820 tx=0x7f859402a270 comp rx=0 tx=0).stop 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 -- 172.21.15.59:0/48433240 shutdown_connections 2024-09-05T11:46:51.112 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 --2- 172.21.15.59:0/48433240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a007b200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 -- 172.21.15.59:0/48433240 >> 172.21.15.59:0/48433240 conn(0x7f85a0102ae0 msgr2=0x7f85a0104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 -- 172.21.15.59:0/48433240 shutdown_connections 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.774+0000 7f85a5e00640 1 -- 172.21.15.59:0/48433240 wait complete. 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.775+0000 7f85a5e00640 1 Processor -- start 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.775+0000 7f85a5e00640 1 -- start start 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010d8e0 con 0x7f85a007cd50 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55774/0 (socket says 172.21.15.59:55774) 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a4a00640 1 -- 172.21.15.59:0/3014935467 learned_addr learned my addr 172.21.15.59:0/3014935467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.776+0000 7f85a4a00640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a019d9d0 con 0x7f85a007cd50 2024-09-05T11:46:51.113 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f85a4a00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8594032ce0 tx=0x7f8594032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8594042070 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85940308c0 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a019dc60 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8594037040 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.777+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a0950 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.778+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8594045070 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.779+0000 7f859d600640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.779+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85a007b280 con 0x7f8588038fc0 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.779+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f859406c060 con 0x7f85a007cd50 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.779+0000 7f859fe00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 msgr2=0x7f858803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.779+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:46:51.114 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.980+0000 7f859fe00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 msgr2=0x7f858803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:46.980+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:47.381+0000 7f859fe00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 msgr2=0x7f858803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:47.381+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:48.182+0000 7f859fe00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 msgr2=0x7f858803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2307158254 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:48.182+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:49.568+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f859406ab20 con 0x7f85a007cd50 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:49.568+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 msgr2=0x7f858803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:49.569+0000 7f859d600640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.569+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f859403c7a0 con 0x7f85a007cd50 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.570+0000 7f859d600640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 0x7f858803eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.570+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85a007b280 con 0x7f858803c700 2024-09-05T11:46:51.115 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.571+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 0x7f858803eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.572+0000 7f859fe00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 0x7f858803eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f858c003980 tx=0x7f858c0092b0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.573+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f85a007b280 con 0x7f858803c700 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.580+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f85a0103dc0 con 0x7f858803c700 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f859d600640 1 -- 172.21.15.59:0/3014935467 <== mgr.14114 v2:172.21.15.59:6800/2057764847 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f85a0103dc0 con 0x7f858803c700 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 msgr2=0x7f858803eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 0x7f858803eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f858c003980 tx=0x7f858c0092b0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 msgr2=0x7f85a019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8594032ce0 tx=0x7f8594032d10 comp rx=0 tx=0).stop 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 shutdown_connections 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f858803c700 0x7f858803eaf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:6800/2307158254,v1:172.21.15.59:6801/2307158254] conn(0x7f8588038fc0 0x7f858803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 --2- 172.21.15.59:0/3014935467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a007cd50 0x7f85a019d490 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:51.116 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.581+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 >> 172.21.15.59:0/3014935467 conn(0x7f85a0102ae0 msgr2=0x7f85a01036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:51.117 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.582+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 shutdown_connections 2024-09-05T11:46:51.117 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:50.582+0000 7f85a5e00640 1 -- 172.21.15.59:0/3014935467 wait complete. 2024-09-05T11:46:51.117 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 5 is available 2024-09-05T11:46:51.117 INFO:teuthology.orchestra.run.smithi059.stdout:Setting orchestrator backend to cephadm... 2024-09-05T11:46:52.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='client.14118 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T11:46:52.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='client.14118 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T11:46:52.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:52.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:52.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:52.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:51 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:46:52.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:52.150+0000 7f1a43c00640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:11:46:52] ENGINE Error in HTTPServer.serve 2024-09-05T11:46:52.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: Traceback (most recent call last): 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._connections.run(self.expiration_interval) 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._run(expiration_interval) 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: s = self.context.wrap_socket( 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T11:46:52.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: return self.sslsocket_class._create( 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self.do_handshake() 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._sslobj.do_handshake() 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T11:46:52.406 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.574+0000 7f29ade00640 1 Processor -- start 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.574+0000 7f29ade00640 1 -- start start 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.575+0000 7f29ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.575+0000 7f29ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29a810a260 con 0x7f29a8109890 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.575+0000 7f29a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:52.416 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.575+0000 7f29a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48670/0 (socket says 172.21.15.59:48670) 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.575+0000 7f29a6a00640 1 -- 172.21.15.59:0/2910963914 learned_addr learned my addr 172.21.15.59:0/2910963914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.576+0000 7f29a6a00640 1 -- 172.21.15.59:0/2910963914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a810a890 con 0x7f29a8109890 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29a6a00640 1 --2- 172.21.15.59:0/2910963914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2998009820 tx=0x7f299802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e57d608f7ec4371 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29a5600640 1 -- 172.21.15.59:0/2910963914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2998037070 con 0x7f29a8109890 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29a5600640 1 -- 172.21.15.59:0/2910963914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2998032900 con 0x7f29a8109890 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29a5600640 1 -- 172.21.15.59:0/2910963914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2998030850 con 0x7f29a8109890 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29ade00640 1 -- 172.21.15.59:0/2910963914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 msgr2=0x7f29a8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.577+0000 7f29ade00640 1 --2- 172.21.15.59:0/2910963914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2998009820 tx=0x7f299802a270 comp rx=0 tx=0).stop 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.578+0000 7f29ade00640 1 -- 172.21.15.59:0/2910963914 shutdown_connections 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.578+0000 7f29ade00640 1 --2- 172.21.15.59:0/2910963914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a8109c90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.578+0000 7f29ade00640 1 -- 172.21.15.59:0/2910963914 >> 172.21.15.59:0/2910963914 conn(0x7f29a807ccc0 msgr2=0x7f29a807d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:52.417 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.578+0000 7f29ade00640 1 -- 172.21.15.59:0/2910963914 shutdown_connections 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.578+0000 7f29ade00640 1 -- 172.21.15.59:0/2910963914 wait complete. 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.579+0000 7f29ade00640 1 Processor -- start 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.579+0000 7f29ade00640 1 -- start start 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.579+0000 7f29ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.579+0000 7f29ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29a810dc40 con 0x7f29a8109890 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.579+0000 7f29a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48686/0 (socket says 172.21.15.59:48686) 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29a6a00640 1 -- 172.21.15.59:0/1012950702 learned_addr learned my addr 172.21.15.59:0/1012950702 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29a6a00640 1 -- 172.21.15.59:0/1012950702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a819dc50 con 0x7f29a8109890 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29a6a00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f29980097f0 tx=0x7f2998030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2998037040 con 0x7f29a8109890 2024-09-05T11:46:52.418 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2998040070 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f299803a390 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.580+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a819dee0 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.581+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a0bd0 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.582+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8109c90 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.582+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f299803b430 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.582+0000 7f29aca00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 0x7f298c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.583+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f299806c2f0 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.583+0000 7f29a6000640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 0x7f298c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.583+0000 7f29a6000640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 0x7f298c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29940098c0 tx=0x7f2994006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.588+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2998032900 con 0x7f29a8109890 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.786+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f29a8107c20 con 0x7f298c038e60 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.792+0000 7f29aca00640 1 -- 172.21.15.59:0/1012950702 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29a8107c20 con 0x7f298c038e60 2024-09-05T11:46:52.419 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 msgr2=0x7f298c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 0x7f298c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29940098c0 tx=0x7f2994006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 msgr2=0x7f29a819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f29980097f0 tx=0x7f2998030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 shutdown_connections 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f298c038e60 0x7f298c03b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 --2- 172.21.15.59:0/1012950702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a8109890 0x7f29a819d710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 >> 172.21.15.59:0/1012950702 conn(0x7f29a807ccc0 msgr2=0x7f29a8107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 shutdown_connections 2024-09-05T11:46:52.420 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:51.794+0000 7f29ade00640 1 -- 172.21.15.59:0/1012950702 wait complete. 2024-09-05T11:46:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:46:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: mgrmap e8: x(active, since 2s) 2024-09-05T11:46:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: [05/Sep/2024:11:46:52] ENGINE Bus STARTING 2024-09-05T11:46:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: [05/Sep/2024:11:46:52] ENGINE Serving on https://172.21.15.59:7150 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: [05/Sep/2024:11:46:52] ENGINE Error in HTTPServer.serve 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: Traceback (most recent call last): 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._connections.run(self.expiration_interval) 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._run(expiration_interval) 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T11:46:53.209 INFO:journalctl@ceph.mon.a.smithi059.stdout: s = self.context.wrap_socket( 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: return self.sslsocket_class._create( 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: self.do_handshake() 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._sslobj.do_handshake() 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: [05/Sep/2024:11:46:52] ENGINE Serving on http://172.21.15.59:8765 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: [05/Sep/2024:11:46:52] ENGINE Bus STARTED 2024-09-05T11:46:53.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:52 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.880+0000 7fc47b000640 1 Processor -- start 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc47b000640 1 -- start start 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc47b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc47b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410a260 con 0x7fc474109890 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc473e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:53.721 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc473e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48694/0 (socket says 172.21.15.59:48694) 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.881+0000 7fc473e00640 1 -- 172.21.15.59:0/463541529 learned_addr learned my addr 172.21.15.59:0/463541529 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.882+0000 7fc473e00640 1 -- 172.21.15.59:0/463541529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47410a890 con 0x7fc474109890 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.883+0000 7fc473e00640 1 --2- 172.21.15.59:0/463541529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc464009e80 tx=0x7fc46402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b8ac84dbab4e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.883+0000 7fc473400640 1 -- 172.21.15.59:0/463541529 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464036070 con 0x7fc474109890 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.883+0000 7fc473400640 1 -- 172.21.15.59:0/463541529 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4640318c0 con 0x7fc474109890 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.883+0000 7fc473400640 1 -- 172.21.15.59:0/463541529 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464007ab0 con 0x7fc474109890 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 -- 172.21.15.59:0/463541529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 msgr2=0x7fc474109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 --2- 172.21.15.59:0/463541529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc464009e80 tx=0x7fc46402a7a0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 -- 172.21.15.59:0/463541529 shutdown_connections 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 --2- 172.21.15.59:0/463541529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc474109c90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 -- 172.21.15.59:0/463541529 >> 172.21.15.59:0/463541529 conn(0x7fc47407ccc0 msgr2=0x7fc47407d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:53.722 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 -- 172.21.15.59:0/463541529 shutdown_connections 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.884+0000 7fc47b000640 1 -- 172.21.15.59:0/463541529 wait complete. 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.885+0000 7fc47b000640 1 Processor -- start 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.885+0000 7fc47b000640 1 -- start start 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.885+0000 7fc47b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.885+0000 7fc47b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410dc40 con 0x7fc474109890 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc473e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc473e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48708/0 (socket says 172.21.15.59:48708) 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc473e00640 1 -- 172.21.15.59:0/4136936835 learned_addr learned my addr 172.21.15.59:0/4136936835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc473e00640 1 -- 172.21.15.59:0/4136936835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419dc50 con 0x7fc474109890 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc473e00640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc46402acc0 tx=0x7fc464031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:53.723 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.886+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464036050 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.887+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc464037040 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.887+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc46403d8c0 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.887+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47419dee0 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.887+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a0bd0 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.888+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc464044020 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.888+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474109c90 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.888+0000 7fc471600640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 0x7fc45803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.888+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc464042020 con 0x7fc474109890 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.889+0000 7fc46b400640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 0x7fc45803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.889+0000 7fc46b400640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 0x7fc45803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0098c0 tx=0x7fc45c006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:53.724 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:52.894+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc46404b020 con 0x7fc474109890 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.081+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc474107b80 con 0x7fc458038fc0 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.082+0000 7fc471600640 1 -- 172.21.15.59:0/4136936835 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc474107b80 con 0x7fc458038fc0 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 msgr2=0x7fc45803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 0x7fc45803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0098c0 tx=0x7fc45c006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 msgr2=0x7fc47419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc46402acc0 tx=0x7fc464031ca0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 shutdown_connections 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fc458038fc0 0x7fc45803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 --2- 172.21.15.59:0/4136936835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc474109890 0x7fc47419d710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 >> 172.21.15.59:0/4136936835 conn(0x7fc47407ccc0 msgr2=0x7fc474107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 shutdown_connections 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:53.084+0000 7fc47b000640 1 -- 172.21.15.59:0/4136936835 wait complete. 2024-09-05T11:46:53.725 INFO:teuthology.orchestra.run.smithi059.stdout:Generating ssh key... 2024-09-05T11:46:53.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:53 smithi059 ceph-mon[30025]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: Generating public/private rsa key pair. 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: Your identification has been saved in /tmp/tmpn330zyta/key 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: Your public key has been saved in /tmp/tmpn330zyta/key.pub 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: The key fingerprint is: 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: SHA256:D3TW338Ib5V8n5+O3q1VcAJKt0wHDgrqOjTAmuf58z4 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: The key's randomart image is: 2024-09-05T11:46:55.128 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: +---[RSA 3072]----+ 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: | . o =.. | 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: |. . . o O + | 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: |.. . o + = o .| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: |.o . . o ..=.| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: |o + . S . .o=| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: | + + o o o*| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: | = . +.=| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: | o. E .o.=| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: | .+o. .oo=o| 2024-09-05T11:46:55.129 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:46:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: +----[SHA256]-----+ 2024-09-05T11:46:55.410 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.219+0000 7f3586800640 1 Processor -- start 2024-09-05T11:46:55.410 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.219+0000 7f3586800640 1 -- start start 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.219+0000 7f3586800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.219+0000 7f3586800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f358010a260 con 0x7f3580109890 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.219+0000 7f357f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.220+0000 7f357f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48720/0 (socket says 172.21.15.59:48720) 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.220+0000 7f357f400640 1 -- 172.21.15.59:0/3059665153 learned_addr learned my addr 172.21.15.59:0/3059665153 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.220+0000 7f357f400640 1 -- 172.21.15.59:0/3059665153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358010a890 con 0x7f3580109890 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.221+0000 7f357f400640 1 --2- 172.21.15.59:0/3059665153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3570009820 tx=0x7f357002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dd8eb36a1222e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.221+0000 7f357e000640 1 -- 172.21.15.59:0/3059665153 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3570037070 con 0x7f3580109890 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.221+0000 7f357e000640 1 -- 172.21.15.59:0/3059665153 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3570032900 con 0x7f3580109890 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.221+0000 7f357e000640 1 -- 172.21.15.59:0/3059665153 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3570030850 con 0x7f3580109890 2024-09-05T11:46:55.411 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 -- 172.21.15.59:0/3059665153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 msgr2=0x7f3580109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 --2- 172.21.15.59:0/3059665153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3570009820 tx=0x7f357002a270 comp rx=0 tx=0).stop 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 -- 172.21.15.59:0/3059665153 shutdown_connections 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 --2- 172.21.15.59:0/3059665153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f3580109c90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 -- 172.21.15.59:0/3059665153 >> 172.21.15.59:0/3059665153 conn(0x7f358007ccc0 msgr2=0x7f358007d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 -- 172.21.15.59:0/3059665153 shutdown_connections 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.222+0000 7f3586800640 1 -- 172.21.15.59:0/3059665153 wait complete. 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.223+0000 7f3586800640 1 Processor -- start 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.223+0000 7f3586800640 1 -- start start 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.223+0000 7f3586800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.223+0000 7f3586800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f358010dc40 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f357f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f357f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48732/0 (socket says 172.21.15.59:48732) 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f357f400640 1 -- 172.21.15.59:0/3118668911 learned_addr learned my addr 172.21.15.59:0/3118668911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f357f400640 1 -- 172.21.15.59:0/3118668911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358019dc50 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f357f400640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35700097f0 tx=0x7f3570030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.224+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3570037040 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.225+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3570040070 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.225+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358019dee0 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.225+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f357003a390 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.225+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801a0bd0 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.226+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580109c90 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.226+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f357003b430 con 0x7f3580109890 2024-09-05T11:46:55.413 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.226+0000 7f3577e00640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 0x7f356403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.226+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f357006c4e0 con 0x7f3580109890 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.226+0000 7f357ea00640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 0x7f356403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.227+0000 7f357ea00640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 0x7f356403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098c0 tx=0x7f356c006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.232+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f357006be30 con 0x7f3580109890 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.426+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3580107c20 con 0x7f3564038fc0 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.794+0000 7f3577e00640 1 -- 172.21.15.59:0/3118668911 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3580107c20 con 0x7f3564038fc0 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 msgr2=0x7f356403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 0x7f356403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098c0 tx=0x7f356c006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 msgr2=0x7f358019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35700097f0 tx=0x7f3570030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 shutdown_connections 2024-09-05T11:46:55.414 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.796+0000 7f3586800640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3564038fc0 0x7f356403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:55.415 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.797+0000 7f3586800640 1 --2- 172.21.15.59:0/3118668911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580109890 0x7f358019d710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:55.415 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.797+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 >> 172.21.15.59:0/3118668911 conn(0x7f358007ccc0 msgr2=0x7f3580107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:55.415 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.797+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 shutdown_connections 2024-09-05T11:46:55.415 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:54.797+0000 7f3586800640 1 -- 172.21.15.59:0/3118668911 wait complete. 2024-09-05T11:46:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:55 smithi059 ceph-mon[30025]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:46:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:55 smithi059 ceph-mon[30025]: Generating ssh key... 2024-09-05T11:46:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:55 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:55 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:46:56.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.857+0000 7f717a800640 1 Processor -- start 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.857+0000 7f717a800640 1 -- start start 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f717a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f717a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f717410a260 con 0x7f7174109890 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f7173400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f7173400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48744/0 (socket says 172.21.15.59:48744) 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f7173400640 1 -- 172.21.15.59:0/1064449357 learned_addr learned my addr 172.21.15.59:0/1064449357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.858+0000 7f7173400640 1 -- 172.21.15.59:0/1064449357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717410a890 con 0x7f7174109890 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.859+0000 7f7173400640 1 --2- 172.21.15.59:0/1064449357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7164009820 tx=0x7f716402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db698d556fd90cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f7172a00640 1 -- 172.21.15.59:0/1064449357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7164037070 con 0x7f7174109890 2024-09-05T11:46:56.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f7172a00640 1 -- 172.21.15.59:0/1064449357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7164032900 con 0x7f7174109890 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f7172a00640 1 -- 172.21.15.59:0/1064449357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7164030850 con 0x7f7174109890 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f717a800640 1 -- 172.21.15.59:0/1064449357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 msgr2=0x7f7174109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f717a800640 1 --2- 172.21.15.59:0/1064449357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7164009820 tx=0x7f716402a270 comp rx=0 tx=0).stop 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f717a800640 1 -- 172.21.15.59:0/1064449357 shutdown_connections 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f717a800640 1 --2- 172.21.15.59:0/1064449357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f7174109c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.860+0000 7f717a800640 1 -- 172.21.15.59:0/1064449357 >> 172.21.15.59:0/1064449357 conn(0x7f717407ccc0 msgr2=0x7f717407d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.861+0000 7f717a800640 1 -- 172.21.15.59:0/1064449357 shutdown_connections 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.861+0000 7f717a800640 1 -- 172.21.15.59:0/1064449357 wait complete. 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.861+0000 7f717a800640 1 Processor -- start 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f717a800640 1 -- start start 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f717a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f717a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f717410dc40 con 0x7f7174109890 2024-09-05T11:46:56.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f7173400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f7173400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48750/0 (socket says 172.21.15.59:48750) 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f7173400640 1 -- 172.21.15.59:0/2291458886 learned_addr learned my addr 172.21.15.59:0/2291458886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.862+0000 7f7173400640 1 -- 172.21.15.59:0/2291458886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717419dc50 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f7173400640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71640097f0 tx=0x7f7164030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7164037040 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7164040070 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717419dee0 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f716403a390 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.863+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71741a0bd0 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.864+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7174109c90 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.864+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f716403b430 con 0x7f7174109890 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.865+0000 7f7170c00640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 0x7f715803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:46:56.703 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.865+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f716406c4e0 con 0x7f7174109890 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.865+0000 7f716aa00640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 0x7f715803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.866+0000 7f716aa00640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 0x7f715803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:55.870+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f716406be30 con 0x7f7174109890 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.060+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7174107c20 con 0x7f7158038fc0 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.061+0000 7f7170c00640 1 -- 172.21.15.59:0/2291458886 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7174107c20 con 0x7f7158038fc0 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 msgr2=0x7f715803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 0x7f715803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098c0 tx=0x7f715c006db0 comp rx=0 tx=0).stop 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 msgr2=0x7f717419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71640097f0 tx=0x7f7164030d80 comp rx=0 tx=0).stop 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 shutdown_connections 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f7158038fc0 0x7f715803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:56.704 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 --2- 172.21.15.59:0/2291458886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7174109890 0x7f717419d710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:46:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 >> 172.21.15.59:0/2291458886 conn(0x7f717407ccc0 msgr2=0x7f7174107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:46:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 shutdown_connections 2024-09-05T11:46:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:56.063+0000 7f717a800640 1 -- 172.21.15.59:0/2291458886 wait complete. 2024-09-05T11:46:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-05T11:46:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:Adding key to root@localhost authorized_keys... 2024-09-05T11:46:56.711 INFO:teuthology.orchestra.run.smithi059.stdout:Adding host smithi059... 2024-09-05T11:46:56.941 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:56 smithi059 ceph-mon[30025]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:46:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:57 smithi059 ceph-mon[30025]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:46:59.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:46:59 smithi059 ceph-mon[30025]: Deploying cephadm binary to smithi059 2024-09-05T11:47:02.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Added host 'smithi059' with addr '172.21.15.59' 2024-09-05T11:47:02.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.165+0000 7f2d63c00640 1 Processor -- start 2024-09-05T11:47:02.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.165+0000 7f2d63c00640 1 -- start start 2024-09-05T11:47:02.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d63c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:02.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d63c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c07bdc0 con 0x7f2d5c107ff0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d61200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d61200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48754/0 (socket says 172.21.15.59:48754) 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d61200640 1 -- 172.21.15.59:0/2426446782 learned_addr learned my addr 172.21.15.59:0/2426446782 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.166+0000 7f2d61200640 1 -- 172.21.15.59:0/2426446782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c10a920 con 0x7f2d5c107ff0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.167+0000 7f2d61200640 1 --2- 172.21.15.59:0/2426446782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009820 tx=0x7f2d4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7ced701da056ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d5be00640 1 -- 172.21.15.59:0/2426446782 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c037070 con 0x7f2d5c107ff0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d5be00640 1 -- 172.21.15.59:0/2426446782 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c032900 con 0x7f2d5c107ff0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d5be00640 1 -- 172.21.15.59:0/2426446782 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c030850 con 0x7f2d5c107ff0 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d63c00640 1 -- 172.21.15.59:0/2426446782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 msgr2=0x7f2d5c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d63c00640 1 --2- 172.21.15.59:0/2426446782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009820 tx=0x7f2d4c02a270 comp rx=0 tx=0).stop 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d63c00640 1 -- 172.21.15.59:0/2426446782 shutdown_connections 2024-09-05T11:47:02.449 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d63c00640 1 --2- 172.21.15.59:0/2426446782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c10a3e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.168+0000 7f2d63c00640 1 -- 172.21.15.59:0/2426446782 >> 172.21.15.59:0/2426446782 conn(0x7f2d5c101b90 msgr2=0x7f2d5c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.169+0000 7f2d63c00640 1 -- 172.21.15.59:0/2426446782 shutdown_connections 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.169+0000 7f2d63c00640 1 -- 172.21.15.59:0/2426446782 wait complete. 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d63c00640 1 Processor -- start 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d63c00640 1 -- start start 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d63c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d63c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c10b900 con 0x7f2d5c107ff0 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d61200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d61200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48756/0 (socket says 172.21.15.59:48756) 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.170+0000 7f2d61200640 1 -- 172.21.15.59:0/3081526696 learned_addr learned my addr 172.21.15.59:0/3081526696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d61200640 1 -- 172.21.15.59:0/3081526696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c19dc30 con 0x7f2d5c107ff0 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d61200640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c032ce0 tx=0x7f2d4c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:02.450 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c042070 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c0308c0 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c037040 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c19dec0 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.171+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a0bb0 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.173+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2d4c045070 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.173+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c108450 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.173+0000 7f2d5a000640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 0x7f2d4003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.173+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d4c06bff0 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.173+0000 7f2d53e00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 0x7f2d4003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.174+0000 7f2d53e00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 0x7f2d4003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d440098c0 tx=0x7f2d44006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.178+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d4c04b170 con 0x7f2d5c107ff0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:46:57.365+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}) -- 0x7f2d5c102dd0 con 0x7f2d40038fc0 2024-09-05T11:47:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.869+0000 7f2d5a000640 1 -- 172.21.15.59:0/3081526696 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2d5c102dd0 con 0x7f2d40038fc0 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 msgr2=0x7f2d4003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 0x7f2d4003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d440098c0 tx=0x7f2d44006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 msgr2=0x7f2d5c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c032ce0 tx=0x7f2d4c032d10 comp rx=0 tx=0).stop 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 shutdown_connections 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f2d40038fc0 0x7f2d4003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 --2- 172.21.15.59:0/3081526696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d5c107ff0 0x7f2d5c19d6f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 >> 172.21.15.59:0/3081526696 conn(0x7f2d5c101b90 msgr2=0x7f2d5c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 shutdown_connections 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:01.872+0000 7f2d63c00640 1 -- 172.21.15.59:0/3081526696 wait complete. 2024-09-05T11:47:02.452 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mon service... 2024-09-05T11:47:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:02 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:02 smithi059 ceph-mon[30025]: Added host smithi059 2024-09-05T11:47:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:02 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.960+0000 7fb926600640 1 Processor -- start 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb926600640 1 -- start start 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb926600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb926600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb92007c8c0 con 0x7fb92007de40 2024-09-05T11:47:03.801 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb91f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb91f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58440/0 (socket says 172.21.15.59:58440) 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.961+0000 7fb91f400640 1 -- 172.21.15.59:0/2971320339 learned_addr learned my addr 172.21.15.59:0/2971320339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.962+0000 7fb91f400640 1 -- 172.21.15.59:0/2971320339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92007ca40 con 0x7fb92007de40 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.962+0000 7fb91f400640 1 --2- 172.21.15.59:0/2971320339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb910009820 tx=0x7fb91002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b773001ce06ff88 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.963+0000 7fb91ea00640 1 -- 172.21.15.59:0/2971320339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb910037070 con 0x7fb92007de40 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.963+0000 7fb91ea00640 1 -- 172.21.15.59:0/2971320339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb910032900 con 0x7fb92007de40 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.963+0000 7fb91ea00640 1 -- 172.21.15.59:0/2971320339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb910030850 con 0x7fb92007de40 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.963+0000 7fb926600640 1 -- 172.21.15.59:0/2971320339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 msgr2=0x7fb92007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.963+0000 7fb926600640 1 --2- 172.21.15.59:0/2971320339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb910009820 tx=0x7fb91002a270 comp rx=0 tx=0).stop 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.964+0000 7fb926600640 1 -- 172.21.15.59:0/2971320339 shutdown_connections 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.964+0000 7fb926600640 1 --2- 172.21.15.59:0/2971320339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92007c2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.964+0000 7fb926600640 1 -- 172.21.15.59:0/2971320339 >> 172.21.15.59:0/2971320339 conn(0x7fb920102ce0 msgr2=0x7fb920105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:03.802 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.964+0000 7fb926600640 1 -- 172.21.15.59:0/2971320339 shutdown_connections 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.964+0000 7fb926600640 1 -- 172.21.15.59:0/2971320339 wait complete. 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb926600640 1 Processor -- start 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb926600640 1 -- start start 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb926600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb926600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010db00 con 0x7fb92007de40 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb91f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb91f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58454/0 (socket says 172.21.15.59:58454) 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.965+0000 7fb91f400640 1 -- 172.21.15.59:0/3925459205 learned_addr learned my addr 172.21.15.59:0/3925459205 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb91f400640 1 -- 172.21.15.59:0/3925459205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92019dbd0 con 0x7fb92007de40 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb91f400640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb9100097f0 tx=0x7fb910030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb910037040 con 0x7fb92007de40 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb910040070 con 0x7fb92007de40 2024-09-05T11:47:03.803 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91003a390 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92019de60 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.966+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a0b50 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.967+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb91003b430 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.967+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92007c370 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.968+0000 7fb91cc00640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 0x7fb90403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.968+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb91006c510 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.968+0000 7fb916a00640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 0x7fb90403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.969+0000 7fb916a00640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 0x7fb90403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9080098c0 tx=0x7fb908006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:02.973+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb91006be60 con 0x7fb92007de40 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.173+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb920103f60 con 0x7fb904038fc0 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.178+0000 7fb91cc00640 1 -- 172.21.15.59:0/3925459205 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb920103f60 con 0x7fb904038fc0 2024-09-05T11:47:03.804 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 msgr2=0x7fb90403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 0x7fb90403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9080098c0 tx=0x7fb908006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 msgr2=0x7fb92019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb9100097f0 tx=0x7fb910030d80 comp rx=0 tx=0).stop 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 shutdown_connections 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fb904038fc0 0x7fb90403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 --2- 172.21.15.59:0/3925459205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92007de40 0x7fb92019d690 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 >> 172.21.15.59:0/3925459205 conn(0x7fb920102ce0 msgr2=0x7fb920103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.180+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 shutdown_connections 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:03.181+0000 7fb926600640 1 -- 172.21.15.59:0/3925459205 wait complete. 2024-09-05T11:47:03.805 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mgr service... 2024-09-05T11:47:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:04 smithi059 ceph-mon[30025]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:04 smithi059 ceph-mon[30025]: Saving service mon spec with placement count:5 2024-09-05T11:47:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:04 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:05.903 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:05 smithi059 ceph-mon[30025]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:05.903 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:05 smithi059 ceph-mon[30025]: Saving service mgr spec with placement count:2 2024-09-05T11:47:05.903 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:05 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:05.903 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:05 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.512+0000 7f3049400640 1 Processor -- start 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.512+0000 7f3049400640 1 -- start start 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.512+0000 7f3049400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.512+0000 7f3049400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f304407b6e0 con 0x7f304407ccf0 2024-09-05T11:47:05.915 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.512+0000 7f3043e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.513+0000 7f3043e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58464/0 (socket says 172.21.15.59:58464) 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.513+0000 7f3043e00640 1 -- 172.21.15.59:0/2629192135 learned_addr learned my addr 172.21.15.59:0/2629192135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.513+0000 7f3043e00640 1 -- 172.21.15.59:0/2629192135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304407b860 con 0x7f304407ccf0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.514+0000 7f3043e00640 1 --2- 172.21.15.59:0/2629192135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3034009820 tx=0x7f303402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9717d02f5c6ce2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.514+0000 7f3042a00640 1 -- 172.21.15.59:0/2629192135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3034037070 con 0x7f304407ccf0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.514+0000 7f3042a00640 1 -- 172.21.15.59:0/2629192135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3034032900 con 0x7f304407ccf0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.514+0000 7f3042a00640 1 -- 172.21.15.59:0/2629192135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3034030850 con 0x7f304407ccf0 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 -- 172.21.15.59:0/2629192135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 msgr2=0x7f304407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 --2- 172.21.15.59:0/2629192135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3034009820 tx=0x7f303402a270 comp rx=0 tx=0).stop 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 -- 172.21.15.59:0/2629192135 shutdown_connections 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 --2- 172.21.15.59:0/2629192135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304407b1a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 -- 172.21.15.59:0/2629192135 >> 172.21.15.59:0/2629192135 conn(0x7f3044102a50 msgr2=0x7f3044104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:05.916 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 -- 172.21.15.59:0/2629192135 shutdown_connections 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.515+0000 7f3049400640 1 -- 172.21.15.59:0/2629192135 wait complete. 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.516+0000 7f3049400640 1 Processor -- start 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.516+0000 7f3049400640 1 -- start start 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.516+0000 7f3049400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.516+0000 7f3049400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f304410d910 con 0x7f304407ccf0 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3043e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3043e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58478/0 (socket says 172.21.15.59:58478) 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3043e00640 1 -- 172.21.15.59:0/3700692173 learned_addr learned my addr 172.21.15.59:0/3700692173 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3043e00640 1 -- 172.21.15.59:0/3700692173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304419d9f0 con 0x7f304407ccf0 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3043e00640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f30340097f0 tx=0x7f3034030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3034037040 con 0x7f304407ccf0 2024-09-05T11:47:05.917 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.517+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3034040070 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.518+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304419dc80 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.518+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f303403a390 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.518+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441a0970 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.519+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f303403b430 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.519+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304407b220 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.519+0000 7f3040c00640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 0x7f302803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.519+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f303406c450 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.520+0000 7f3043400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 0x7f302803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.520+0000 7f3043400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 0x7f302803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30300098c0 tx=0x7f3030006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.524+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3034032900 con 0x7f304407ccf0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.728+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3044103d60 con 0x7f3028038fc0 2024-09-05T11:47:05.918 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.732+0000 7f3040c00640 1 -- 172.21.15.59:0/3700692173 <== mgr.14114 v2:172.21.15.59:6800/2057764847 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3044103d60 con 0x7f3028038fc0 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 msgr2=0x7f302803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 0x7f302803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30300098c0 tx=0x7f3030006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 msgr2=0x7f304419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f30340097f0 tx=0x7f3034030d80 comp rx=0 tx=0).stop 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 shutdown_connections 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f3028038fc0 0x7f302803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 --2- 172.21.15.59:0/3700692173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304407ccf0 0x7f304419d4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 >> 172.21.15.59:0/3700692173 conn(0x7f3044102a50 msgr2=0x7f3044103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.734+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 shutdown_connections 2024-09-05T11:47:05.919 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:04.735+0000 7f3049400640 1 -- 172.21.15.59:0/3700692173 wait complete. 2024-09-05T11:47:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.374+0000 7f0b19000640 1 Processor -- start 2024-09-05T11:47:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.374+0000 7f0b19000640 1 -- start start 2024-09-05T11:47:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.375+0000 7f0b19000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.375+0000 7f0b19000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14107ee0 con 0x7f0b14107510 2024-09-05T11:47:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.375+0000 7f0b11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.375+0000 7f0b11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58486/0 (socket says 172.21.15.59:58486) 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.375+0000 7f0b11e00640 1 -- 172.21.15.59:0/3174230202 learned_addr learned my addr 172.21.15.59:0/3174230202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.376+0000 7f0b11e00640 1 -- 172.21.15.59:0/3174230202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b141086c0 con 0x7f0b14107510 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b11e00640 1 --2- 172.21.15.59:0/3174230202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009820 tx=0x7f0b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d8c5b7b1d8646e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b11400640 1 -- 172.21.15.59:0/3174230202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b04037070 con 0x7f0b14107510 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b11400640 1 -- 172.21.15.59:0/3174230202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04032920 con 0x7f0b14107510 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b11400640 1 -- 172.21.15.59:0/3174230202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b04030810 con 0x7f0b14107510 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b19000640 1 -- 172.21.15.59:0/3174230202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 msgr2=0x7f0b14107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b19000640 1 --2- 172.21.15.59:0/3174230202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009820 tx=0x7f0b0402a270 comp rx=0 tx=0).stop 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.377+0000 7f0b19000640 1 -- 172.21.15.59:0/3174230202 shutdown_connections 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.378+0000 7f0b19000640 1 --2- 172.21.15.59:0/3174230202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b14107910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:07.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.378+0000 7f0b19000640 1 -- 172.21.15.59:0/3174230202 >> 172.21.15.59:0/3174230202 conn(0x7f0b14102ce0 msgr2=0x7f0b14105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.378+0000 7f0b19000640 1 -- 172.21.15.59:0/3174230202 shutdown_connections 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.378+0000 7f0b19000640 1 -- 172.21.15.59:0/3174230202 wait complete. 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b19000640 1 Processor -- start 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b19000640 1 -- start start 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b19000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b19000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1410ba70 con 0x7f0b14107510 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58496/0 (socket says 172.21.15.59:58496) 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.379+0000 7f0b11e00640 1 -- 172.21.15.59:0/2130743499 learned_addr learned my addr 172.21.15.59:0/2130743499 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b11e00640 1 -- 172.21.15.59:0/2130743499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1419ba20 con 0x7f0b14107510 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b11e00640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097f0 tx=0x7f0b040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b04037040 con 0x7f0b14107510 2024-09-05T11:47:07.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04030cc0 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1419bcb0 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b0403a640 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.380+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1419e9a0 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.382+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1410bbf0 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.382+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0b0403b3f0 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.382+0000 7f0b0aa00640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 0x7f0af803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.382+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b0406c570 con 0x7f0b14107510 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.382+0000 7f0b0be00640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 0x7f0af803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.383+0000 7f0b0be00640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 0x7f0af803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0098c0 tx=0x7f0afc006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:07.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.387+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b04042e10 con 0x7f0b14107510 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.576+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0b14107990 con 0x7f0b14107510 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.578+0000 7f0b0aa00640 1 -- 172.21.15.59:0/2130743499 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0b040328c0 con 0x7f0b14107510 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 msgr2=0x7f0af803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 0x7f0af803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0098c0 tx=0x7f0afc006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 msgr2=0x7f0b1419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b040097f0 tx=0x7f0b040327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 shutdown_connections 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0af8038fc0 0x7f0af803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 --2- 172.21.15.59:0/2130743499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b14107510 0x7f0b1419b4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 >> 172.21.15.59:0/2130743499 conn(0x7f0b14102ce0 msgr2=0x7f0b14103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 shutdown_connections 2024-09-05T11:47:07.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:06.581+0000 7f0b19000640 1 -- 172.21.15.59:0/2130743499 wait complete. 2024-09-05T11:47:07.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:07 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:07.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:07 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2130743499' entity='client.admin' 2024-09-05T11:47:08.953 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.692+0000 7fa26e000640 1 Processor -- start 2024-09-05T11:47:08.953 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.692+0000 7fa26e000640 1 -- start start 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.692+0000 7fa26e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.692+0000 7fa26e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2680758a0 con 0x7fa268107ff0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.693+0000 7fa267400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.693+0000 7fa267400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58512/0 (socket says 172.21.15.59:58512) 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.693+0000 7fa267400640 1 -- 172.21.15.59:0/1836477442 learned_addr learned my addr 172.21.15.59:0/1836477442 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.694+0000 7fa267400640 1 -- 172.21.15.59:0/1836477442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26810a920 con 0x7fa268107ff0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.695+0000 7fa267400640 1 --2- 172.21.15.59:0/1836477442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa258009820 tx=0x7fa25802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44071396c912f961 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.695+0000 7fa266000640 1 -- 172.21.15.59:0/1836477442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258037070 con 0x7fa268107ff0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.695+0000 7fa266000640 1 -- 172.21.15.59:0/1836477442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa258032920 con 0x7fa268107ff0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.695+0000 7fa266000640 1 -- 172.21.15.59:0/1836477442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258030810 con 0x7fa268107ff0 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 -- 172.21.15.59:0/1836477442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 msgr2=0x7fa26810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:08.954 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 --2- 172.21.15.59:0/1836477442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa258009820 tx=0x7fa25802a270 comp rx=0 tx=0).stop 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 -- 172.21.15.59:0/1836477442 shutdown_connections 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 --2- 172.21.15.59:0/1836477442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa26810a3e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 -- 172.21.15.59:0/1836477442 >> 172.21.15.59:0/1836477442 conn(0x7fa268101af0 msgr2=0x7fa268103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 -- 172.21.15.59:0/1836477442 shutdown_connections 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.696+0000 7fa26e000640 1 -- 172.21.15.59:0/1836477442 wait complete. 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.697+0000 7fa26e000640 1 Processor -- start 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa26e000640 1 -- start start 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa26e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa26e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa26810ba00 con 0x7fa268107ff0 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa267400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa267400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58526/0 (socket says 172.21.15.59:58526) 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.698+0000 7fa267400640 1 -- 172.21.15.59:0/2146419532 learned_addr learned my addr 172.21.15.59:0/2146419532 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa267400640 1 -- 172.21.15.59:0/2146419532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa268107db0 con 0x7fa268107ff0 2024-09-05T11:47:08.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa267400640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097f0 tx=0x7fa2580327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258037040 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa258030cc0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa25803a6b0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.699+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa268105fa0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.700+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2681064e0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.701+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa25803a8a0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.701+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa268108450 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.701+0000 7fa25fe00640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 0x7fa24c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.701+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa25803b8b0 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.701+0000 7fa266a00640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 0x7fa24c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.702+0000 7fa266a00640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 0x7fa24c03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098c0 tx=0x7fa254006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.707+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa25806c380 con 0x7fa268107ff0 2024-09-05T11:47:08.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.897+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa268106bf0 con 0x7fa268107ff0 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.900+0000 7fa25fe00640 1 -- 172.21.15.59:0/2146419532 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa25806c000 con 0x7fa268107ff0 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 msgr2=0x7fa24c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 0x7fa24c03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098c0 tx=0x7fa254006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 msgr2=0x7fa268107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097f0 tx=0x7fa2580327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 shutdown_connections 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7fa24c038f70 0x7fa24c03b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 --2- 172.21.15.59:0/2146419532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268107ff0 0x7fa268107870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:08.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 >> 172.21.15.59:0/2146419532 conn(0x7fa268101af0 msgr2=0x7fa268102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:08.958 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 shutdown_connections 2024-09-05T11:47:08.958 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:07.902+0000 7fa26e000640 1 -- 172.21.15.59:0/2146419532 wait complete. 2024-09-05T11:47:08.958 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling the dashboard module... 2024-09-05T11:47:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2146419532' entity='client.admin' 2024-09-05T11:47:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:09 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:09 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1406311353' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-05T11:47:10.909 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ignoring --setuser ceph since I am not root 2024-09-05T11:47:10.910 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ignoring --setgroup ceph since I am not root 2024-09-05T11:47:10.910 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:10.827+0000 7faa9362d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T11:47:10.910 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:10.907+0000 7faa9362d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T11:47:11.188 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:11.103+0000 7faa9362d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T11:47:11.188 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:11.186+0000 7faa9362d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T11:47:11.221 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.476+0000 7f49a9c00640 1 Processor -- start 2024-09-05T11:47:11.221 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.477+0000 7f49a9c00640 1 -- start start 2024-09-05T11:47:11.221 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.477+0000 7f49a9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:11.221 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.477+0000 7f49a9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4107ee0 con 0x7f49a4107510 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.478+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.478+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58534/0 (socket says 172.21.15.59:58534) 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.478+0000 7f49a2a00640 1 -- 172.21.15.59:0/3673467738 learned_addr learned my addr 172.21.15.59:0/3673467738 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.479+0000 7f49a2a00640 1 -- 172.21.15.59:0/3673467738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a41086c0 con 0x7f49a4107510 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.480+0000 7f49a2a00640 1 --2- 172.21.15.59:0/3673467738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4994009820 tx=0x7f499402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ecf3bbb73a1e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.481+0000 7f49a1600640 1 -- 172.21.15.59:0/3673467738 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4994037070 con 0x7f49a4107510 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.481+0000 7f49a1600640 1 -- 172.21.15.59:0/3673467738 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994032920 con 0x7f49a4107510 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.481+0000 7f49a1600640 1 -- 172.21.15.59:0/3673467738 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4994030810 con 0x7f49a4107510 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.481+0000 7f49a9c00640 1 -- 172.21.15.59:0/3673467738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 msgr2=0x7f49a4107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 --2- 172.21.15.59:0/3673467738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4994009820 tx=0x7f499402a270 comp rx=0 tx=0).stop 2024-09-05T11:47:11.222 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 -- 172.21.15.59:0/3673467738 shutdown_connections 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 --2- 172.21.15.59:0/3673467738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a4107910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 -- 172.21.15.59:0/3673467738 >> 172.21.15.59:0/3673467738 conn(0x7f49a4102ce0 msgr2=0x7f49a4105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 -- 172.21.15.59:0/3673467738 shutdown_connections 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.482+0000 7f49a9c00640 1 -- 172.21.15.59:0/3673467738 wait complete. 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.483+0000 7f49a9c00640 1 Processor -- start 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.483+0000 7f49a9c00640 1 -- start start 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.483+0000 7f49a9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.483+0000 7f49a9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49a407c230 con 0x7f49a4107510 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.484+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.484+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58550/0 (socket says 172.21.15.59:58550) 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.484+0000 7f49a2a00640 1 -- 172.21.15.59:0/1406311353 learned_addr learned my addr 172.21.15.59:0/1406311353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.484+0000 7f49a2a00640 1 -- 172.21.15.59:0/1406311353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a419dc30 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.484+0000 7f49a2a00640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49940097f0 tx=0x7f49940327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.485+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4994037040 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.485+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994030cc0 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.485+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a419dec0 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.485+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f499403a640 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.485+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a0bb0 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.486+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a4107910 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.486+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f499403b3f0 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.487+0000 7f499b400640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 0x7f498803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.487+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f499406c5a0 con 0x7f49a4107510 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.487+0000 7f49a2000640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 0x7f498803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.488+0000 7f49a2000640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 0x7f498803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49900098c0 tx=0x7f4990006db0 comp rx=0 tx=0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:11.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.493+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4994042e10 con 0x7f49a4107510 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:09.730+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f49a402d9c0 con 0x7f49a4107510 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.578+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f499406bef0 con 0x7f49a4107510 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.579+0000 7f499b400640 1 -- 172.21.15.59:0/1406311353 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f499403a940 con 0x7f49a4107510 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.581+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 msgr2=0x7f498803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 0x7f498803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49900098c0 tx=0x7f4990006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 msgr2=0x7f49a419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49940097f0 tx=0x7f49940327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 shutdown_connections 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4988038f70 0x7f498803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 --2- 172.21.15.59:0/1406311353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49a4107510 0x7f49a419d6f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 >> 172.21.15.59:0/1406311353 conn(0x7f49a4102ce0 msgr2=0x7f49a410ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 shutdown_connections 2024-09-05T11:47:11.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:10.582+0000 7f49a9c00640 1 -- 172.21.15.59:0/1406311353 wait complete. 2024-09-05T11:47:11.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:11.369+0000 7faa9362d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:47:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-mon[30025]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-mon[30025]: from='mgr.14114 172.21.15.59:0/3789469035' entity='mgr.x' 2024-09-05T11:47:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1406311353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-05T11:47:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-mon[30025]: mgrmap e9: x(active, since 21s) 2024-09-05T11:47:11.814 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:11.548+0000 7faa9362d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T11:47:12.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:11.942+0000 7faa9362d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T11:47:12.208 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:12.016+0000 7faa9362d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T11:47:12.491 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:12.236+0000 7faa9362d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T11:47:12.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:12 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4069074986' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.367+0000 7f41a3e00640 1 Processor -- start 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.367+0000 7f41a3e00640 1 -- start start 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.367+0000 7f41a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:13.323 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.367+0000 7f41a3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f419c10a260 con 0x7f419c109890 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.368+0000 7f41a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.368+0000 7f41a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60374/0 (socket says 172.21.15.59:60374) 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.368+0000 7f41a1400640 1 -- 172.21.15.59:0/2837719924 learned_addr learned my addr 172.21.15.59:0/2837719924 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.368+0000 7f41a1400640 1 -- 172.21.15.59:0/2837719924 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c10a890 con 0x7f419c109890 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.369+0000 7f41a1400640 1 --2- 172.21.15.59:0/2837719924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4190009820 tx=0x7f419002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4309b08fd627c357 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f419be00640 1 -- 172.21.15.59:0/2837719924 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4190037070 con 0x7f419c109890 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f419be00640 1 -- 172.21.15.59:0/2837719924 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4190032920 con 0x7f419c109890 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f419be00640 1 -- 172.21.15.59:0/2837719924 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4190030810 con 0x7f419c109890 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 -- 172.21.15.59:0/2837719924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 msgr2=0x7f419c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 --2- 172.21.15.59:0/2837719924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4190009820 tx=0x7f419002a270 comp rx=0 tx=0).stop 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 -- 172.21.15.59:0/2837719924 shutdown_connections 2024-09-05T11:47:13.324 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 --2- 172.21.15.59:0/2837719924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c109c90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:13.325 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 -- 172.21.15.59:0/2837719924 >> 172.21.15.59:0/2837719924 conn(0x7f419c07ccc0 msgr2=0x7f419c07d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:13.325 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.370+0000 7f41a3e00640 1 -- 172.21.15.59:0/2837719924 shutdown_connections 2024-09-05T11:47:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.371+0000 7f41a3e00640 1 -- 172.21.15.59:0/2837719924 wait complete. 2024-09-05T11:47:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.371+0000 7f41a3e00640 1 Processor -- start 2024-09-05T11:47:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a3e00640 1 -- start start 2024-09-05T11:47:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:13.328 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f419c10dc40 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60382/0 (socket says 172.21.15.59:60382) 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.372+0000 7f41a1400640 1 -- 172.21.15.59:0/4069074986 learned_addr learned my addr 172.21.15.59:0/4069074986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f41a1400640 1 -- 172.21.15.59:0/4069074986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c19dc60 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f41a1400640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41900097f0 tx=0x7f41900327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4190037040 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4190030cc0 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c19def0 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f419003a640 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.373+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c1a0be0 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.374+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f419003b3f0 con 0x7f419c109890 2024-09-05T11:47:13.329 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.374+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c109d10 con 0x7f419c109890 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.375+0000 7f419a000640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 0x7f418403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.375+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f419006c5c0 con 0x7f419c109890 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.375+0000 7f41a0a00640 1 -- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 msgr2=0x7f418403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.375+0000 7f41a0a00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 0x7f418403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.380+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41900371e0 con 0x7f419c109890 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.576+0000 7f41a0a00640 1 -- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 msgr2=0x7f418403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.576+0000 7f41a0a00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 0x7f418403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.616+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f419c1a0db0 con 0x7f419c109890 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.617+0000 7f419a000640 1 -- 172.21.15.59:0/4069074986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f419c1a0db0 con 0x7f419c109890 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.619+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 msgr2=0x7f418403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.619+0000 7f41a3e00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 0x7f418403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 msgr2=0x7f419c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:13.330 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41900097f0 tx=0x7f41900327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 shutdown_connections 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f4184039020 0x7f418403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 --2- 172.21.15.59:0/4069074986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f419c109890 0x7f419c19d720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 >> 172.21.15.59:0/4069074986 conn(0x7f419c07ccc0 msgr2=0x7f419c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 shutdown_connections 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:12.620+0000 7f41a3e00640 1 -- 172.21.15.59:0/4069074986 wait complete. 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-09-05T11:47:13.331 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 9... 2024-09-05T11:47:13.604 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:13.428+0000 7faa9362d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T11:47:13.605 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:13.513+0000 7faa9362d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T11:47:14.628 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:14.482+0000 7faa9362d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T11:47:14.629 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:14.626+0000 7faa9362d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T11:47:14.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:14.797+0000 7faa9362d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:47:14.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:14.872+0000 7faa9362d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T11:47:15.401 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.051+0000 7faa9362d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T11:47:15.401 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.136+0000 7faa9362d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T11:47:15.402 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.212+0000 7faa9362d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T11:47:15.402 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.399+0000 7faa9362d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T11:47:15.665 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.480+0000 7faa9362d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T11:47:15.665 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.562+0000 7faa9362d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T11:47:15.665 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.663+0000 7faa9362d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T11:47:15.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.736+0000 7faa9362d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T11:47:15.958 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:15.809+0000 7faa9362d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T11:47:17.031 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:16.758+0000 7faa9362d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T11:47:17.031 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: Active manager daemon x restarted 2024-09-05T11:47:17.031 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: Activating manager daemon x 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: osdmap e3: 0 total, 0 up, 0 in 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: mgrmap e10: x(active, starting, since 0.00355777s) 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: Manager daemon x is now available 2024-09-05T11:47:17.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T11:47:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T11:47:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:17 smithi059 ceph-mon[30025]: mgrmap e11: x(active, since 1.0074s) 2024-09-05T11:47:18.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:18.446+0000 7fa9f0c00640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:11:47:18] ENGINE Error in HTTPServer.serve 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: Traceback (most recent call last): 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._connections.run(self.expiration_interval) 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._run(expiration_interval) 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: s = self.context.wrap_socket( 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T11:47:18.709 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: return self.sslsocket_class._create( 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self.do_handshake() 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: self._sslobj.do_handshake() 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T11:47:18.710 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.862+0000 7f0262400640 1 Processor -- start 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.862+0000 7f0262400640 1 -- start start 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.862+0000 7f0262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.862+0000 7f0262400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0757c0 con 0x7f025c107e00 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.863+0000 7f0261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.863+0000 7f0261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60388/0 (socket says 172.21.15.59:60388) 2024-09-05T11:47:18.902 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.863+0000 7f0261000640 1 -- 172.21.15.59:0/3121584058 learned_addr learned my addr 172.21.15.59:0/3121584058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.863+0000 7f0261000640 1 -- 172.21.15.59:0/3121584058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c10a730 con 0x7f025c107e00 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.864+0000 7f0261000640 1 --2- 172.21.15.59:0/3121584058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0250009820 tx=0x7f025002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6e1c4dfa52d7a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.864+0000 7f025b400640 1 -- 172.21.15.59:0/3121584058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250037070 con 0x7f025c107e00 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f025b400640 1 -- 172.21.15.59:0/3121584058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0250032920 con 0x7f025c107e00 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f025b400640 1 -- 172.21.15.59:0/3121584058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250030810 con 0x7f025c107e00 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f0262400640 1 -- 172.21.15.59:0/3121584058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 msgr2=0x7f025c10a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f0262400640 1 --2- 172.21.15.59:0/3121584058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0250009820 tx=0x7f025002a270 comp rx=0 tx=0).stop 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f0262400640 1 -- 172.21.15.59:0/3121584058 shutdown_connections 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f0262400640 1 --2- 172.21.15.59:0/3121584058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c10a1f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.865+0000 7f0262400640 1 -- 172.21.15.59:0/3121584058 >> 172.21.15.59:0/3121584058 conn(0x7f025c1019a0 msgr2=0x7f025c103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.866+0000 7f0262400640 1 -- 172.21.15.59:0/3121584058 shutdown_connections 2024-09-05T11:47:18.903 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.866+0000 7f0262400640 1 -- 172.21.15.59:0/3121584058 wait complete. 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0262400640 1 Processor -- start 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0262400640 1 -- start start 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0262400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f025c10b750 con 0x7f025c107e00 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60390/0 (socket says 172.21.15.59:60390) 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.867+0000 7f0261000640 1 -- 172.21.15.59:0/872421280 learned_addr learned my addr 172.21.15.59:0/872421280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0261000640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c19da60 con 0x7f025c107e00 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0261000640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02500097f0 tx=0x7f02500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250037040 con 0x7f025c107e00 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0250030cc0 con 0x7f025c107e00 2024-09-05T11:47:18.904 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025003a6b0 con 0x7f025c107e00 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.868+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c19dcf0 con 0x7f025c107e00 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.869+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a09e0 con 0x7f025c107e00 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.870+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f025003baf0 con 0x7f025c107e00 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.870+0000 7f0259600640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.870+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f024403bbf0 con 0x7f0244039020 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.871+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f025006c630 con 0x7f025c107e00 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.871+0000 7f025be00640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 msgr2=0x7f024403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:13.871+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:14.072+0000 7f025be00640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 msgr2=0x7f024403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:14.072+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:14.473+0000 7f025be00640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 msgr2=0x7f024403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:18.905 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:14.473+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:15.274+0000 7f025be00640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 msgr2=0x7f024403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/2057764847 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:15.274+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:16.763+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f025006b020 con 0x7f025c107e00 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:16.763+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 msgr2=0x7f024403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:16.763+0000 7f0259600640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.765+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f025003b3f0 con 0x7f025c107e00 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.765+0000 7f0259600640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 0x7f024403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.765+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f024403bbf0 con 0x7f024403cae0 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.766+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 0x7f024403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.767+0000 7f025be00640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 0x7f024403eed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0248003980 tx=0x7f02480092b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.768+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f024403bbf0 con 0x7f024403cae0 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.781+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f025c108260 con 0x7f024403cae0 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0259600640 1 -- 172.21.15.59:0/872421280 <== mgr.14146 v2:172.21.15.59:6800/23374249 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f025c108260 con 0x7f024403cae0 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 msgr2=0x7f024403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 0x7f024403eed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0248003980 tx=0x7f02480092b0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 msgr2=0x7f025c19d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02500097f0 tx=0x7f02500327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 shutdown_connections 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f024403cae0 0x7f024403eed0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:6800/2057764847,v1:172.21.15.59:6801/2057764847] conn(0x7f0244039020 0x7f024403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 --2- 172.21.15.59:0/872421280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025c107e00 0x7f025c19d520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 >> 172.21.15.59:0/872421280 conn(0x7f025c1019a0 msgr2=0x7f025c1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.782+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 shutdown_connections 2024-09-05T11:47:18.907 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:17.783+0000 7f0262400640 1 -- 172.21.15.59:0/872421280 wait complete. 2024-09-05T11:47:18.908 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 9 is available 2024-09-05T11:47:18.908 INFO:teuthology.orchestra.run.smithi059.stdout:Generating a dashboard self-signed certificate... 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-mon[30025]: [05/Sep/2024:11:47:18] ENGINE Bus STARTING 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-mon[30025]: [05/Sep/2024:11:47:18] ENGINE Serving on https://172.21.15.59:7150 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-mon[30025]: [05/Sep/2024:11:47:18] ENGINE Error in HTTPServer.serve 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: Traceback (most recent call last): 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._connections.run(self.expiration_interval) 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._run(expiration_interval) 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T11:47:19.147 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: s = self.context.wrap_socket( 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: return self.sslsocket_class._create( 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: self.do_handshake() 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: self._sslobj.do_handshake() 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-mon[30025]: [05/Sep/2024:11:47:18] ENGINE Serving on http://172.21.15.59:8765 2024-09-05T11:47:19.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:18 smithi059 ceph-mon[30025]: [05/Sep/2024:11:47:18] ENGINE Bus STARTED 2024-09-05T11:47:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:19 smithi059 ceph-mon[30025]: mgrmap e12: x(active, since 2s) 2024-09-05T11:47:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:19 smithi059 ceph-mon[30025]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.377+0000 7f6446a00640 1 Processor -- start 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f6446a00640 1 -- start start 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f6446a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f6446a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f644010a260 con 0x7f6440109890 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f643fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:20.306 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f643fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60484/0 (socket says 172.21.15.59:60484) 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.378+0000 7f643fe00640 1 -- 172.21.15.59:0/4128024654 learned_addr learned my addr 172.21.15.59:0/4128024654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.379+0000 7f643fe00640 1 -- 172.21.15.59:0/4128024654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644010a890 con 0x7f6440109890 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.380+0000 7f643fe00640 1 --2- 172.21.15.59:0/4128024654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6430009820 tx=0x7f643002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac47d0030c3667c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.380+0000 7f643ea00640 1 -- 172.21.15.59:0/4128024654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6430037070 con 0x7f6440109890 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.380+0000 7f643ea00640 1 -- 172.21.15.59:0/4128024654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6430032920 con 0x7f6440109890 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.380+0000 7f643ea00640 1 -- 172.21.15.59:0/4128024654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6430030810 con 0x7f6440109890 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 -- 172.21.15.59:0/4128024654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 msgr2=0x7f6440109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 --2- 172.21.15.59:0/4128024654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6430009820 tx=0x7f643002a270 comp rx=0 tx=0).stop 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 -- 172.21.15.59:0/4128024654 shutdown_connections 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 --2- 172.21.15.59:0/4128024654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f6440109c90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:20.307 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 -- 172.21.15.59:0/4128024654 >> 172.21.15.59:0/4128024654 conn(0x7f644007ccc0 msgr2=0x7f644007d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 -- 172.21.15.59:0/4128024654 shutdown_connections 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.381+0000 7f6446a00640 1 -- 172.21.15.59:0/4128024654 wait complete. 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.382+0000 7f6446a00640 1 Processor -- start 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f6446a00640 1 -- start start 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f6446a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f6446a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f644010dc40 con 0x7f6440109890 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f643fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f643fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60490/0 (socket says 172.21.15.59:60490) 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f643fe00640 1 -- 172.21.15.59:0/325419022 learned_addr learned my addr 172.21.15.59:0/325419022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.383+0000 7f643fe00640 1 -- 172.21.15.59:0/325419022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019dc60 con 0x7f6440109890 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f643fe00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f64300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6430037040 con 0x7f6440109890 2024-09-05T11:47:20.308 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6430030cc0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f643003a640 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019def0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.384+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64401a0be0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.385+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f643003b3f0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.385+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440109d10 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.386+0000 7f643cc00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 0x7f642403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.386+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f643006c6a0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.386+0000 7f643f400640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 0x7f642403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.387+0000 7f643f400640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 0x7f642403b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098c0 tx=0x7f642c006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.391+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64300328e0 con 0x7f6440109890 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.583+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6440107cb0 con 0x7f6424039020 2024-09-05T11:47:20.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.678+0000 7f643cc00640 1 -- 172.21.15.59:0/325419022 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6440107cb0 con 0x7f6424039020 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 msgr2=0x7f642403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 0x7f642403b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098c0 tx=0x7f642c006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 msgr2=0x7f644019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f64300327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 shutdown_connections 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.680+0000 7f6446a00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6424039020 0x7f642403b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.681+0000 7f6446a00640 1 --2- 172.21.15.59:0/325419022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6440109890 0x7f644019d720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.681+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 >> 172.21.15.59:0/325419022 conn(0x7f644007ccc0 msgr2=0x7f64401075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.681+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 shutdown_connections 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:19.681+0000 7f6446a00640 1 -- 172.21.15.59:0/325419022 wait complete. 2024-09-05T11:47:20.310 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial admin user... 2024-09-05T11:47:21.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:21.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:22.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ahiNuEC0/c3qAOkqZW6SQeMO1tc0N8jXGmSZciFyHt89MWoC/rEd.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725536841, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.821+0000 7f7809e00640 1 Processor -- start 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.821+0000 7f7809e00640 1 -- start start 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7809e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7809e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f780407c7d0 con 0x7f780407dde0 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7802a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7802a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54918/0 (socket says 172.21.15.59:54918) 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7802a00640 1 -- 172.21.15.59:0/3486866212 learned_addr learned my addr 172.21.15.59:0/3486866212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:22.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.822+0000 7f7802a00640 1 -- 172.21.15.59:0/3486866212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780407c950 con 0x7f780407dde0 2024-09-05T11:47:22.389 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.823+0000 7f7802a00640 1 --2- 172.21.15.59:0/3486866212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009820 tx=0x7f77f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e956bd4a55f356df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:22.389 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7802000640 1 -- 172.21.15.59:0/3486866212 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4037070 con 0x7f780407dde0 2024-09-05T11:47:22.389 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7802000640 1 -- 172.21.15.59:0/3486866212 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4032920 con 0x7f780407dde0 2024-09-05T11:47:22.389 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7802000640 1 -- 172.21.15.59:0/3486866212 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4030810 con 0x7f780407dde0 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 -- 172.21.15.59:0/3486866212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 msgr2=0x7f780407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 --2- 172.21.15.59:0/3486866212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009820 tx=0x7f77f402a270 comp rx=0 tx=0).stop 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 -- 172.21.15.59:0/3486866212 shutdown_connections 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 --2- 172.21.15.59:0/3486866212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780407c290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 -- 172.21.15.59:0/3486866212 >> 172.21.15.59:0/3486866212 conn(0x7f7804102ce0 msgr2=0x7f7804105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 -- 172.21.15.59:0/3486866212 shutdown_connections 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.824+0000 7f7809e00640 1 -- 172.21.15.59:0/3486866212 wait complete. 2024-09-05T11:47:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.825+0000 7f7809e00640 1 Processor -- start 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.825+0000 7f7809e00640 1 -- start start 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7809e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7809e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f780410db60 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7802a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7802a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54932/0 (socket says 172.21.15.59:54932) 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7802a00640 1 -- 172.21.15.59:0/2748774308 learned_addr learned my addr 172.21.15.59:0/2748774308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.826+0000 7f7802a00640 1 -- 172.21.15.59:0/2748774308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780419dbf0 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7802a00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77f40097f0 tx=0x7f77f40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4037040 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4030cc0 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f403a6b0 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780419de80 con 0x7f780407dde0 2024-09-05T11:47:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.827+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a0b70 con 0x7f780407dde0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.828+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f77f403baf0 con 0x7f780407dde0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.828+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780407c310 con 0x7f780407dde0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.829+0000 7f7808a00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 0x7f77e803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.829+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77f406c680 con 0x7f780407dde0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.829+0000 7f77fa000640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 0x7f77e803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.830+0000 7f77fa000640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 0x7f77e803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0098c0 tx=0x7f77ec006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:20.835+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77f4042e10 con 0x7f780407dde0 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.039+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f780419e360 con 0x7f77e8039020 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.289+0000 7f7808a00640 1 -- 172.21.15.59:0/2748774308 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f780419e360 con 0x7f77e8039020 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.291+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 msgr2=0x7f77e803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.291+0000 7f7809e00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 0x7f77e803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0098c0 tx=0x7f77ec006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.291+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 msgr2=0x7f780419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.291+0000 7f7809e00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77f40097f0 tx=0x7f77f40327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 shutdown_connections 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f77e8039020 0x7f77e803b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 --2- 172.21.15.59:0/2748774308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f780407dde0 0x7f780419d6b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 >> 172.21.15.59:0/2748774308 conn(0x7f7804102ce0 msgr2=0x7f7804103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 shutdown_connections 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:21.292+0000 7f7809e00640 1 -- 172.21.15.59:0/2748774308 wait complete. 2024-09-05T11:47:22.393 INFO:teuthology.orchestra.run.smithi059.stdout:Fetching dashboard port number... 2024-09-05T11:47:22.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:22 smithi059 ceph-mon[30025]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:23.977 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 8443 2024-09-05T11:47:23.977 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a2b200640 1 Processor -- start 2024-09-05T11:47:23.977 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a2b200640 1 -- start start 2024-09-05T11:47:23.977 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a2b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a2b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c107ce0 con 0x7f7a2c107310 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a29e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a29e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54940/0 (socket says 172.21.15.59:54940) 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.205+0000 7f7a29e00640 1 -- 172.21.15.59:0/2077752028 learned_addr learned my addr 172.21.15.59:0/2077752028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.206+0000 7f7a29e00640 1 -- 172.21.15.59:0/2077752028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1084c0 con 0x7f7a2c107310 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.207+0000 7f7a29e00640 1 --2- 172.21.15.59:0/2077752028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009820 tx=0x7f7a1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fcbfe8094698f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.207+0000 7f7a29400640 1 -- 172.21.15.59:0/2077752028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c037070 con 0x7f7a2c107310 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.207+0000 7f7a29400640 1 -- 172.21.15.59:0/2077752028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c032920 con 0x7f7a2c107310 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.207+0000 7f7a29400640 1 -- 172.21.15.59:0/2077752028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c030810 con 0x7f7a2c107310 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 -- 172.21.15.59:0/2077752028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 msgr2=0x7f7a2c107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2077752028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009820 tx=0x7f7a1c02a270 comp rx=0 tx=0).stop 2024-09-05T11:47:23.978 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 -- 172.21.15.59:0/2077752028 shutdown_connections 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2077752028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c107710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 -- 172.21.15.59:0/2077752028 >> 172.21.15.59:0/2077752028 conn(0x7f7a2c102ae0 msgr2=0x7f7a2c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 -- 172.21.15.59:0/2077752028 shutdown_connections 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.208+0000 7f7a2b200640 1 -- 172.21.15.59:0/2077752028 wait complete. 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a2b200640 1 Processor -- start 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a2b200640 1 -- start start 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a2b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a2b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10b870 con 0x7f7a2c107310 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a29e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a29e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54948/0 (socket says 172.21.15.59:54948) 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.209+0000 7f7a29e00640 1 -- 172.21.15.59:0/2629043686 learned_addr learned my addr 172.21.15.59:0/2629043686 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a29e00640 1 -- 172.21.15.59:0/2629043686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c19b820 con 0x7f7a2c107310 2024-09-05T11:47:23.979 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a29e00640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c030560 tx=0x7f7a1c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c037040 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c040070 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c19bab0 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.210+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a1c03a740 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.211+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c19e7a0 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.212+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7a1c03bb90 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.212+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c10b9f0 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.212+0000 7f7a22a00640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 0x7f7a1003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.212+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a1c06c660 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.212+0000 7f7a23e00640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 0x7f7a1003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.213+0000 7f7a23e00640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 0x7f7a1003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a140098c0 tx=0x7f7a14006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.217+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a1c04a200 con 0x7f7a2c107310 2024-09-05T11:47:23.980 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.380+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7a1c03a8e0 con 0x7f7a2c107310 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.406+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7a2c19e970 con 0x7f7a2c107310 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.407+0000 7f7a22a00640 1 -- 172.21.15.59:0/2629043686 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7a1c031e80 con 0x7f7a2c107310 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 msgr2=0x7f7a1003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 0x7f7a1003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a140098c0 tx=0x7f7a14006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 msgr2=0x7f7a2c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c030560 tx=0x7f7a1c032670 comp rx=0 tx=0).stop 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 shutdown_connections 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7a10039020 0x7f7a1003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 --2- 172.21.15.59:0/2629043686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a2c107310 0x7f7a2c19b2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 >> 172.21.15.59:0/2629043686 conn(0x7f7a2c102ae0 msgr2=0x7f7a2c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 shutdown_connections 2024-09-05T11:47:23.981 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:23.409+0000 7f7a2b200640 1 -- 172.21.15.59:0/2629043686 wait complete. 2024-09-05T11:47:23.993 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T11:47:23.993 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-09-05T11:47:24.007 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T11:47:24.007 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-09-05T11:47:24.007 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-09-05T11:47:24.007 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-05T11:47:24.009 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph Dashboard is now available at: 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout: URL: https://smithi059.front.sepia.ceph.com:8443/ 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout: User: admin 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout: Password: p8649qefvf 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:24.010 INFO:teuthology.orchestra.run.smithi059.stdout:Saving cluster configuration to /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config directory 2024-09-05T11:47:24.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:24 smithi059 ceph-mon[30025]: mgrmap e13: x(active, since 6s) 2024-09-05T11:47:24.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:24 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2629043686' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.502+0000 7f4ae0a00640 1 Processor -- start 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.502+0000 7f4ae0a00640 1 -- start start 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.502+0000 7f4ae0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.502+0000 7f4ae0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc107ee0 con 0x7f4adc107510 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.503+0000 7f4ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:25.467 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.503+0000 7f4ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54962/0 (socket says 172.21.15.59:54962) 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.503+0000 7f4ad9e00640 1 -- 172.21.15.59:0/2749077554 learned_addr learned my addr 172.21.15.59:0/2749077554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.503+0000 7f4ad9e00640 1 -- 172.21.15.59:0/2749077554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc1086c0 con 0x7f4adc107510 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.504+0000 7f4ad9e00640 1 --2- 172.21.15.59:0/2749077554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009820 tx=0x7f4acc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fc65b52304c35d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ad8a00640 1 -- 172.21.15.59:0/2749077554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4acc037070 con 0x7f4adc107510 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ad8a00640 1 -- 172.21.15.59:0/2749077554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc032920 con 0x7f4adc107510 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ad8a00640 1 -- 172.21.15.59:0/2749077554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4acc030810 con 0x7f4adc107510 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 -- 172.21.15.59:0/2749077554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 msgr2=0x7f4adc107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/2749077554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009820 tx=0x7f4acc02a270 comp rx=0 tx=0).stop 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 -- 172.21.15.59:0/2749077554 shutdown_connections 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/2749077554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc107910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 -- 172.21.15.59:0/2749077554 >> 172.21.15.59:0/2749077554 conn(0x7f4adc102ce0 msgr2=0x7f4adc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:25.468 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 -- 172.21.15.59:0/2749077554 shutdown_connections 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.505+0000 7f4ae0a00640 1 -- 172.21.15.59:0/2749077554 wait complete. 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.506+0000 7f4ae0a00640 1 Processor -- start 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.506+0000 7f4ae0a00640 1 -- start start 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ae0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ae0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc10ba70 con 0x7f4adc107510 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54968/0 (socket says 172.21.15.59:54968) 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ad9e00640 1 -- 172.21.15.59:0/1374532557 learned_addr learned my addr 172.21.15.59:0/1374532557 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.507+0000 7f4ad9e00640 1 -- 172.21.15.59:0/1374532557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc19dc60 con 0x7f4adc107510 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ad9e00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097f0 tx=0x7f4acc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4acc037040 con 0x7f4adc107510 2024-09-05T11:47:25.469 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc030cc0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4acc03a640 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4adc19def0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.508+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4adc1a0be0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.509+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4adc10bbf0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.510+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4acc03b3f0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.510+0000 7f4ad2a00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 0x7f4ac003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.510+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4acc06c810 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.510+0000 7f4ad9400640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 0x7f4ac003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.511+0000 7f4ad9400640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 0x7f4ac003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80098c0 tx=0x7f4ac8006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.515+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4acc0371e0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.796+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f4adc1a0db0 con 0x7f4adc107510 2024-09-05T11:47:25.470 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.798+0000 7f4ad2a00640 1 -- 172.21.15.59:0/1374532557 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f4acc03b6e0 con 0x7f4adc107510 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 msgr2=0x7f4ac003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 0x7f4ac003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80098c0 tx=0x7f4ac8006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 msgr2=0x7f4adc19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097f0 tx=0x7f4acc0327b0 comp rx=0 tx=0).stop 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 shutdown_connections 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ac00390c0 0x7f4ac003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 --2- 172.21.15.59:0/1374532557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4adc107510 0x7f4adc19d720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 >> 172.21.15.59:0/1374532557 conn(0x7f4adc102ce0 msgr2=0x7f4adc103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 shutdown_connections 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-09-05T11:47:24.801+0000 7f4ae0a00640 1 -- 172.21.15.59:0/1374532557 wait complete. 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-05T11:47:25.471 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout:Or, if you are only running a single cluster on this host: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: ceph telemetry on 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout:For more information see: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.472 INFO:teuthology.orchestra.run.smithi059.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-05T11:47:25.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:25.473 INFO:teuthology.orchestra.run.smithi059.stdout:Bootstrap complete. 2024-09-05T11:47:25.521 INFO:tasks.cephadm:Fetching config... 2024-09-05T11:47:25.521 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:47:25.521 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-05T11:47:25.557 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-05T11:47:25.557 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:47:25.557 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-05T11:47:25.634 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-05T11:47:25.634 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:47:25.634 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-05T11:47:25.732 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-05T11:47:25.732 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:47:25.732 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-05T11:47:25.809 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-05T11:47:25.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T11:47:25.933 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:47:25.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T11:47:25.998 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:47:26.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:25 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1374532557' entity='client.admin' 2024-09-05T11:47:26.011 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T11:47:26.052 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSpc7zGlpiAYsOtvh1TXG57QJBCVh/Zt31CqcqW4zKet02yJVKSvk09GFJjq856f+jcWyejO8hBVT2M14oQwvLFGRRFB3KdqGb6dswnRoM9E85H1lZL9QZ1Sx8GJYzqhOjSAnf3ERTU8p1FH2gx5bQbruE12BZMBVcM1P85WKnLtzd3RFlllSnlib5Ji1890By9qW551vOu1+I7Y09vx+1SU4aIyiiEMpx5d3EGELru2ttUQsHbLdR7GcE964DBzrz3XSm9lyrif5GcDbO27IDJfGbJRkqVkZp1I6MvqP8RuSoOrv/jRxMh5b6aprDc4z2+4f4X0s5u3yOaDUd6DuqPAKca74BsluSDU9nacFpiGtLLC15x+HygrCNfL0UpoLdA8R7F3ipRkHehFWB4Pl2QRy03+DpM0up7KUxNJpc7kzxEDJtGAS2tVkrgC+900CyC3VIwsIF8lS2BZrm8vfwYbnP2mDKQZBtq6/FQTEngUR5xKVFuU2qVrpQ0kALEmk= ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:47:26.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-05T11:47:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:26.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.979+0000 7fbafcc00640 1 -- 172.21.15.59:0/2448895884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 msgr2=0x7fbaf8076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:26.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.979+0000 7fbafcc00640 1 --2- 172.21.15.59:0/2448895884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf8076fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009880 tx=0x7fbae802a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.980+0000 7fbafcc00640 1 -- 172.21.15.59:0/2448895884 shutdown_connections 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.980+0000 7fbafcc00640 1 --2- 172.21.15.59:0/2448895884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf8076fa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.980+0000 7fbafcc00640 1 -- 172.21.15.59:0/2448895884 >> 172.21.15.59:0/2448895884 conn(0x7fbaf80fefc0 msgr2=0x7fbaf8101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.980+0000 7fbafcc00640 1 -- 172.21.15.59:0/2448895884 shutdown_connections 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.980+0000 7fbafcc00640 1 -- 172.21.15.59:0/2448895884 wait complete. 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.981+0000 7fbafcc00640 1 Processor -- start 2024-09-05T11:47:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.981+0000 7fbafcc00640 1 -- start start 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.981+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.981+0000 7fbafcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf810dcf0 con 0x7fbaf8076ba0 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.982+0000 7fbaf5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.982+0000 7fbaf5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54980/0 (socket says 172.21.15.59:54980) 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.982+0000 7fbaf5e00640 1 -- 172.21.15.59:0/1357442873 learned_addr learned my addr 172.21.15.59:0/1357442873 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.982+0000 7fbaf5e00640 1 -- 172.21.15.59:0/1357442873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf819d8a0 con 0x7fbaf8076ba0 2024-09-05T11:47:26.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.982+0000 7fbaf5e00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbae8004f60 tx=0x7fbae80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.983+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae803f070 con 0x7fbaf8076ba0 2024-09-05T11:47:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.983+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8003ad0 con 0x7fbaf8076ba0 2024-09-05T11:47:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.983+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae8035030 con 0x7fbaf8076ba0 2024-09-05T11:47:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.983+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf819db30 con 0x7fbaf8076ba0 2024-09-05T11:47:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.983+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a0820 con 0x7fbaf8076ba0 2024-09-05T11:47:26.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.984+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf810e470 con 0x7fbaf8076ba0 2024-09-05T11:47:26.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.984+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbae8042070 con 0x7fbaf8076ba0 2024-09-05T11:47:26.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.985+0000 7fbaeea00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 0x7fbadc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:26.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.985+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbae806ced0 con 0x7fbaf8076ba0 2024-09-05T11:47:26.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.985+0000 7fbaf5400640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 0x7fbadc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:26.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.986+0000 7fbaf5400640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 0x7fbadc03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbae40098c0 tx=0x7fbae4006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:26.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:26.990+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbae803a3c0 con 0x7fbaf8076ba0 2024-09-05T11:47:27.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.179+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fbaf802d830 con 0x7fbaf8076ba0 2024-09-05T11:47:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.181+0000 7fbaeea00640 1 -- 172.21.15.59:0/1357442873 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fbae8030d90 con 0x7fbaf8076ba0 2024-09-05T11:47:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 msgr2=0x7fbadc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 0x7fbadc03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbae40098c0 tx=0x7fbae4006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 msgr2=0x7fbaf819d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbae8004f60 tx=0x7fbae80048f0 comp rx=0 tx=0).stop 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 shutdown_connections 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbadc0390c0 0x7fbadc03b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 --2- 172.21.15.59:0/1357442873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaf8076ba0 0x7fbaf819d360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 >> 172.21.15.59:0/1357442873 conn(0x7fbaf80fefc0 msgr2=0x7fbaf80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 shutdown_connections 2024-09-05T11:47:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:27.184+0000 7fbafcc00640 1 -- 172.21.15.59:0/1357442873 wait complete. 2024-09-05T11:47:28.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:28 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1357442873' entity='client.admin' 2024-09-05T11:47:28.416 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-05T11:47:28.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-05T11:47:29.025 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.519+0000 7fb4a1200640 1 -- 172.21.15.59:0/2198054617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 msgr2=0x7fb49c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.519+0000 7fb4a1200640 1 --2- 172.21.15.59:0/2198054617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c103bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009880 tx=0x7fb48c02a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.519+0000 7fb4a1200640 1 -- 172.21.15.59:0/2198054617 shutdown_connections 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.519+0000 7fb4a1200640 1 --2- 172.21.15.59:0/2198054617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c103bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.519+0000 7fb4a1200640 1 -- 172.21.15.59:0/2198054617 >> 172.21.15.59:0/2198054617 conn(0x7fb49c0fefc0 msgr2=0x7fb49c101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.520+0000 7fb4a1200640 1 -- 172.21.15.59:0/2198054617 shutdown_connections 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.520+0000 7fb4a1200640 1 -- 172.21.15.59:0/2198054617 wait complete. 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.520+0000 7fb4a1200640 1 Processor -- start 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb4a1200640 1 -- start start 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb4a1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb4a1200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c109900 con 0x7fb49c1037f0 2024-09-05T11:47:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb499e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb499e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54990/0 (socket says 172.21.15.59:54990) 2024-09-05T11:47:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.521+0000 7fb499e00640 1 -- 172.21.15.59:0/1174530600 learned_addr learned my addr 172.21.15.59:0/1174530600 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.522+0000 7fb499e00640 1 -- 172.21.15.59:0/1174530600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c1994e0 con 0x7fb49c1037f0 2024-09-05T11:47:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.522+0000 7fb499e00640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb48c004f60 tx=0x7fb48c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.522+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb48c03f070 con 0x7fb49c1037f0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.522+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c004cc0 con 0x7fb49c1037f0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.522+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c199770 con 0x7fb49c1037f0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.523+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb48c035030 con 0x7fb49c1037f0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.523+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c19c460 con 0x7fb49c1037f0 2024-09-05T11:47:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.524+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c10a080 con 0x7fb49c1037f0 2024-09-05T11:47:29.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.524+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb48c04c060 con 0x7fb49c1037f0 2024-09-05T11:47:29.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.525+0000 7fb492a00640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 0x7fb48003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:29.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.525+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb48c04a070 con 0x7fb49c1037f0 2024-09-05T11:47:29.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.525+0000 7fb499400640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 0x7fb48003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:29.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.526+0000 7fb499400640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 0x7fb48003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098c0 tx=0x7fb488006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.533+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb48c03be20 con 0x7fb49c1037f0 2024-09-05T11:47:29.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.724+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb49c02d830 con 0x7fb480039060 2024-09-05T11:47:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.728+0000 7fb492a00640 1 -- 172.21.15.59:0/1174530600 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb49c02d830 con 0x7fb480039060 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 msgr2=0x7fb48003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 0x7fb48003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098c0 tx=0x7fb488006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 msgr2=0x7fb49c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb48c004f60 tx=0x7fb48c0048f0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 shutdown_connections 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb480039060 0x7fb48003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 --2- 172.21.15.59:0/1174530600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb49c1037f0 0x7fb49c198fa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 >> 172.21.15.59:0/1174530600 conn(0x7fb49c0fefc0 msgr2=0x7fb49c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 shutdown_connections 2024-09-05T11:47:29.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:29.730+0000 7fb4a1200640 1 -- 172.21.15.59:0/1174530600 wait complete. 2024-09-05T11:47:30.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:30 smithi059 ceph-mon[30025]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:30.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:31.132 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi089 2024-09-05T11:47:31.132 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:47:31.132 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.conf 2024-09-05T11:47:31.152 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:47:31.152 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:31.210 INFO:tasks.cephadm:Adding host smithi089 to orchestrator... 2024-09-05T11:47:31.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch host add smithi089 2024-09-05T11:47:31.603 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-05T11:47:32.089 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:31 smithi059 ceph-mon[30025]: Updating smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:47:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.209+0000 7ff21ee00640 1 -- 172.21.15.59:0/485302163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 msgr2=0x7ff218103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.209+0000 7ff21ee00640 1 --2- 172.21.15.59:0/485302163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218103bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff208009880 tx=0x7ff20802a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.210+0000 7ff21ee00640 1 -- 172.21.15.59:0/485302163 shutdown_connections 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.210+0000 7ff21ee00640 1 --2- 172.21.15.59:0/485302163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218103bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.210+0000 7ff21ee00640 1 -- 172.21.15.59:0/485302163 >> 172.21.15.59:0/485302163 conn(0x7ff2180fefc0 msgr2=0x7ff218101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.210+0000 7ff21ee00640 1 -- 172.21.15.59:0/485302163 shutdown_connections 2024-09-05T11:47:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.210+0000 7ff21ee00640 1 -- 172.21.15.59:0/485302163 wait complete. 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff21ee00640 1 Processor -- start 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff21ee00640 1 -- start start 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff21ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff21ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff218109900 con 0x7ff2181037f0 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff217e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff217e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35860/0 (socket says 172.21.15.59:35860) 2024-09-05T11:47:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.211+0000 7ff217e00640 1 -- 172.21.15.59:0/1555531691 learned_addr learned my addr 172.21.15.59:0/1555531691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff217e00640 1 -- 172.21.15.59:0/1555531691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181994b0 con 0x7ff2181037f0 2024-09-05T11:47:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff217e00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff208004f60 tx=0x7ff2080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20803f070 con 0x7ff2181037f0 2024-09-05T11:47:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208004cc0 con 0x7ff2181037f0 2024-09-05T11:47:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff208035030 con 0x7ff2181037f0 2024-09-05T11:47:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.212+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff218199740 con 0x7ff2181037f0 2024-09-05T11:47:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.213+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21819c430 con 0x7ff2181037f0 2024-09-05T11:47:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.214+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21810a080 con 0x7ff2181037f0 2024-09-05T11:47:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.214+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff208042070 con 0x7ff2181037f0 2024-09-05T11:47:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.214+0000 7ff215600640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 0x7ff1fc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.214+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff20806cdd0 con 0x7ff2181037f0 2024-09-05T11:47:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.215+0000 7ff20fe00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 0x7ff1fc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.215+0000 7ff20fe00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 0x7ff1fc03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff2000098c0 tx=0x7ff200006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:32.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.219+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff208040260 con 0x7ff2181037f0 2024-09-05T11:47:32.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:32.431+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}) -- 0x7ff21802d830 con 0x7ff1fc0390c0 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: Updating smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:33.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:33.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:33.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:47:33.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:34.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:34 smithi059 ceph-mon[30025]: Deploying cephadm binary to smithi089 2024-09-05T11:47:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.262+0000 7ff215600640 1 -- 172.21.15.59:0/1555531691 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff21802d830 con 0x7ff1fc0390c0 2024-09-05T11:47:36.264 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi089' with addr '172.21.15.89' 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 msgr2=0x7ff1fc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 0x7ff1fc03b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff2000098c0 tx=0x7ff200006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 msgr2=0x7ff218198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff208004f60 tx=0x7ff2080048f0 comp rx=0 tx=0).stop 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 shutdown_connections 2024-09-05T11:47:36.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1fc0390c0 0x7ff1fc03b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 --2- 172.21.15.59:0/1555531691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2181037f0 0x7ff218198f70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 >> 172.21.15.59:0/1555531691 conn(0x7ff2180fefc0 msgr2=0x7ff2180ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 shutdown_connections 2024-09-05T11:47:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:36.265+0000 7ff21ee00640 1 -- 172.21.15.59:0/1555531691 wait complete. 2024-09-05T11:47:36.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T11:47:37.191 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:37 smithi059 ceph-mon[30025]: Added host smithi089 2024-09-05T11:47:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 -- 172.21.15.59:0/2722612929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 msgr2=0x7f9d10107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 --2- 172.21.15.59:0/2722612929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10107a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009880 tx=0x7f9d0002a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 -- 172.21.15.59:0/2722612929 shutdown_connections 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 --2- 172.21.15.59:0/2722612929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10107a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 -- 172.21.15.59:0/2722612929 >> 172.21.15.59:0/2722612929 conn(0x7f9d100fd230 msgr2=0x7f9d100ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.699+0000 7f9d17200640 1 -- 172.21.15.59:0/2722612929 shutdown_connections 2024-09-05T11:47:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.700+0000 7f9d17200640 1 -- 172.21.15.59:0/2722612929 wait complete. 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.701+0000 7f9d17200640 1 Processor -- start 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.701+0000 7f9d17200640 1 -- start start 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.701+0000 7f9d17200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.701+0000 7f9d17200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1010bc70 con 0x7f9d10107640 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.701+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35886/0 (socket says 172.21.15.59:35886) 2024-09-05T11:47:37.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0fe00640 1 -- 172.21.15.59:0/1875515385 learned_addr learned my addr 172.21.15.59:0/1875515385 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:37.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0fe00640 1 -- 172.21.15.59:0/1875515385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10199840 con 0x7f9d10107640 2024-09-05T11:47:37.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0fe00640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009230 tx=0x7f9d00003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:37.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0003f070 con 0x7f9d10107640 2024-09-05T11:47:37.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d0003bd40 con 0x7f9d10107640 2024-09-05T11:47:37.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.702+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d00035030 con 0x7f9d10107640 2024-09-05T11:47:37.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.703+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d10199ad0 con 0x7f9d10107640 2024-09-05T11:47:37.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.703+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1019dbd0 con 0x7f9d10107640 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.704+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9d00042070 con 0x7f9d10107640 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.704+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d10102cb0 con 0x7f9d10107640 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.704+0000 7f9d0cc00640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 0x7f9cf403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.704+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9d0006ccd0 con 0x7f9d10107640 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.704+0000 7f9d0f400640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 0x7f9cf403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:37.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.705+0000 7f9d0f400640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 0x7f9cf403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098c0 tx=0x7f9cfc006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:37.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.710+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d000412c0 con 0x7f9d10107640 2024-09-05T11:47:37.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.901+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d1019a660 con 0x7f9cf40390c0 2024-09-05T11:47:37.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.902+0000 7f9d0cc00640 1 -- 172.21.15.59:0/1875515385 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f9d1019a660 con 0x7f9cf40390c0 2024-09-05T11:47:37.904 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:37.904 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}] 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 msgr2=0x7f9cf403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 0x7f9cf403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098c0 tx=0x7f9cfc006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 msgr2=0x7f9d10199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009230 tx=0x7f9d00003d80 comp rx=0 tx=0).stop 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 shutdown_connections 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9cf40390c0 0x7f9cf403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 --2- 172.21.15.59:0/1875515385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d10107640 0x7f9d10199300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 >> 172.21.15.59:0/1875515385 conn(0x7f9d100fd230 msgr2=0x7f9d10075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 shutdown_connections 2024-09-05T11:47:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:37.905+0000 7f9d17200640 1 -- 172.21.15.59:0/1875515385 wait complete. 2024-09-05T11:47:38.522 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-05T11:47:38.523 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:47:38.523 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-05T11:47:38.542 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:47:38.542 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:38.600 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-05T11:47:38.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch host add smithi189 2024-09-05T11:47:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:38 smithi059 ceph-mon[30025]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 -- 172.21.15.59:0/313050789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 msgr2=0x7fd448109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 --2- 172.21.15.59:0/313050789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd448109b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd438009c20 tx=0x7fd43802aa90 comp rx=0 tx=0).stop 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 -- 172.21.15.59:0/313050789 shutdown_connections 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 --2- 172.21.15.59:0/313050789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd448109b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 -- 172.21.15.59:0/313050789 >> 172.21.15.59:0/313050789 conn(0x7fd4480ff450 msgr2=0x7fd448101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 -- 172.21.15.59:0/313050789 shutdown_connections 2024-09-05T11:47:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.464+0000 7fd44d000640 1 -- 172.21.15.59:0/313050789 wait complete. 2024-09-05T11:47:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.465+0000 7fd44d000640 1 Processor -- start 2024-09-05T11:47:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.465+0000 7fd44d000640 1 -- start start 2024-09-05T11:47:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.465+0000 7fd44d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.465+0000 7fd44d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd44810de30 con 0x7fd448109770 2024-09-05T11:47:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.466+0000 7fd445e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.466+0000 7fd445e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35896/0 (socket says 172.21.15.59:35896) 2024-09-05T11:47:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.466+0000 7fd445e00640 1 -- 172.21.15.59:0/3714504931 learned_addr learned my addr 172.21.15.59:0/3714504931 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.466+0000 7fd445e00640 1 -- 172.21.15.59:0/3714504931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4481998e0 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.466+0000 7fd445e00640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d40 tx=0x7fd43802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.467+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd438047070 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.467+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438003e40 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.467+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd438033050 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.467+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd448199b70 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.467+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd448199fd0 con 0x7fd448109770 2024-09-05T11:47:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.468+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd43804a070 con 0x7fd448109770 2024-09-05T11:47:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.468+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd448104eb0 con 0x7fd448109770 2024-09-05T11:47:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.468+0000 7fd43f400640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 0x7fd42c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.469+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd43803e030 con 0x7fd448109770 2024-09-05T11:47:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.469+0000 7fd43d400640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 0x7fd42c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.469+0000 7fd43d400640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 0x7fd42c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098c0 tx=0x7fd430006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.474+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd43802e3c0 con 0x7fd448109770 2024-09-05T11:47:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:39.662+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7fd44819a700 con 0x7fd42c0390c0 2024-09-05T11:47:39.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:39 smithi059 ceph-mon[30025]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:40.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:47:40.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:40.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:47:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:41 smithi059 ceph-mon[30025]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-05T11:47:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:41 smithi059 ceph-mon[30025]: Updating smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:47:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:42 smithi059 ceph-mon[30025]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:42 smithi059 ceph-mon[30025]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:42 smithi059 ceph-mon[30025]: Deploying cephadm binary to smithi189 2024-09-05T11:47:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:42 smithi059 ceph-mon[30025]: Updating smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T11:47:43.632 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-05T11:47:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.630+0000 7fd43f400640 1 -- 172.21.15.59:0/3714504931 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd44819a700 con 0x7fd42c0390c0 2024-09-05T11:47:43.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 msgr2=0x7fd42c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:43.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 0x7fd42c03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098c0 tx=0x7fd430006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:43.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 msgr2=0x7fd4481993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:43.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d40 tx=0x7fd43802fc10 comp rx=0 tx=0).stop 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 shutdown_connections 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd42c0390c0 0x7fd42c03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 --2- 172.21.15.59:0/3714504931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd448109770 0x7fd4481993a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 >> 172.21.15.59:0/3714504931 conn(0x7fd4480ff450 msgr2=0x7fd448107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 shutdown_connections 2024-09-05T11:47:43.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:43.633+0000 7fd44d000640 1 -- 172.21.15.59:0/3714504931 wait complete. 2024-09-05T11:47:44.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T11:47:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:44.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: Added host smithi189 2024-09-05T11:47:44.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:44.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:44.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:45.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 -- 172.21.15.59:0/587462356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 msgr2=0x7f8f4c109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:45.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 --2- 172.21.15.59:0/587462356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c109b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009880 tx=0x7f8f3c02a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:45.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 -- 172.21.15.59:0/587462356 shutdown_connections 2024-09-05T11:47:45.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 --2- 172.21.15.59:0/587462356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c109b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:45.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 -- 172.21.15.59:0/587462356 >> 172.21.15.59:0/587462356 conn(0x7f8f4c0ff450 msgr2=0x7f8f4c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 -- 172.21.15.59:0/587462356 shutdown_connections 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.228+0000 7f8f52400640 1 -- 172.21.15.59:0/587462356 wait complete. 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.230+0000 7f8f52400640 1 Processor -- start 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.230+0000 7f8f52400640 1 -- start start 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.230+0000 7f8f52400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:45.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.230+0000 7f8f52400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c10de30 con 0x7f8f4c109770 2024-09-05T11:47:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.231+0000 7f8f4b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.231+0000 7f8f4b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52720/0 (socket says 172.21.15.59:52720) 2024-09-05T11:47:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.231+0000 7f8f4b400640 1 -- 172.21.15.59:0/2586340836 learned_addr learned my addr 172.21.15.59:0/2586340836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.231+0000 7f8f4b400640 1 -- 172.21.15.59:0/2586340836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c1998e0 con 0x7f8f4c109770 2024-09-05T11:47:45.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f4b400640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009230 tx=0x7f8f3c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c03f070 con 0x7f8f4c109770 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c03bc80 con 0x7f8f4c109770 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c035030 con 0x7f8f4c109770 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c199bd0 con 0x7f8f4c109770 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.232+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c19dc70 con 0x7f8f4c109770 2024-09-05T11:47:45.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.233+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f3c042070 con 0x7f8f4c109770 2024-09-05T11:47:45.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.233+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c104eb0 con 0x7f8f4c109770 2024-09-05T11:47:45.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.234+0000 7f8f48c00640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 0x7f8f3003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:45.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.234+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f3c06c9f0 con 0x7f8f4c109770 2024-09-05T11:47:45.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.234+0000 7f8f43e00640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 0x7f8f3003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:45.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.235+0000 7f8f43e00640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 0x7f8f3003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8f340098c0 tx=0x7f8f34006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:45.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.239+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f3c0372d0 con 0x7f8f4c109770 2024-09-05T11:47:45.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.429+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f4c109c30 con 0x7f8f300390c0 2024-09-05T11:47:45.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.431+0000 7f8f48c00640 1 -- 172.21.15.59:0/2586340836 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8f4c109c30 con 0x7f8f300390c0 2024-09-05T11:47:45.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:47:45.432 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-05T11:47:45.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 msgr2=0x7f8f3003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:45.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 0x7f8f3003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8f340098c0 tx=0x7f8f34006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 msgr2=0x7f8f4c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009230 tx=0x7f8f3c003d40 comp rx=0 tx=0).stop 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 shutdown_connections 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f300390c0 0x7f8f3003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 --2- 172.21.15.59:0/2586340836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f4c109770 0x7f8f4c1993a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 >> 172.21.15.59:0/2586340836 conn(0x7f8f4c0ff450 msgr2=0x7f8f4c107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 shutdown_connections 2024-09-05T11:47:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:45.433+0000 7f8f52400640 1 -- 172.21.15.59:0/2586340836 wait complete. 2024-09-05T11:47:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:45 smithi059 ceph-mon[30025]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:46.069 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:45 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:46.069 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:45 smithi059 ceph-mon[30025]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:47:46.106 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-05T11:47:46.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd crush tunables default 2024-09-05T11:47:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:47:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.011+0000 7fd697200640 1 -- 172.21.15.59:0/1920677070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 msgr2=0x7fd690107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.011+0000 7fd697200640 1 --2- 172.21.15.59:0/1920677070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd680009880 tx=0x7fd68002a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.012+0000 7fd697200640 1 -- 172.21.15.59:0/1920677070 shutdown_connections 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.012+0000 7fd697200640 1 --2- 172.21.15.59:0/1920677070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.012+0000 7fd697200640 1 -- 172.21.15.59:0/1920677070 >> 172.21.15.59:0/1920677070 conn(0x7fd6900fd290 msgr2=0x7fd6900ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.012+0000 7fd697200640 1 -- 172.21.15.59:0/1920677070 shutdown_connections 2024-09-05T11:47:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.012+0000 7fd697200640 1 -- 172.21.15.59:0/1920677070 wait complete. 2024-09-05T11:47:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.013+0000 7fd697200640 1 Processor -- start 2024-09-05T11:47:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.013+0000 7fd697200640 1 -- start start 2024-09-05T11:47:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd697200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd697200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd69010bc90 con 0x7fd690107660 2024-09-05T11:47:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52742/0 (socket says 172.21.15.59:52742) 2024-09-05T11:47:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd68fe00640 1 -- 172.21.15.59:0/1252131333 learned_addr learned my addr 172.21.15.59:0/1252131333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:47:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.014+0000 7fd68fe00640 1 -- 172.21.15.59:0/1252131333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd690199880 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd68fe00640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd680009230 tx=0x7fd680003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd68003f070 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68003bd40 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd680035030 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd690199b10 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.015+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69019dc10 con 0x7fd690107660 2024-09-05T11:47:47.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.016+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd680042070 con 0x7fd690107660 2024-09-05T11:47:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.016+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd690102cd0 con 0x7fd690107660 2024-09-05T11:47:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.016+0000 7fd68cc00640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.017+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd68006ccd0 con 0x7fd690107660 2024-09-05T11:47:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.017+0000 7fd68f400640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.017+0000 7fd68f400640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 0x7fd67403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0098c0 tx=0x7fd67c006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:47.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.021+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6800412c0 con 0x7fd690107660 2024-09-05T11:47:47.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.209+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd69019a6a0 con 0x7fd690107660 2024-09-05T11:47:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.982+0000 7fd68cc00640 1 -- 172.21.15.59:0/1252131333 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd68003b860 con 0x7fd690107660 2024-09-05T11:47:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-09-05T11:47:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.984+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 msgr2=0x7fd67403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.984+0000 7fd697200640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 0x7fd67403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0098c0 tx=0x7fd67c006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.984+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 msgr2=0x7fd690199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.984+0000 7fd697200640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd680009230 tx=0x7fd680003d80 comp rx=0 tx=0).stop 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 shutdown_connections 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 --2- 172.21.15.59:0/1252131333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd690107660 0x7fd690199340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 >> 172.21.15.59:0/1252131333 conn(0x7fd6900fd290 msgr2=0x7fd690075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 shutdown_connections 2024-09-05T11:47:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:47:47.985+0000 7fd697200640 1 -- 172.21.15.59:0/1252131333 wait complete. 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1252131333' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:48.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:47:48.666 INFO:tasks.cephadm:Adding mon.a on smithi059 2024-09-05T11:47:48.666 INFO:tasks.cephadm:Adding mon.b on smithi089 2024-09-05T11:47:48.666 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-09-05T11:47:48.667 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch apply mon '3;smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c' 2024-09-05T11:47:48.908 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:47:49.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 -- 172.21.15.189:0/178392312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 msgr2=0x7f241c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:49.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 --2- 172.21.15.189:0/178392312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c107a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2418009880 tx=0x7f241802a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 -- 172.21.15.189:0/178392312 shutdown_connections 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 --2- 172.21.15.189:0/178392312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c107a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 -- 172.21.15.189:0/178392312 >> 172.21.15.189:0/178392312 conn(0x7f241c0fd260 msgr2=0x7f241c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 -- 172.21.15.189:0/178392312 shutdown_connections 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 -- 172.21.15.189:0/178392312 wait complete. 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.316+0000 7f2424460640 1 Processor -- start 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f2424460640 1 -- start start 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f2424460640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f2424460640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10bca0 con 0x7f241c107670 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f24221fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f24221fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.189:56880/0 (socket says 172.21.15.189:56880) 2024-09-05T11:47:49.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.317+0000 7f24221fc640 1 -- 172.21.15.189:0/673956993 learned_addr learned my addr 172.21.15.189:0/673956993 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:47:49.320 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f24221fc640 1 -- 172.21.15.189:0/673956993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c0763e0 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f24221fc640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2418009230 tx=0x7f2418003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241803f070 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241803bd40 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c074a30 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2418035030 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.318+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c074ef0 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.319+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2418042070 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.319+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c102ce0 con 0x7f241c107670 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.319+0000 7f24137fe640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 0x7f240803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:49.321 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.319+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f241806c950 con 0x7f241c107670 2024-09-05T11:47:49.322 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.320+0000 7f24219fb640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 0x7f240803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:49.322 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.320+0000 7f24219fb640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 0x7f240803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098c0 tx=0x7f240c006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:49.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.325+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2418044e10 con 0x7f241c107670 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1252131333' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:47:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T11:47:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:49.510 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.508+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7f241c075ae0 con 0x7f24080390c0 2024-09-05T11:47:49.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.515+0000 7f24137fe640 1 -- 172.21.15.189:0/673956993 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f241c075ae0 con 0x7f24080390c0 2024-09-05T11:47:49.517 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 msgr2=0x7f240803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 0x7f240803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098c0 tx=0x7f240c006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 msgr2=0x7f241c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2418009230 tx=0x7f2418003d80 comp rx=0 tx=0).stop 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 shutdown_connections 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24080390c0 0x7f240803b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 --2- 172.21.15.189:0/673956993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f241c107670 0x7f241c0744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:49.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 >> 172.21.15.189:0/673956993 conn(0x7f241c0fd260 msgr2=0x7f241c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:49.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 shutdown_connections 2024-09-05T11:47:49.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:49.517+0000 7f2424460640 1 -- 172.21.15.189:0/673956993 wait complete. 2024-09-05T11:47:50.199 DEBUG:teuthology.orchestra.run.smithi089:mon.b> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.b.service 2024-09-05T11:47:50.202 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.c.service 2024-09-05T11:47:50.204 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T11:47:50.205 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T11:47:50.903 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:47:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c;count:3 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:47:50.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:47:50.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:47:50.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:47:50.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:50 smithi059 ceph-mon[30025]: Deploying daemon mon.c on smithi189 2024-09-05T11:47:51.637 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 -- 172.21.15.189:0/3838069680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 msgr2=0x7efe20107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:51.637 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 --2- 172.21.15.189:0/3838069680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe10009880 tx=0x7efe1002a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:51.637 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 -- 172.21.15.189:0/3838069680 shutdown_connections 2024-09-05T11:47:51.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 --2- 172.21.15.189:0/3838069680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:51.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 -- 172.21.15.189:0/3838069680 >> 172.21.15.189:0/3838069680 conn(0x7efe200fd2b0 msgr2=0x7efe200ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:51.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.635+0000 7efe256f1640 1 -- 172.21.15.189:0/3838069680 shutdown_connections 2024-09-05T11:47:51.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.636+0000 7efe256f1640 1 -- 172.21.15.189:0/3838069680 wait complete. 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.636+0000 7efe256f1640 1 Processor -- start 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.636+0000 7efe256f1640 1 -- start start 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe256f1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe256f1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010bc90 con 0x7efe20107660 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.189:48998/0 (socket says 172.21.15.189:48998) 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe1effd640 1 -- 172.21.15.189:0/456136110 learned_addr learned my addr 172.21.15.189:0/456136110 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:47:51.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.637+0000 7efe1effd640 1 -- 172.21.15.189:0/456136110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201976d0 con 0x7efe20107660 2024-09-05T11:47:51.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe1effd640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe10009230 tx=0x7efe10003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:51.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe1003f070 con 0x7efe20107660 2024-09-05T11:47:51.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe20197960 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1003bc80 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe10035030 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.638+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe20197dc0 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.639+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efe10042070 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.639+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20102cd0 con 0x7efe20107660 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.639+0000 7efe1cff9640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 0x7efe0c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:51.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.639+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe1006ccd0 con 0x7efe20107660 2024-09-05T11:47:51.642 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.639+0000 7efe167fc640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 0x7efe0c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:51.642 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.640+0000 7efe167fc640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 0x7efe0c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe080098c0 tx=0x7efe08006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:51.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.645+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe10031e20 con 0x7efe20107660 2024-09-05T11:47:51.882 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.881+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efe20198550 con 0x7efe20107660 2024-09-05T11:47:51.883 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.881+0000 7efe1cff9640 1 -- 172.21.15.189:0/456136110 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efe10031570 con 0x7efe20107660 2024-09-05T11:47:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:47:51.883 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","modified":"2024-09-05T11:46:20.317245Z","created":"2024-09-05T11:46:20.317245Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T11:47:51.884 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-05T11:47:51.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 msgr2=0x7efe0c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:51.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 0x7efe0c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe080098c0 tx=0x7efe08006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:51.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 msgr2=0x7efe20197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:51.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe10009230 tx=0x7efe10003d40 comp rx=0 tx=0).stop 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 shutdown_connections 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efe0c039060 0x7efe0c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 --2- 172.21.15.189:0/456136110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe20107660 0x7efe20197190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.883+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 >> 172.21.15.189:0/456136110 conn(0x7efe200fd2b0 msgr2=0x7efe200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.884+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 shutdown_connections 2024-09-05T11:47:51.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:51.884+0000 7efe256f1640 1 -- 172.21.15.189:0/456136110 wait complete. 2024-09-05T11:47:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:52 smithi059 ceph-mon[30025]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:52 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/456136110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:47:53.745 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T11:47:53.745 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T11:47:54.000 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:47:54.529 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 -- 172.21.15.189:0/464633952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 msgr2=0x7f5f20106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:54.529 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 --2- 172.21.15.189:0/464633952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20106a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009880 tx=0x7f5f1002a5d0 comp rx=0 tx=0).stop 2024-09-05T11:47:54.529 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 -- 172.21.15.189:0/464633952 shutdown_connections 2024-09-05T11:47:54.529 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 --2- 172.21.15.189:0/464633952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20106a10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:54.529 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 -- 172.21.15.189:0/464633952 >> 172.21.15.189:0/464633952 conn(0x7f5f200fc220 msgr2=0x7f5f200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 -- 172.21.15.189:0/464633952 shutdown_connections 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 -- 172.21.15.189:0/464633952 wait complete. 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.527+0000 7f5f26105640 1 Processor -- start 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f26105640 1 -- start start 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f26105640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:54.530 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f26105640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2010ac60 con 0x7f5f20106630 2024-09-05T11:47:54.531 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:54.531 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.189:49018/0 (socket says 172.21.15.189:49018) 2024-09-05T11:47:54.531 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.528+0000 7f5f1f7fe640 1 -- 172.21.15.189:0/532804188 learned_addr learned my addr 172.21.15.189:0/532804188 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:47:54.531 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f1f7fe640 1 -- 172.21.15.189:0/532804188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f20197780 con 0x7f5f20106630 2024-09-05T11:47:54.531 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f1f7fe640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:54.532 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1003f070 con 0x7f5f20106630 2024-09-05T11:47:54.532 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f20197a10 con 0x7f5f20106630 2024-09-05T11:47:54.532 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1003bc80 con 0x7f5f20106630 2024-09-05T11:47:54.532 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10035030 con 0x7f5f20106630 2024-09-05T11:47:54.532 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.529+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2019bb10 con 0x7f5f20106630 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.530+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f20101ca0 con 0x7f5f20106630 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.530+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5f10042070 con 0x7f5f20106630 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.530+0000 7f5f1cff9640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 0x7f5f0803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.530+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5f1006cdd0 con 0x7f5f20106630 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.531+0000 7f5f1effd640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 0x7f5f0803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:47:54.533 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.531+0000 7f5f1effd640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 0x7f5f0803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098c0 tx=0x7f5f14006db0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:47:54.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.536+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f100372d0 con 0x7f5f20106630 2024-09-05T11:47:54.797 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.795+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5f20106a90 con 0x7f5f20106630 2024-09-05T11:47:54.798 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.796+0000 7f5f1cff9640 1 -- 172.21.15.189:0/532804188 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5f10031570 con 0x7f5f20106630 2024-09-05T11:47:54.798 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:47:54.798 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","modified":"2024-09-05T11:46:20.317245Z","created":"2024-09-05T11:46:20.317245Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T11:47:54.798 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 msgr2=0x7f5f0803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 0x7f5f0803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098c0 tx=0x7f5f14006db0 comp rx=0 tx=0).stop 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 msgr2=0x7f5f20197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10003d40 comp rx=0 tx=0).stop 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 shutdown_connections 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f08039060 0x7f5f0803b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 --2- 172.21.15.189:0/532804188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20106630 0x7f5f20197240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 >> 172.21.15.189:0/532804188 conn(0x7f5f200fc220 msgr2=0x7f5f2006a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:47:54.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 shutdown_connections 2024-09-05T11:47:54.801 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:47:54.798+0000 7f5f26105640 1 -- 172.21.15.189:0/532804188 wait complete. 2024-09-05T11:47:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:47:54 smithi059 ceph-mon[30025]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:47:55.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:47:55 smithi189 ceph-mon[34772]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T11:47:55.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:47:55 smithi189 bash[34740]: 71c5ef535066c20ac73bf738235203241c1a878122df8c904c1f7d1314feb17b 2024-09-05T11:47:55.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:47:55 smithi189 systemd[1]: Started Ceph mon.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:47:56.506 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T11:47:56.506 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T11:47:56.764 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:48:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 -- 172.21.15.189:0/331112792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 msgr2=0x7f1d18005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 --2- 172.21.15.189:0/331112792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d18005580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d28011c50 tx=0x7f1d2802be50 comp rx=0 tx=0).stop 2024-09-05T11:48:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 -- 172.21.15.189:0/331112792 shutdown_connections 2024-09-05T11:48:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 --2- 172.21.15.189:0/331112792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d18005580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 -- 172.21.15.189:0/331112792 >> 172.21.15.189:0/331112792 conn(0x7f1d2c0fc220 msgr2=0x7f1d2c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 -- 172.21.15.189:0/331112792 shutdown_connections 2024-09-05T11:48:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.218+0000 7f1d33a78640 1 -- 172.21.15.189:0/331112792 wait complete. 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.219+0000 7f1d33a78640 1 Processor -- start 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.219+0000 7f1d33a78640 1 -- start start 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d33a78640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d2c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d33a78640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d33a78640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1075a0 con 0x7f1d2c106630 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d33a78640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c107420 con 0x7f1d2c1999f0 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:60086/0 (socket says 172.21.15.189:60086) 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 -- 172.21.15.189:0/2943870190 learned_addr learned my addr 172.21.15.189:0/2943870190 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 -- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 msgr2=0x7f1d2c19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 -- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 msgr2=0x7f1d2c19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T11:48:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31814640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d2c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31013640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31814640 1 -- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 msgr2=0x7f1d2c19dd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.220+0000 7f1d31814640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d31814640 1 -- 172.21.15.189:0/2943870190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c19a010 con 0x7f1d2c106630 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d31814640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d2c1994b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d28011c50 tx=0x7f1d28005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1d280055f0 con 0x7f1d2c106630 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d28043070 con 0x7f1d2c106630 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c19e2c0 con 0x7f1d2c106630 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1d28004c90 con 0x7f1d2c106630 2024-09-05T11:48:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.221+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c19e880 con 0x7f1d2c106630 2024-09-05T11:48:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.222+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1d28048460 con 0x7f1d2c106630 2024-09-05T11:48:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.222+0000 7f1d1effd640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 0x7f1d1003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.222+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c101ca0 con 0x7f1d2c106630 2024-09-05T11:48:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.223+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d2803b040 con 0x7f1d2c106630 2024-09-05T11:48:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.223+0000 7f1d31013640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 0x7f1d1003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:00.226 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.223+0000 7f1d31013640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 0x7f1d1003b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009100 tx=0x7f1d20008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:00.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.228+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d280465c0 con 0x7f1d2c106630 2024-09-05T11:48:00.462 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.461+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d2c18ed70 con 0x7f1d2c106630 2024-09-05T11:48:00.463 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.461+0000 7f1d1effd640 1 -- 172.21.15.189:0/2943870190 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1d28046360 con 0x7f1d2c106630 2024-09-05T11:48:00.463 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:48:00.463 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","modified":"2024-09-05T11:47:55.141798Z","created":"2024-09-05T11:46:20.317245Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T11:48:00.463 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-05T11:48:00.465 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 msgr2=0x7f1d1003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:00.465 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 0x7f1d1003b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009100 tx=0x7f1d20008040 comp rx=0 tx=0).stop 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 msgr2=0x7f1d2c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d2c1994b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d28011c50 tx=0x7f1d28005270 comp rx=0 tx=0).stop 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 shutdown_connections 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d10038e00 0x7f1d1003b2c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d2c1999f0 0x7f1d2c19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 --2- 172.21.15.189:0/2943870190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c106630 0x7f1d2c1994b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 >> 172.21.15.189:0/2943870190 conn(0x7f1d2c0fc220 msgr2=0x7f1d2c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 shutdown_connections 2024-09-05T11:48:00.466 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:00.464+0000 7f1d33a78640 1 -- 172.21.15.189:0/2943870190 wait complete. 2024-09-05T11:48:00.466 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: mon.a calling monitor election 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: mon.c calling monitor election 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: monmap epoch 2 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: last_changed 2024-09-05T11:47:55.141798+0000 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: min_mon_release 19 (squid) 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: election_strategy: 1 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: fsmap 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: mgrmap e13: x(active, since 43s) 2024-09-05T11:48:00.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T11:48:00.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:00.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:00.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: mon.a calling monitor election 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: mon.c calling monitor election 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:00.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: monmap epoch 2 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: last_changed 2024-09-05T11:47:55.141798+0000 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: min_mon_release 19 (squid) 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: election_strategy: 1 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: fsmap 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: mgrmap e13: x(active, since 43s) 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T11:48:00.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:00.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:00.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:01 smithi189 ceph-mon[34772]: Deploying daemon mon.b on smithi089 2024-09-05T11:48:01.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:01 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/2943870190' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:01.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:01.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:01 smithi059 ceph-mon[30025]: Deploying daemon mon.b on smithi089 2024-09-05T11:48:01.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:01 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/2943870190' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:01.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:02.105 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T11:48:02.105 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T11:48:02.356 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:48:02.477 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:02 smithi189 ceph-mon[34772]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:02.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:02 smithi059 ceph-mon[30025]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:02.788 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.784+0000 7f68bb6bd640 1 -- 172.21.15.189:0/1664318390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b4107670 msgr2=0x7f68b4107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.784+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/1664318390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b4107670 0x7f68b4107a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009c20 tx=0x7f68b002aa90 comp rx=0 tx=0).stop 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.784+0000 7f68bb6bd640 1 -- 172.21.15.189:0/1664318390 shutdown_connections 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.784+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/1664318390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b4107670 0x7f68b4107a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.784+0000 7f68bb6bd640 1 -- 172.21.15.189:0/1664318390 >> 172.21.15.189:0/1664318390 conn(0x7f68b40fd330 msgr2=0x7f68b40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 -- 172.21.15.189:0/1664318390 shutdown_connections 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 -- 172.21.15.189:0/1664318390 wait complete. 2024-09-05T11:48:02.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 Processor -- start 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 -- start start 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 0x7f68b419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f68b410bd30 con 0x7f68b41978f0 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.785+0000 7f68bb6bd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68b410bbb0 con 0x7f68b4107670 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54948/0 (socket says 172.21.15.189:54948) 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 -- 172.21.15.189:0/2242516256 learned_addr learned my addr 172.21.15.189:0/2242516256 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:48:02.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 -- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 msgr2=0x7f68b41973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b8c58640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 0x7f68b419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 -- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 msgr2=0x7f68b41973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b9459640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b8c58640 1 -- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 msgr2=0x7f68b41973b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b8c58640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b8c58640 1 -- 172.21.15.189:0/2242516256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b4197f10 con 0x7f68b41978f0 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.786+0000 7f68b8c58640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 0x7f68b419bc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b3d0 tx=0x7f68a400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.787+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f68a4011630 con 0x7f68b41978f0 2024-09-05T11:48:02.791 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.787+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b419c1c0 con 0x7f68b41978f0 2024-09-05T11:48:02.792 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.787+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b419c700 con 0x7f68b41978f0 2024-09-05T11:48:02.792 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.787+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a4011cb0 con 0x7f68b41978f0 2024-09-05T11:48:02.792 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.787+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f68a40193b0 con 0x7f68b41978f0 2024-09-05T11:48:02.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.788+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898005290 con 0x7f68b41978f0 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.788+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f68a4018020 con 0x7f68b41978f0 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.788+0000 7f68aa7fc640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 0x7f68a003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.788+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f68a404bb10 con 0x7f68b41978f0 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.788+0000 7f68b9459640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 0x7f68a003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.789+0000 7f68b9459640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 0x7f68a003b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68b002b080 tx=0x7f68b0005dc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:02.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:02.793+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68a401f020 con 0x7f68b41978f0 2024-09-05T11:48:03.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.020+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6898005580 con 0x7f68b41978f0 2024-09-05T11:48:03.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.021+0000 7f68aa7fc640 1 -- 172.21.15.189:0/2242516256 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f68a4018570 con 0x7f68b41978f0 2024-09-05T11:48:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:48:03.023 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","modified":"2024-09-05T11:47:55.141798Z","created":"2024-09-05T11:46:20.317245Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T11:48:03.023 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-05T11:48:03.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 msgr2=0x7f68a003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 0x7f68a003b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68b002b080 tx=0x7f68b0005dc0 comp rx=0 tx=0).stop 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 msgr2=0x7f68b419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 0x7f68b419bc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b3d0 tx=0x7f68a400b890 comp rx=0 tx=0).stop 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 shutdown_connections 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f68a0038e50 0x7f68a003b310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68b41978f0 0x7f68b419bc80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 --2- 172.21.15.189:0/2242516256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b4107670 0x7f68b41973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 >> 172.21.15.189:0/2242516256 conn(0x7f68b40fd330 msgr2=0x7f68b40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 shutdown_connections 2024-09-05T11:48:03.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:03.024+0000 7f68bb6bd640 1 -- 172.21.15.189:0/2242516256 wait complete. 2024-09-05T11:48:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:03 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/2242516256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:03.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:03 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/2242516256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:04.008 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T11:48:04.008 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T11:48:04.008 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.env: 0x5619fabb8940 2024-09-05T11:48:04.008 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T11:48:04.008 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.info_log: 0x5619fd1016e0 2024-09-05T11:48:04.009 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T11:48:04.009 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.statistics: (nil) 2024-09-05T11:48:04.009 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.use_fsync: 0 2024-09-05T11:48:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_log_file_size: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_fallocate: 1 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.use_direct_reads: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.db_log_dir: 2024-09-05T11:48:04.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.wal_dir: 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.write_buffer_manager: 0x5619fd110dc0 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T11:48:04.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.unordered_write: 0 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.row_cache: None 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.wal_filter: None 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T11:48:04.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T11:48:04.439 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.two_write_queues: 0 2024-09-05T11:48:04.439 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.wal_compression: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.atomic_flush: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.log_readahead_size: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T11:48:04.440 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_background_jobs: 2 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_background_compactions: -1 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_subcompactions: 1 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T11:48:04.441 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_open_files: -1 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_background_flushes: -1 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Compression algorithms supported: 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kZSTD supported: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kXpressCompression supported: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kBZip2Compression supported: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kLZ4Compression supported: 1 2024-09-05T11:48:04.442 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kZlibCompression supported: 1 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: kSnappyCompression supported: 1 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.merge_operator: 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_filter: None 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T11:48:04.443 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5619fd101320) 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: cache_index_and_filter_blocks: 1 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: pin_top_level_index_and_filter: 1 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_type: 0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: data_block_index_type: 0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_shortening: 1 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: checksum: 4 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: no_block_cache: 0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache: 0x5619fd0f51f0 2024-09-05T11:48:04.444 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_name: BinnedLRUCache 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_options: 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: capacity : 536870912 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: num_shard_bits : 4 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: strict_capacity_limit : 0 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: high_pri_pool_ratio: 0.000 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_compressed: (nil) 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: persistent_cache: (nil) 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_size: 4096 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_size_deviation: 10 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_restart_interval: 16 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_block_restart_interval: 1 2024-09-05T11:48:04.445 INFO:journalctl@ceph.mon.b.smithi089.stdout: metadata_block_size: 4096 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: partition_filters: 0 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: use_delta_encoding: 1 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: filter_policy: bloomfilter 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: whole_key_filtering: 1 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: verify_compression: 0 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: read_amp_bytes_per_bit: 0 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: format_version: 5 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: enable_index_compression: 1 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_align: 0 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: max_auto_readahead_size: 262144 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: prepopulate_block_cache: 0 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: initial_auto_readahead_size: 8192 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T11:48:04.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression: NoCompression 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.num_levels: 7 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T11:48:04.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T11:48:04.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T11:48:04.449 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T11:48:04.450 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T11:48:04.451 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.inplace_update_support: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.bloom_locality: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.max_successive_merges: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.ttl: 2592000 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T11:48:04.452 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enable_blob_files: false 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.min_blob_size: 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-05T11:48:04.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 73439bb7-3ddd-4845-9986-a8ab939a5596 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536884007914, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536884008311, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725536884, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "73439bb7-3ddd-4845-9986-a8ab939a5596", "db_session_id": "5SXWCYDWCGJ1WHXMV62H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725536884008403, "job": 1, "event": "recovery_finished"} 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5619fd122e00 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: DB pointer 0x5619fd21e000 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** DB Stats ** 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T11:48:04.454 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T11:48:04.455 INFO:journalctl@ceph.mon.b.smithi089.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T11:48:04.456 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: Block cache BinnedLRUCache@0x5619fd0f51f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: using public_addr v2:172.21.15.89:0/0 -> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: starting mon.b rank -1 at public addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] at bind addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(???) e0 preinit fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).mds e1 new map 2024-09-05T11:48:04.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: e1 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: btime 2024-09-05T11:46:24:063581+0000 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: legacy client fscid: -1 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout: No filesystems configured 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-05T11:48:04.458 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1252131333' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.459 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1252131333' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.460 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c;count:3 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Deploying daemon mon.c on smithi189 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/456136110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:04.461 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.a calling monitor election 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.c calling monitor election 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.462 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: monmap epoch 2 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: last_changed 2024-09-05T11:47:55.141798+0000 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: min_mon_release 19 (squid) 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: election_strategy: 1 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: fsmap 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mgrmap e13: x(active, since 43s) 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.463 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: Deploying daemon mon.b on smithi089 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/2943870190' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/2242516256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 ceph-mon[34499]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 bash[34480]: 576bd771f338f6dc0e8772e5384b597b1af7df96e143a164e3ab52c9d3dd8b9c 2024-09-05T11:48:04.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:04 smithi089 systemd[1]: Started Ceph mon.b for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:48:04.662 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T11:48:04.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T11:48:04.912 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:48:09.186 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.184+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4213598627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a0106630 msgr2=0x7fe18c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:09.186 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.184+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4213598627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a0106630 0x7fe18c005580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe19c014d10 tx=0x7fe19c02c570 comp rx=0 tx=0).stop 2024-09-05T11:48:09.187 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.184+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4213598627 shutdown_connections 2024-09-05T11:48:09.187 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.184+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4213598627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a0106630 0x7fe18c005580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.187 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.184+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4213598627 >> 172.21.15.189:0/4213598627 conn(0x7fe1a00fc220 msgr2=0x7fe1a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:09.187 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.185+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4213598627 shutdown_connections 2024-09-05T11:48:09.187 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.185+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4213598627 wait complete. 2024-09-05T11:48:09.190 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 Processor -- start 2024-09-05T11:48:09.190 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 -- start start 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 0x7fe1a019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0108660 con 0x7fe1a019c0c0 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01084e0 con 0x7fe1a0106630 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a7d8c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01087e0 con 0x7fe1a01977f0 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54954/0 (socket says 172.21.15.189:54954) 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 -- 172.21.15.189:0/4006447237 learned_addr learned my addr 172.21.15.189:0/4006447237 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:48:09.191 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a6329640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 0x7fe1a019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 msgr2=0x7fe1a019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 msgr2=0x7fe1a019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5b28640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.186+0000 7fe1a5327640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a5b28640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 msgr2=0x7fe1a01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a5b28640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 msgr2=0x7fe1a01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a5b28640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a5b28640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 msgr2=0x7fe1a019bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:48:09.192 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 msgr2=0x7fe1a01972b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 -- 172.21.15.189:0/4006447237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a01a3e30 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a6329640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 0x7fe1a019c570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe19800bd50 tx=0x7fe19800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198019070 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe198014920 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a01a4060 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198012990 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.187+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a01a45f0 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.188+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe19801c070 con 0x7fe1a019c0c0 2024-09-05T11:48:09.193 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.188+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0109750 con 0x7fe1a019c0c0 2024-09-05T11:48:09.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.189+0000 7fe196ffd640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 0x7fe18403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:09.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.189+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe19804be40 con 0x7fe1a019c0c0 2024-09-05T11:48:09.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.189+0000 7fe1a5b28640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 0x7fe18403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:09.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.189+0000 7fe1a5b28640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 0x7fe18403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe19c004f80 tx=0x7fe19c005da0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:09.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.194+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe198050070 con 0x7fe1a019c0c0 2024-09-05T11:48:09.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.441+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe1a01a4950 con 0x7fe1a019c0c0 2024-09-05T11:48:09.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:09.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: mon.a calling monitor election 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: mon.c calling monitor election 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: monmap epoch 3 2024-09-05T11:48:09.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: min_mon_release 19 (squid) 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: election_strategy: 1 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: fsmap 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: mgrmap e13: x(active, since 52s) 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.442+0000 7fe196ffd640 1 -- 172.21.15.189:0/4006447237 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fe19800e090 con 0x7fe1a019c0c0 2024-09-05T11:48:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:48:09.446 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","modified":"2024-09-05T11:48:04.032507Z","created":"2024-09-05T11:46:20.317245Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 msgr2=0x7fe18403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 0x7fe18403b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe19c004f80 tx=0x7fe19c005da0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 msgr2=0x7fe1a019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 0x7fe1a019c570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe19800bd50 tx=0x7fe19800be40 comp rx=0 tx=0).stop 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 shutdown_connections 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1840394f0 0x7fe18403b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a019c0c0 0x7fe1a019c570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a01977f0 0x7fe1a019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 --2- 172.21.15.189:0/4006447237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1a0106630 0x7fe1a01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:09.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 >> 172.21.15.189:0/4006447237 conn(0x7fe1a00fc220 msgr2=0x7fe1a006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:09.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 shutdown_connections 2024-09-05T11:48:09.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:09.444+0000 7fe1a7d8c640 1 -- 172.21.15.189:0/4006447237 wait complete. 2024-09-05T11:48:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: mon.a calling monitor election 2024-09-05T11:48:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: mon.c calling monitor election 2024-09-05T11:48:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:09.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: monmap epoch 3 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: min_mon_release 19 (squid) 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: election_strategy: 1 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: fsmap 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: mgrmap e13: x(active, since 52s) 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T11:48:09.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:09.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:10.058 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-05T11:48:10.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph config generate-minimal-conf 2024-09-05T11:48:10.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:10 smithi189 ceph-mon[34772]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:10.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:10.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:10 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/4006447237' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:10.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:48:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:10 smithi059 ceph-mon[30025]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:10 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/4006447237' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 -- 172.21.15.59:0/468329936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc074510 msgr2=0x7fe8dc0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/468329936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc074510 0x7fe8dc0748f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009880 tx=0x7fe8cc02a670 comp rx=0 tx=0).stop 2024-09-05T11:48:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 -- 172.21.15.59:0/468329936 shutdown_connections 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/468329936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc074510 0x7fe8dc0748f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 -- 172.21.15.59:0/468329936 >> 172.21.15.59:0/468329936 conn(0x7fe8dc0fd290 msgr2=0x7fe8dc0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 -- 172.21.15.59:0/468329936 shutdown_connections 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.971+0000 7fe8e0c00640 1 -- 172.21.15.59:0/468329936 wait complete. 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.972+0000 7fe8e0c00640 1 Processor -- start 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.972+0000 7fe8e0c00640 1 -- start start 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.972+0000 7fe8e0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8e0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8e0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8dc19e430 0x7fe8dc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8e0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc10dea0 con 0x7fe8dc199b60 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8e0c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc10dd20 con 0x7fe8dc074510 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8e0c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc10e020 con 0x7fe8dc19e430 2024-09-05T11:48:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44818/0 (socket says 172.21.15.59:44818) 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 -- 172.21.15.59:0/289058918 learned_addr learned my addr 172.21.15.59:0/289058918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8da800640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8dc19e430 0x7fe8dc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9e00640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 msgr2=0x7fe8dc199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9e00640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 msgr2=0x7fe8dc199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9e00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9e00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8dc19e430 msgr2=0x7fe8dc19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8dc19e430 0x7fe8dc19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 msgr2=0x7fe8dc199620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.973+0000 7fe8d9400640 1 -- 172.21.15.59:0/289058918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc1a61a0 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8d9400640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800b520 tx=0x7fe8c800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004230 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c80043d0 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8003d60 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc1a63d0 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.974+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc1a6890 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.975+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8c800e070 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.976+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc074970 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.976+0000 7fe8d2a00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 0x7fe8c003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.976+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8c804ba20 con 0x7fe8dc199b60 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.976+0000 7fe8d9e00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 0x7fe8c003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.977+0000 7fe8d9e00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 0x7fe8c003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc005ab0 tx=0x7fe8cc005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:10.983+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8c8014020 con 0x7fe8dc199b60 2024-09-05T11:48:11.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.180+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe8dc1a6c80 con 0x7fe8dc199b60 2024-09-05T11:48:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.180+0000 7fe8d2a00640 1 -- 172.21.15.59:0/289058918 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fe8c8010d70 con 0x7fe8dc199b60 2024-09-05T11:48:11.182 INFO:teuthology.orchestra.run.smithi059.stdout:# minimal ceph.conf for 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:11.182 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-09-05T11:48:11.182 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:11.183 INFO:teuthology.orchestra.run.smithi059.stdout: mon_host = [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-09-05T11:48:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 msgr2=0x7fe8c003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 0x7fe8c003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc005ab0 tx=0x7fe8cc005a20 comp rx=0 tx=0).stop 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 msgr2=0x7fe8dc19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800b520 tx=0x7fe8c800b9e0 comp rx=0 tx=0).stop 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 shutdown_connections 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8c00394f0 0x7fe8c003b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8dc19e430 0x7fe8dc19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8dc199b60 0x7fe8dc19def0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 --2- 172.21.15.59:0/289058918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8dc074510 0x7fe8dc199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.183+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 >> 172.21.15.59:0/289058918 conn(0x7fe8dc0fd290 msgr2=0x7fe8dc1078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.184+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 shutdown_connections 2024-09-05T11:48:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:11.184+0000 7fe8e0c00640 1 -- 172.21.15.59:0/289058918 wait complete. 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.a calling monitor election 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.c calling monitor election 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: monmap epoch 3 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:11.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: min_mon_release 19 (squid) 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: election_strategy: 1 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: fsmap 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mgrmap e13: x(active, since 52s) 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:11.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:11.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:11.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:11.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:11.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/4006447237' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T11:48:11.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:11.960 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-05T11:48:11.960 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:48:11.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T11:48:12.021 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:48:12.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:48:12.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mon.b calling monitor election 2024-09-05T11:48:12.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:12.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-05T11:48:12.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mon.b calling monitor election 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mon.c calling monitor election 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mon.a calling monitor election 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: monmap epoch 3 2024-09-05T11:48:12.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:12.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:12.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:12.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: min_mon_release 19 (squid) 2024-09-05T11:48:12.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: election_strategy: 1 2024-09-05T11:48:12.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: fsmap 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: mgrmap e13: x(active, since 54s) 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T11:48:12.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/289058918' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: Updating smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.122 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:48:12.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T11:48:12.146 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.b calling monitor election 2024-09-05T11:48:12.146 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:12.146 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-05T11:48:12.146 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.b calling monitor election 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.c calling monitor election 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.a calling monitor election 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T11:48:12.147 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: monmap epoch 3 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: min_mon_release 19 (squid) 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: election_strategy: 1 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:12.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:12.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:12.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: fsmap 2024-09-05T11:48:12.149 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: mgrmap e13: x(active, since 54s) 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/289058918' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.150 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: Updating smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.154 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:48:12.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:48:12.220 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:48:12.220 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mon.b calling monitor election 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mon.b calling monitor election 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mon.c calling monitor election 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mon.a calling monitor election 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: monmap epoch 3 2024-09-05T11:48:12.244 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: last_changed 2024-09-05T11:48:04.032507+0000 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: created 2024-09-05T11:46:20.317245+0000 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: min_mon_release 19 (squid) 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: election_strategy: 1 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: fsmap 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T11:48:12.245 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: mgrmap e13: x(active, since 54s) 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/289058918' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: Updating smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.246 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.248 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:48:12.248 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T11:48:12.314 INFO:tasks.cephadm:Adding mgr.x on smithi059 2024-09-05T11:48:12.314 INFO:tasks.cephadm:Adding mgr.y on smithi089 2024-09-05T11:48:12.314 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-09-05T11:48:12.314 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch apply mgr '3;smithi059=x;smithi089=y;smithi189=z' 2024-09-05T11:48:12.593 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:48:12.968 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.968 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: Reconfiguring daemon mon.a on smithi059 2024-09-05T11:48:12.969 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:13.072 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 -- 172.21.15.189:0/3135433157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61841005f0 msgr2=0x7f61841009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:13.072 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 --2- 172.21.15.189:0/3135433157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61841005f0 0x7f61841009f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6174009880 tx=0x7f617402a5d0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 -- 172.21.15.189:0/3135433157 shutdown_connections 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 --2- 172.21.15.189:0/3135433157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61841005f0 0x7f61841009f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 -- 172.21.15.189:0/3135433157 >> 172.21.15.189:0/3135433157 conn(0x7f61840fbda0 msgr2=0x7f61840fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 -- 172.21.15.189:0/3135433157 shutdown_connections 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 -- 172.21.15.189:0/3135433157 wait complete. 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.045+0000 7f618a011640 1 Processor -- start 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 -- start start 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 0x7f618418c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:13.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f618418c920 0x7f61841a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6184106700 con 0x7f618418bf60 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6184106580 con 0x7f618418c920 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f618a011640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6184106880 con 0x7f61841005f0 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:48312/0 (socket says 172.21.15.189:48312) 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 -- 172.21.15.189:0/3230495428 learned_addr learned my addr 172.21.15.189:0/3230495428 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f6183fff640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f618418c920 0x7f61841a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f6182ffd640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 0x7f618418c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 -- 172.21.15.189:0/3230495428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f618418c920 msgr2=0x7f61841a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f618418c920 0x7f61841a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 -- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 msgr2=0x7f618418c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 0x7f618418c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 -- 172.21.15.189:0/3230495428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61841a15e0 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f6182ffd640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 0x7f618418c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.046+0000 7f61837fe640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ab0 tx=0x7f6174032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617403f030 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6174003ce0 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61841a1870 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174035070 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841a1dd0 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.047+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6174042030 con 0x7f61841005f0 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.048+0000 7f6180ff9640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 0x7f616c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:13.075 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.048+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f617403b4f0 con 0x7f61841005f0 2024-09-05T11:48:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.048+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61841a1fd0 con 0x7f61841005f0 2024-09-05T11:48:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.048+0000 7f6182ffd640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 0x7f616c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.049+0000 7f6182ffd640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 0x7f616c03b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6178006fd0 tx=0x7f6178008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.053+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f617403bdc0 con 0x7f61841005f0 2024-09-05T11:48:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:13.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:13.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: Reconfiguring daemon mon.a on smithi059 2024-09-05T11:48:13.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:13.186 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:48:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:48:13.031+0000 7faa31200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-05T11:48:13.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T11:48:13.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:13.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:13.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:13.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: Reconfiguring daemon mon.a on smithi059 2024-09-05T11:48:13.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:48:13.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.230+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi089=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f618406ab40 con 0x7f616c0394f0 2024-09-05T11:48:13.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.236+0000 7f6180ff9640 1 -- 172.21.15.189:0/3230495428 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f618406ab40 con 0x7f616c0394f0 2024-09-05T11:48:13.237 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 msgr2=0x7f616c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 0x7f616c03b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6178006fd0 tx=0x7f6178008040 comp rx=0 tx=0).stop 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 msgr2=0x7f618418d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ab0 tx=0x7f6174032d00 comp rx=0 tx=0).stop 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 shutdown_connections 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f616c0394f0 0x7f616c03b9b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f618418c920 0x7f61841a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618418bf60 0x7f618418c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.241 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 --2- 172.21.15.189:0/3230495428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61841005f0 0x7f618418d910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:13.241 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 >> 172.21.15.189:0/3230495428 conn(0x7f61840fbda0 msgr2=0x7f618406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:13.241 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 shutdown_connections 2024-09-05T11:48:13.241 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:48:13.238+0000 7f618a011640 1 -- 172.21.15.189:0/3230495428 wait complete. 2024-09-05T11:48:13.835 DEBUG:teuthology.orchestra.run.smithi089:mgr.y> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.y.service 2024-09-05T11:48:13.838 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.z.service 2024-09-05T11:48:13.841 INFO:tasks.cephadm:Deploying OSDs... 2024-09-05T11:48:13.841 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:48:13.841 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T11:48:13.881 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T11:48:13.881 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-09-05T11:48:13.959 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-05 11:47:29.483563485 +0000 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-05 11:43:07.804268085 +0000 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-05 11:43:07.804268085 +0000 2024-09-05T11:48:13.960 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-05 11:43:07.804268085 +0000 2024-09-05T11:48:13.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T11:48:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-05T11:48:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-05T11:48:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000280062 s, 1.8 MB/s 2024-09-05T11:48:14.059 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T11:48:14.099 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-05 11:47:29.524562936 +0000 2024-09-05T11:48:14.175 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-05 11:43:08.269261956 +0000 2024-09-05T11:48:14.176 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-05 11:43:08.269261956 +0000 2024-09-05T11:48:14.176 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-05 11:43:08.269261956 +0000 2024-09-05T11:48:14.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T11:48:14.279 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-05T11:48:14.279 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-05T11:48:14.279 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000293114 s, 1.7 MB/s 2024-09-05T11:48:14.281 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T11:48:14.318 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-09-05T11:48:14.397 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T11:48:14.397 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-05 11:47:29.565562387 +0000 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-05 11:43:08.730255879 +0000 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-05 11:43:08.730255879 +0000 2024-09-05T11:48:14.398 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-05 11:43:08.730255879 +0000 2024-09-05T11:48:14.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi089=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: Saving service mgr spec with placement smithi059=x;smithi089=y;smithi189=z;count:3 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T11:48:14.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:14.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:14.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:14.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:14 smithi059 ceph-mon[30025]: Reconfiguring daemon mgr.x on smithi059 2024-09-05T11:48:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-05T11:48:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-05T11:48:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000296266 s, 1.7 MB/s 2024-09-05T11:48:14.495 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T11:48:14.575 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-09-05T11:48:14.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi089=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: Saving service mgr spec with placement smithi059=x;smithi089=y;smithi189=z;count:3 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:14 smithi189 ceph-mon[34772]: Reconfiguring daemon mgr.x on smithi059 2024-09-05T11:48:14.650 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T11:48:14.650 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:14.650 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-05T11:48:14.650 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:14.651 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:14.651 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-05 11:47:29.603561878 +0000 2024-09-05T11:48:14.651 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-05 11:43:09.198249710 +0000 2024-09-05T11:48:14.651 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-05 11:43:09.198249710 +0000 2024-09-05T11:48:14.651 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-05 11:43:09.198249710 +0000 2024-09-05T11:48:14.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T11:48:14.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:14.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi089=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:14.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: Saving service mgr spec with placement smithi059=x;smithi089=y;smithi189=z;count:3 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:14.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:14 smithi089 ceph-mon[34499]: Reconfiguring daemon mgr.x on smithi059 2024-09-05T11:48:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-05T11:48:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-05T11:48:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000245016 s, 2.1 MB/s 2024-09-05T11:48:14.749 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T11:48:14.792 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-05T11:48:14.792 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T11:48:14.814 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T11:48:14.814 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-05 11:48:03.470211550 +0000 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-05 11:42:30.766391372 +0000 2024-09-05T11:48:14.873 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-05 11:42:30.766391372 +0000 2024-09-05T11:48:14.874 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-05 11:42:30.766391372 +0000 2024-09-05T11:48:14.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T11:48:14.940 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-05T11:48:14.941 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-05T11:48:14.941 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000303991 s, 1.7 MB/s 2024-09-05T11:48:14.942 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T11:48:15.001 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-05 11:48:03.470211550 +0000 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-05 11:42:31.102383246 +0000 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-05 11:42:31.102383246 +0000 2024-09-05T11:48:15.059 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-05 11:42:31.102383246 +0000 2024-09-05T11:48:15.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T11:48:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-05T11:48:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-05T11:48:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000177261 s, 2.9 MB/s 2024-09-05T11:48:15.126 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T11:48:15.183 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-05T11:48:15.240 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-05 11:48:03.471211526 +0000 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-05 11:42:31.439375095 +0000 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-05 11:42:31.439375095 +0000 2024-09-05T11:48:15.241 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-05 11:42:31.439375095 +0000 2024-09-05T11:48:15.241 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T11:48:15.305 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-05T11:48:15.305 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-05T11:48:15.305 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000184002 s, 2.8 MB/s 2024-09-05T11:48:15.306 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T11:48:15.363 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-05T11:48:15.418 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-05 11:48:03.471211526 +0000 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-05 11:42:31.766367186 +0000 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-05 11:42:31.766367186 +0000 2024-09-05T11:48:15.419 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-05 11:42:31.766367186 +0000 2024-09-05T11:48:15.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T11:48:15.483 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-05T11:48:15.483 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-05T11:48:15.484 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000218475 s, 2.3 MB/s 2024-09-05T11:48:15.485 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T11:48:15.541 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-05T11:48:15.542 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T11:48:15.562 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T11:48:15.562 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-05 11:47:54.255668026 +0000 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-05 11:42:30.779950678 +0000 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-05 11:42:30.779950678 +0000 2024-09-05T11:48:15.621 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-05 11:42:30.779950678 +0000 2024-09-05T11:48:15.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T11:48:15.689 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-05T11:48:15.689 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-05T11:48:15.689 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00023207 s, 2.2 MB/s 2024-09-05T11:48:15.690 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T11:48:15.749 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-05 11:47:54.256668005 +0000 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-05 11:42:31.109943689 +0000 2024-09-05T11:48:15.806 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-05 11:42:31.109943689 +0000 2024-09-05T11:48:15.807 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-05 11:42:31.109943689 +0000 2024-09-05T11:48:15.807 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T11:48:15.871 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-05T11:48:15.872 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-05T11:48:15.872 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000215083 s, 2.4 MB/s 2024-09-05T11:48:15.873 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T11:48:15.930 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-05T11:48:15.987 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T11:48:15.987 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-05 11:47:54.256668005 +0000 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-05 11:42:31.449936488 +0000 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-05 11:42:31.449936488 +0000 2024-09-05T11:48:15.988 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-05 11:42:31.449936488 +0000 2024-09-05T11:48:15.988 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T11:48:16.052 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-05T11:48:16.052 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-05T11:48:16.052 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.0002315 s, 2.2 MB/s 2024-09-05T11:48:16.053 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T11:48:16.111 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T11:48:16.256 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-05 11:47:54.257667984 +0000 2024-09-05T11:48:16.257 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-05 11:42:31.782929436 +0000 2024-09-05T11:48:16.257 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-05 11:42:31.782929436 +0000 2024-09-05T11:48:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-05 11:42:31.782929436 +0000 2024-09-05T11:48:16.257 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: Reconfiguring mon.b (monmap changed)... 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:16.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:16.320 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:16.320 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:16 smithi189 ceph-mon[34772]: Reconfiguring daemon mon.b on smithi089 2024-09-05T11:48:16.321 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-05T11:48:16.321 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-05T11:48:16.321 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000222612 s, 2.3 MB/s 2024-09-05T11:48:16.322 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T11:48:16.379 INFO:tasks.cephadm:Deploying osd.0 on smithi059 with /dev/vg_nvme/lv_4... 2024-09-05T11:48:16.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T11:48:16.545 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: Reconfiguring mon.b (monmap changed)... 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:16.546 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:16 smithi089 ceph-mon[34499]: Reconfiguring daemon mon.b on smithi089 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: Reconfiguring mon.b (monmap changed)... 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:16 smithi059 ceph-mon[30025]: Reconfiguring daemon mon.b on smithi089 2024-09-05T11:48:16.756 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: Reconfiguring mon.c (monmap changed)... 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:17.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:17 smithi189 ceph-mon[34772]: Reconfiguring daemon mon.c on smithi189 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: Reconfiguring mon.c (monmap changed)... 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:17.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:17 smithi059 ceph-mon[30025]: Reconfiguring daemon mon.c on smithi189 2024-09-05T11:48:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: Reconfiguring mon.c (monmap changed)... 2024-09-05T11:48:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T11:48:17.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T11:48:17.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:17.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:17 smithi089 ceph-mon[34499]: Reconfiguring daemon mon.c on smithi189 2024-09-05T11:48:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:18.980 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:18.980 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.980 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.980 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:18.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:19.607 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:48:19.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-09-05T11:48:19.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:19 smithi189 ceph-mon[34772]: Deploying daemon mgr.z on smithi189 2024-09-05T11:48:19.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:19 smithi059 ceph-mon[30025]: Deploying daemon mgr.z on smithi189 2024-09-05T11:48:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:19 smithi089 ceph-mon[34499]: Deploying daemon mgr.z on smithi189 2024-09-05T11:48:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:48:20.340 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:20.259+0000 7fe8ca41f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T11:48:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 -- 172.21.15.59:0/558908826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0105920 msgr2=0x7f18d010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 --2- 172.21.15.59:0/558908826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0105920 0x7f18d010c240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ca0 tx=0x7f18c802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 -- 172.21.15.59:0/558908826 shutdown_connections 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 --2- 172.21.15.59:0/558908826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0105920 0x7f18d010c240 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 --2- 172.21.15.59:0/558908826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0104f60 0x7f18d01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 --2- 172.21.15.59:0/558908826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d0104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 -- 172.21.15.59:0/558908826 >> 172.21.15.59:0/558908826 conn(0x7f18d00ff5b0 msgr2=0x7f18d01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.509+0000 7f18d7c00640 1 -- 172.21.15.59:0/558908826 shutdown_connections 2024-09-05T11:48:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.510+0000 7f18d7c00640 1 -- 172.21.15.59:0/558908826 wait complete. 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.510+0000 7f18d7c00640 1 Processor -- start 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 -- start start 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0104f60 0x7f18d019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0105920 0x7f18d01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010fae0 con 0x7f18d0104f60 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010f960 con 0x7f18d0105920 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d7c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18d010fc60 con 0x7f18d0103e00 2024-09-05T11:48:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:33742/0 (socket says 172.21.15.59:33742) 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 -- 172.21.15.59:0/29518953 learned_addr learned my addr 172.21.15.59:0/29518953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0104f60 0x7f18d019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0105920 0x7f18d01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 -- 172.21.15.59:0/29518953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0105920 msgr2=0x7f18d01a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0105920 0x7f18d01a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.511+0000 7f18d5200640 1 -- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0104f60 msgr2=0x7f18d019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18d5200640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0104f60 0x7f18d019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18d5200640 1 -- 172.21.15.59:0/29518953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a1cb0 con 0x7f18d0103e00 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18d5200640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9f0 tx=0x7f18c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c400cd80 con 0x7f18d0103e00 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c4010910 con 0x7f18d0103e00 2024-09-05T11:48:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a1fa0 con 0x7f18d0103e00 2024-09-05T11:48:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4018990 con 0x7f18d0103e00 2024-09-05T11:48:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.512+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a23f0 con 0x7f18d0103e00 2024-09-05T11:48:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.514+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f18c4018b30 con 0x7f18d0103e00 2024-09-05T11:48:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.514+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d010b120 con 0x7f18d0103e00 2024-09-05T11:48:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.514+0000 7f18cd600640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 0x7f18b803b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.514+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f18c404c910 con 0x7f18d0103e00 2024-09-05T11:48:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.514+0000 7f18cfe00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 0x7f18b803b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.515+0000 7f18cfe00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 0x7f18b803b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0095f0 tx=0x7f18bc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.519+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18c4018e20 con 0x7f18d0103e00 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:20.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:20.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:20.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:20.714+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f18d0062630 con 0x7f18b80392c0 2024-09-05T11:48:20.763 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:20.407+0000 7fe8ca41f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T11:48:20.764 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:20.473+0000 7fe8ca41f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T11:48:20.765 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:20.622+0000 7fe8ca41f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:48:20.765 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:20.762+0000 7fe8ca41f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T11:48:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T11:48:20.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:21.328 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:21.074+0000 7fe8ca41f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T11:48:21.328 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:21.134+0000 7fe8ca41f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T11:48:21.588 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:21.326+0000 7fe8ca41f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T11:48:21.893 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:21 smithi089 ceph-mon[34499]: Deploying daemon mgr.y on smithi089 2024-09-05T11:48:21.893 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:21 smithi089 ceph-mon[34499]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:21.893 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:21 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:21.893 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:21 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:21.893 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:21 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:21 smithi059 ceph-mon[30025]: Deploying daemon mgr.y on smithi089 2024-09-05T11:48:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:21 smithi059 ceph-mon[30025]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-mon[34772]: Deploying daemon mgr.y on smithi089 2024-09-05T11:48:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-mon[34772]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:21 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:22.184 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 systemd[1]: Starting Ceph mgr.y for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 podman[35847]: 2024-09-05 11:48:22.302786231 +0000 UTC m=+0.108938331 container create 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 podman[35847]: 2024-09-05 11:48:22.209553153 +0000 UTC m=+0.015705256 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 podman[35847]: 2024-09-05 11:48:22.49709617 +0000 UTC m=+0.303248284 container init 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 podman[35847]: 2024-09-05 11:48:22.49994408 +0000 UTC m=+0.306096181 container start 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-mgr[35866]: -- 172.21.15.89:0/4292743863 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a705910e0 con 0x558a7056f400 2024-09-05T11:48:22.573 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 bash[35847]: 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c 2024-09-05T11:48:22.588 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:22.267+0000 7fe8ca41f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T11:48:22.589 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:22.336+0000 7fe8ca41f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T11:48:22.850 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-mon[34499]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:22.850 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 systemd[1]: Started Ceph mgr.y for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:48:22.850 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:22.632+0000 7f3630726040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T11:48:22.850 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:22.695+0000 7f3630726040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T11:48:22.850 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:22.847+0000 7f3630726040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T11:48:22.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:22 smithi059 ceph-mon[30025]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:22 smithi189 ceph-mon[34772]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:23.184 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:22.917+0000 7f3630726040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T11:48:23.184 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:23.074+0000 7f3630726040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:48:23.372 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.089+0000 7fe8ca41f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T11:48:23.372 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.227+0000 7fe8ca41f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T11:48:23.523 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:23.245+0000 7f3630726040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T11:48:23.653 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.370+0000 7fe8ca41f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:48:23.653 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.436+0000 7fe8ca41f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T11:48:23.654 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.591+0000 7fe8ca41f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T11:48:23.831 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:23.572+0000 7f3630726040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:23.634+0000 7f3630726040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T11:48:23.832 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:23.830+0000 7f3630726040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T11:48:23.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:23.972 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.661+0000 7fe8ca41f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.722+0000 7fe8ca41f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.900+0000 7fe8ca41f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T11:48:23.973 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:23.971+0000 7fe8ca41f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T11:48:24.247 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:24.037+0000 7fe8ca41f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T11:48:24.247 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:24.123+0000 7fe8ca41f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T11:48:24.247 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:24.183+0000 7fe8ca41f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T11:48:24.247 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:24.245+0000 7fe8ca41f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T11:48:24.934 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:24.722+0000 7f3630726040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T11:48:24.934 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:24.788+0000 7f3630726040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1658041207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]': finished 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:24 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/630138423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1658041207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]': finished 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:24.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:24 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/630138423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:25.029 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1658041207' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d274feb-cee7-4082-a14e-2c4c9ed0bc40"}]': finished 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:25.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:25.031 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:25.031 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:25.031 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:24 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/630138423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:25.338 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z[36584]: 2024-09-05T11:48:25.027+0000 7fe8ca41f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T11:48:25.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:25.663+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f18c40173d0 con 0x7f18d0103e00 2024-09-05T11:48:25.753 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-mon[34499]: Standby manager daemon z started 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:25.497+0000 7f3630726040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T11:48:25.754 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:25.610+0000 7f3630726040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T11:48:25.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:25 smithi059 ceph-mon[30025]: Standby manager daemon z started 2024-09-05T11:48:25.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:25 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T11:48:25.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:25 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:25.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:25 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T11:48:25.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:25 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:26.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:25.752+0000 7f3630726040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T11:48:26.030 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:25.813+0000 7f3630726040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T11:48:26.030 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:25.959+0000 7f3630726040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T11:48:26.030 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.028+0000 7f3630726040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T11:48:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-mon[34772]: Standby manager daemon z started 2024-09-05T11:48:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T11:48:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T11:48:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:25 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.189:0/332739802' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:26.309 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.087+0000 7f3630726040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T11:48:26.309 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.243+0000 7f3630726040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T11:48:26.309 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.307+0000 7f3630726040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T11:48:26.572 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.373+0000 7f3630726040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T11:48:26.572 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.453+0000 7f3630726040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T11:48:26.572 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.512+0000 7f3630726040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T11:48:26.572 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:26.570+0000 7f3630726040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T11:48:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-mon[34499]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-mon[34499]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T11:48:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T11:48:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:26 smithi059 ceph-mon[30025]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:26 smithi059 ceph-mon[30025]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T11:48:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T11:48:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:26 smithi189 ceph-mon[34772]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:26 smithi189 ceph-mon[34772]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T11:48:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T11:48:27.684 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y[35862]: 2024-09-05T11:48:27.350+0000 7f3630726040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T11:48:27.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:27.826+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mon.1 v2:172.21.15.189:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f18c4010430 con 0x7f18d0103e00 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: Standby manager daemon y started 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T11:48:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:27 smithi189 ceph-mon[34772]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: Standby manager daemon y started 2024-09-05T11:48:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T11:48:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:28.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T11:48:28.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:27 smithi089 ceph-mon[34499]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: Standby manager daemon y started 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T11:48:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:27 smithi059 ceph-mon[30025]: from='mgr.? 172.21.15.89:0/3512430085' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T11:48:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:28 smithi059 ceph-mon[30025]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-05T11:48:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T11:48:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:28 smithi189 ceph-mon[34772]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-05T11:48:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T11:48:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:28 smithi089 ceph-mon[34499]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-05T11:48:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T11:48:30.085 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:29 smithi089 ceph-mon[34499]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:30.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:29 smithi189 ceph-mon[34772]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:30.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:29 smithi059 ceph-mon[30025]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:32.088 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:31 smithi059 ceph-mon[30025]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:32.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:31 smithi189 ceph-mon[34772]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:32.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:31 smithi089 ceph-mon[34499]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:33.091 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T11:48:33.091 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:33.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:32 smithi059 ceph-mon[30025]: Deploying daemon osd.0 on smithi059 2024-09-05T11:48:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T11:48:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:32 smithi089 ceph-mon[34499]: Deploying daemon osd.0 on smithi059 2024-09-05T11:48:33.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T11:48:33.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:33.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:32 smithi189 ceph-mon[34772]: Deploying daemon osd.0 on smithi059 2024-09-05T11:48:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:33 smithi059 ceph-mon[30025]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:33 smithi089 ceph-mon[34499]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:34.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:33 smithi189 ceph-mon[34772]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:35.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:35 smithi059 ceph-mon[30025]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:36.145 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:35 smithi189 ceph-mon[34772]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:35 smithi089 ceph-mon[34499]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:37.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:37 smithi059 ceph-mon[30025]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:37 smithi089 ceph-mon[34499]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:38.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:37 smithi189 ceph-mon[34772]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:38.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:38.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:38.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:39.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:39.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:39.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:39.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:40.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:40 smithi059 ceph-mon[30025]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:40 smithi089 ceph-mon[34499]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:40 smithi189 ceph-mon[34772]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:42.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:42.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:42.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:43.412 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 0 on host 'smithi059' 2024-09-05T11:48:43.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.409+0000 7f18cd600640 1 -- 172.21.15.59:0/29518953 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f18d0062630 con 0x7f18b80392c0 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 msgr2=0x7f18b803b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 0x7f18b803b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0095f0 tx=0x7f18bc005f90 comp rx=0 tx=0).stop 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 msgr2=0x7f18d019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9f0 tx=0x7f18c400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 shutdown_connections 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18b80392c0 0x7f18b803b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18d0105920 0x7f18d01a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18d0104f60 0x7f18d019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 --2- 172.21.15.59:0/29518953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0103e00 0x7f18d019a080 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:43.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.412+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 >> 172.21.15.59:0/29518953 conn(0x7f18d00ff5b0 msgr2=0x7f18d01082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:43.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.413+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 shutdown_connections 2024-09-05T11:48:43.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:48:43.413+0000 7f18d7c00640 1 -- 172.21.15.59:0/29518953 wait complete. 2024-09-05T11:48:44.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:43 smithi059 ceph-mon[30025]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:44.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:44.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.137 DEBUG:teuthology.orchestra.run.smithi059:osd.0> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.0.service 2024-09-05T11:48:44.141 INFO:tasks.cephadm:Deploying osd.1 on smithi089 with /dev/vg_nvme/lv_4... 2024-09-05T11:48:44.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T11:48:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:43 smithi189 ceph-mon[34772]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.391 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:48:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:43 smithi089 ceph-mon[34499]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:48:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:44.644 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:48:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T11:48:44.363+0000 7f98ed239640 -1 osd.0 0 log_to_monitors true 2024-09-05T11:48:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:44 smithi089 ceph-mon[34499]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:44 smithi089 ceph-mon[34499]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:45.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:44 smithi189 ceph-mon[34772]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:45.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:44 smithi189 ceph-mon[34772]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:45.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:44 smithi059 ceph-mon[30025]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:45.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:44 smithi059 ceph-mon[30025]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T11:48:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T11:48:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T11:48:46.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:46.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:45 smithi059 ceph-mon[30025]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.209 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:48:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T11:48:46.008+0000 7f98e7a00640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T11:48:46.209 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:48:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T11:48:46.011+0000 7f98e2a00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.243 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:45 smithi089 ceph-mon[34499]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: from='osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:45 smithi189 ceph-mon[34772]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:48:46.983 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:48:47.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-09-05T11:48:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:47 smithi059 ceph-mon[30025]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:48:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:47 smithi059 ceph-mon[30025]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T11:48:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.256 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:48:47.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:47 smithi189 ceph-mon[34772]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:48:47.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:47 smithi189 ceph-mon[34772]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T11:48:47.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:47 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:47 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:47 smithi089 ceph-mon[34499]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:48:47.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:47 smithi089 ceph-mon[34499]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T11:48:47.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:47 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:47 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 -- 172.21.15.89:0/4031186565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 msgr2=0x7fd27c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 --2- 172.21.15.89:0/4031186565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c103330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd270009900 tx=0x7fd27002a410 comp rx=0 tx=0).stop 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 -- 172.21.15.89:0/4031186565 shutdown_connections 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 --2- 172.21.15.89:0/4031186565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c103870 0x7fd27c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 --2- 172.21.15.89:0/4031186565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c103330 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:47.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 --2- 172.21.15.89:0/4031186565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c0ffed0 0x7fd27c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 -- 172.21.15.89:0/4031186565 >> 172.21.15.89:0/4031186565 conn(0x7fd27c0fba40 msgr2=0x7fd27c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:48:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 -- 172.21.15.89:0/4031186565 shutdown_connections 2024-09-05T11:48:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.733+0000 7fd281f60640 1 -- 172.21.15.89:0/4031186565 wait complete. 2024-09-05T11:48:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 Processor -- start 2024-09-05T11:48:47.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 -- start start 2024-09-05T11:48:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c0ffed0 0x7fd27c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 0x7fd27c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c10d8b0 con 0x7fd27c0ffed0 2024-09-05T11:48:47.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c10d730 con 0x7fd27c102eb0 2024-09-05T11:48:47.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd281f60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c10da30 con 0x7fd27c103870 2024-09-05T11:48:47.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd27affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:47.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd27affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51098/0 (socket says 172.21.15.89:51098) 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd27affd640 1 -- 172.21.15.89:0/2069005395 learned_addr learned my addr 172.21.15.89:0/2069005395 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd27b7fe640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c0ffed0 0x7fd27c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.734+0000 7fd27bfff640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 0x7fd27c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 -- 172.21.15.89:0/2069005395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 msgr2=0x7fd27c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 0x7fd27c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:47.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 -- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c0ffed0 msgr2=0x7fd27c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c0ffed0 0x7fd27c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 -- 172.21.15.89:0/2069005395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c19fba0 con 0x7fd27c102eb0 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27bfff640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 0x7fd27c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd27affd640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd270031470 tx=0x7fd270032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27003f030 con 0x7fd27c102eb0 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c100ca0 con 0x7fd27c102eb0 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270003ce0 con 0x7fd27c102eb0 2024-09-05T11:48:47.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270035070 con 0x7fd27c102eb0 2024-09-05T11:48:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.735+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c101180 con 0x7fd27c102eb0 2024-09-05T11:48:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.736+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c109100 con 0x7fd27c102eb0 2024-09-05T11:48:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.737+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fd270042030 con 0x7fd27c102eb0 2024-09-05T11:48:47.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.737+0000 7fd278ff9640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 0x7fd2640a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:48:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.738+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7fd2700edae0 con 0x7fd27c102eb0 2024-09-05T11:48:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.738+0000 7fd27b7fe640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 0x7fd2640a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:48:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.738+0000 7fd27b7fe640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 0x7fd2640a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06a590 tx=0x7fd26c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:48:47.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.742+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2700bc4b0 con 0x7fd27c102eb0 2024-09-05T11:48:47.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:48:47.915+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd27c1025d0 con 0x7fd2640a2b20 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] boot 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi059 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 17094M 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:48.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] boot 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi059 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 17094M 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:48.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:48:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: osd.0 [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] boot 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi059 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:48:48.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 17094M 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:48:48.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:49.231 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:49 smithi089 ceph-mon[34499]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:49.231 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:49 smithi089 ceph-mon[34499]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T11:48:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:49 smithi189 ceph-mon[34772]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:49.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:49 smithi189 ceph-mon[34772]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T11:48:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:49 smithi059 ceph-mon[30025]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:48:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:49 smithi059 ceph-mon[30025]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T11:48:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:50 smithi089 ceph-mon[34499]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:50 smithi189 ceph-mon[34772]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:50 smithi059 ceph-mon[30025]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/2200016968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]': finished 2024-09-05T11:48:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T11:48:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:48:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:51 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/3380572236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/2200016968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]': finished 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:48:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:51 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/3380572236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:51.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/2200016968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]: dispatch 2024-09-05T11:48:51.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b53955-a3d9-45d3-9a4f-c36d00907942"}]': finished 2024-09-05T11:48:51.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T11:48:51.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:48:51.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:51 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/3380572236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:48:52.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:52 smithi189 ceph-mon[34772]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:52.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:52 smithi089 ceph-mon[34499]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:52.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:52 smithi059 ceph-mon[30025]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:54.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:54 smithi189 ceph-mon[34772]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:54.629 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:54 smithi089 ceph-mon[34499]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:54.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:54 smithi059 ceph-mon[30025]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:56.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:56 smithi189 ceph-mon[34772]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:56.621 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:56 smithi089 ceph-mon[34499]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:56.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:56 smithi059 ceph-mon[30025]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:58.572 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:58 smithi089 ceph-mon[34499]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:58.572 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:58 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T11:48:58.573 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:58 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:58 smithi189 ceph-mon[34772]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:58 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T11:48:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:58 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:58.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:58 smithi059 ceph-mon[30025]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:48:58.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:58 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T11:48:58.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:58 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:48:59.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:48:59 smithi089 ceph-mon[34499]: Deploying daemon osd.1 on smithi089 2024-09-05T11:48:59.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:48:59 smithi189 ceph-mon[34772]: Deploying daemon osd.1 on smithi089 2024-09-05T11:48:59.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:48:59 smithi059 ceph-mon[30025]: Deploying daemon osd.1 on smithi089 2024-09-05T11:49:00.541 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:00 smithi089 ceph-mon[34499]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:00.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:00 smithi189 ceph-mon[34772]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:00.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:00 smithi059 ceph-mon[30025]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:02.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:02 smithi189 ceph-mon[34772]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:02.628 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:02 smithi089 ceph-mon[34499]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:02.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:02 smithi059 ceph-mon[30025]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:04 smithi089 ceph-mon[34499]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:04.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:04 smithi189 ceph-mon[34772]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:04.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:04.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:04.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:04.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:04 smithi059 ceph-mon[30025]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:04.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:04.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:04.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.459+0000 7fd278ff9640 1 -- 172.21.15.89:0/2069005395 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd27c1025d0 con 0x7fd2640a2b20 2024-09-05T11:49:07.461 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 1 on host 'smithi089' 2024-09-05T11:49:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 msgr2=0x7fd2640a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 0x7fd2640a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06a590 tx=0x7fd26c005f90 comp rx=0 tx=0).stop 2024-09-05T11:49:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 msgr2=0x7fd27c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd270031470 tx=0x7fd270032c70 comp rx=0 tx=0).stop 2024-09-05T11:49:07.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 shutdown_connections 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd2640a2b20 0x7fd2640a4fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd27c103870 0x7fd27c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd27c102eb0 0x7fd27c1984b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 --2- 172.21.15.89:0/2069005395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd27c0ffed0 0x7fd27c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 >> 172.21.15.89:0/2069005395 conn(0x7fd27c0fba40 msgr2=0x7fd27c1062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 shutdown_connections 2024-09-05T11:49:07.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:49:07.462+0000 7fd281f60640 1 -- 172.21.15.89:0/2069005395 wait complete. 2024-09-05T11:49:07.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:07 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:07.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:07 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:07 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:07 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:07.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:07 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:07.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:07 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:08.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:07 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:07 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:07 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:08.091 DEBUG:teuthology.orchestra.run.smithi089:osd.1> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.1.service 2024-09-05T11:49:08.093 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-09-05T11:49:08.093 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T11:49:08.344 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:49:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:08 smithi189 ceph-mon[34772]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:08 smithi059 ceph-mon[30025]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:08 smithi089 ceph-mon[34499]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:09.654 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T11:49:09.538+0000 7fc31c833640 -1 osd.1 0 log_to_monitors true 2024-09-05T11:49:09.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:09 smithi189 ceph-mon[34772]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:09.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:09 smithi189 ceph-mon[34772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:09.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:09 smithi089 ceph-mon[34499]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:09.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:09 smithi089 ceph-mon[34499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:09.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:09 smithi059 ceph-mon[30025]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:09.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:09 smithi059 ceph-mon[30025]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T11:49:10.915 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:10.916 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T11:49:10.916 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:10.916 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T11:49:10.916 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:10.916 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:10 smithi189 ceph-mon[34772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:49:10.944 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:10 smithi059 ceph-mon[30025]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:10.997 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T11:49:10.706+0000 7fc3187b6640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T11:49:10.997 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T11:49:10.713+0000 7fc3145de640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: from='osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:10.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:10 smithi089 ceph-mon[34499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:49:11.202 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:49:11.660 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.650+0000 7fcf76b45640 1 -- 172.21.15.189:0/6866456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 msgr2=0x7fcf70104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.650+0000 7fcf76b45640 1 --2- 172.21.15.189:0/6866456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf70104a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009900 tx=0x7fcf6402a410 comp rx=0 tx=0).stop 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- 172.21.15.189:0/6866456 shutdown_connections 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- 172.21.15.189:0/6866456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf70109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- 172.21.15.189:0/6866456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf70104fd0 0x7fcf701073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- 172.21.15.189:0/6866456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf70104a90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- 172.21.15.189:0/6866456 >> 172.21.15.189:0/6866456 conn(0x7fcf700fc2a0 msgr2=0x7fcf700fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- 172.21.15.189:0/6866456 shutdown_connections 2024-09-05T11:49:11.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- 172.21.15.189:0/6866456 wait complete. 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 Processor -- start 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- start start 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf70104fd0 0x7fcf701985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf7019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010d990 con 0x7fcf70104fd0 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010d810 con 0x7fcf70107900 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf76b45640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010db10 con 0x7fcf701026a0 2024-09-05T11:49:11.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf748e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf748e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52610/0 (socket says 172.21.15.189:52610) 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.651+0000 7fcf748e1640 1 -- 172.21.15.189:0/1879797302 learned_addr learned my addr 172.21.15.189:0/1879797302 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf750e2640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf7019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 -- 172.21.15.189:0/1879797302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 msgr2=0x7fcf7019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf7019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 -- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf70104fd0 msgr2=0x7fcf701985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf70104fd0 0x7fcf701985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 -- 172.21.15.189:0/1879797302 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7019fcd0 con 0x7fcf701026a0 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf750e2640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf7019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf748e1640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009230 tx=0x7fcf64004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:49:11.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6403f030 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf64003ce0 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7019ff60 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64035070 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.652+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701a0430 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.653+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf700fdd90 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.654+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fcf64042030 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.655+0000 7fcf6dffb640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 0x7fcf600a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.655+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fcf640ede50 con 0x7fcf701026a0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.655+0000 7fcf6ffff640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 0x7fcf600a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:11.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.656+0000 7fcf6ffff640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 0x7fcf600a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf701995c0 tx=0x7fcf5c006c90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:49:11.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.660+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf640bc720 con 0x7fcf701026a0 2024-09-05T11:49:11.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:11.836+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcf70062630 con 0x7fcf600a2b20 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:12.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:12.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:12.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:12.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:49:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T11:49:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi089 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] boot 2024-09-05T11:49:12.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T11:49:12.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:12.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:12.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:12.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi089 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] boot 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T11:49:13.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:13.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:13.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:13.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:49:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:49:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi089 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: osd.1 [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] boot 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:13.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:14.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:13 smithi189 ceph-mon[34772]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:14.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:13 smithi189 ceph-mon[34772]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T11:49:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:13 smithi089 ceph-mon[34499]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:13 smithi089 ceph-mon[34499]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T11:49:14.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:13 smithi059 ceph-mon[30025]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:14.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:13 smithi059 ceph-mon[30025]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/1967376231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]': finished 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:14 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/1700980345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/1967376231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]': finished 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:14 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/1700980345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/1967376231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]: dispatch 2024-09-05T11:49:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03ca7ba-23bc-4f72-b35e-1784080f9550"}]': finished 2024-09-05T11:49:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T11:49:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:14 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/1700980345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:16.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:15 smithi189 ceph-mon[34772]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:15 smithi089 ceph-mon[34499]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:16.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:15 smithi059 ceph-mon[30025]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:17 smithi089 ceph-mon[34499]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:17 smithi059 ceph-mon[30025]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:18.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:17 smithi189 ceph-mon[34772]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:20.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:19 smithi189 ceph-mon[34772]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:20.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:19 smithi089 ceph-mon[34499]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:19 smithi059 ceph-mon[30025]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:21.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:21 smithi189 ceph-mon[34772]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:22.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:21 smithi089 ceph-mon[34499]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:21 smithi059 ceph-mon[30025]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T11:49:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:22 smithi189 ceph-mon[34772]: Deploying daemon osd.2 on smithi189 2024-09-05T11:49:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T11:49:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:22 smithi089 ceph-mon[34499]: Deploying daemon osd.2 on smithi189 2024-09-05T11:49:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T11:49:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:22 smithi059 ceph-mon[30025]: Deploying daemon osd.2 on smithi189 2024-09-05T11:49:23.981 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:23 smithi189 ceph-mon[34772]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:24.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:23 smithi059 ceph-mon[30025]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:24.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:23 smithi089 ceph-mon[34499]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:25 smithi189 ceph-mon[34772]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:26.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:25 smithi089 ceph-mon[34499]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:25 smithi059 ceph-mon[30025]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:27 smithi089 ceph-mon[34499]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:28.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:28.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:27 smithi059 ceph-mon[30025]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:28.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:27 smithi189 ceph-mon[34772]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:28.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:28.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:28.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:30.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:31.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.335+0000 7fcf6dffb640 1 -- 172.21.15.189:0/1879797302 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcf70062630 con 0x7fcf600a2b20 2024-09-05T11:49:31.338 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 msgr2=0x7fcf600a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 0x7fcf600a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf701995c0 tx=0x7fcf5c006c90 comp rx=0 tx=0).stop 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 msgr2=0x7fcf701980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009230 tx=0x7fcf64004430 comp rx=0 tx=0).stop 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 shutdown_connections 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcf600a2b20 0x7fcf600a4fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf70107900 0x7fcf7019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf70104fd0 0x7fcf701985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:31.340 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 --2- 172.21.15.189:0/1879797302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf701026a0 0x7fcf701980a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:31.341 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 >> 172.21.15.189:0/1879797302 conn(0x7fcf700fc2a0 msgr2=0x7fcf70102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:49:31.341 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 shutdown_connections 2024-09-05T11:49:31.341 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:49:31.338+0000 7fcf76b45640 1 -- 172.21.15.189:0/1879797302 wait complete. 2024-09-05T11:49:31.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:31 smithi189 ceph-mon[34772]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:31.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:31.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:31.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:31.924 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.2.service 2024-09-05T11:49:31.926 INFO:tasks.cephadm:Deploying osd.3 on smithi059 with /dev/vg_nvme/lv_3... 2024-09-05T11:49:31.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T11:49:32.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:31 smithi089 ceph-mon[34499]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:32.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:32.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:32.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:31 smithi059 ceph-mon[30025]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:32.303 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:49:33.821 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T11:49:33.554+0000 7f20e1025640 -1 osd.2 0 log_to_monitors true 2024-09-05T11:49:33.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:33 smithi059 ceph-mon[30025]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:33.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:33 smithi059 ceph-mon[30025]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:33.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:33 smithi059 ceph-mon[30025]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:33 smithi189 ceph-mon[34772]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:33 smithi189 ceph-mon[34772]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:33 smithi189 ceph-mon[34772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:33 smithi089 ceph-mon[34499]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:33 smithi089 ceph-mon[34499]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:33 smithi089 ceph-mon[34499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:34.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi189 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:34.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.089 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T11:49:34.834+0000 7f20dcfa8640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T11:49:35.089 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T11:49:34.841+0000 7f20d8dd0640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:49:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi189 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi189 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:49:35.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:35.615 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:49:35.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-09-05T11:49:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:35 smithi059 ceph-mon[30025]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:35 smithi059 ceph-mon[30025]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:49:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:35 smithi059 ceph-mon[30025]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T11:49:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:49:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:35 smithi089 ceph-mon[34499]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:35 smithi089 ceph-mon[34499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:49:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:35 smithi089 ceph-mon[34499]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T11:49:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:36.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:36.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:35 smithi189 ceph-mon[34772]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:36.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:35 smithi189 ceph-mon[34772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:49:36.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:35 smithi189 ceph-mon[34772]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T11:49:36.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:36.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 -- 172.21.15.59:0/4211876356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290076ba0 msgr2=0x7fa290076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 --2- 172.21.15.59:0/4211876356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290076ba0 0x7fa290076fa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa280009900 tx=0x7fa28002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 -- 172.21.15.59:0/4211876356 shutdown_connections 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 --2- 172.21.15.59:0/4211876356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290078790 0x7fa290079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 --2- 172.21.15.59:0/4211876356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290077df0 0x7fa290078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 --2- 172.21.15.59:0/4211876356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290076ba0 0x7fa290076fa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 -- 172.21.15.59:0/4211876356 >> 172.21.15.59:0/4211876356 conn(0x7fa2900ff3e0 msgr2=0x7fa290101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 -- 172.21.15.59:0/4211876356 shutdown_connections 2024-09-05T11:49:36.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.712+0000 7fa295600640 1 -- 172.21.15.59:0/4211876356 wait complete. 2024-09-05T11:49:36.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.713+0000 7fa295600640 1 Processor -- start 2024-09-05T11:49:36.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 -- start start 2024-09-05T11:49:36.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290076ba0 0x7fa29019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290078790 0x7fa2901a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa290113d60 con 0x7fa290077df0 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa290113be0 con 0x7fa290078790 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa295600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa290113ee0 con 0x7fa290076ba0 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa28f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa28f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37808/0 (socket says 172.21.15.59:37808) 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa28f400640 1 -- 172.21.15.59:0/3718980441 learned_addr learned my addr 172.21.15.59:0/3718980441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa294c00640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290078790 0x7fa2901a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.714+0000 7fa28fe00640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290076ba0 0x7fa29019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 -- 172.21.15.59:0/3718980441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290076ba0 msgr2=0x7fa29019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290076ba0 0x7fa29019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 -- 172.21.15.59:0/3718980441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290078790 msgr2=0x7fa2901a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:49:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290078790 0x7fa2901a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:49:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 -- 172.21.15.59:0/3718980441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2901a5f40 con 0x7fa290077df0 2024-09-05T11:49:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28f400640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e9f0 tx=0x7fa27c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:49:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00cd80 con 0x7fa290077df0 2024-09-05T11:49:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2901a6230 con 0x7fa290077df0 2024-09-05T11:49:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c010910 con 0x7fa290077df0 2024-09-05T11:49:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.715+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0188f0 con 0x7fa290077df0 2024-09-05T11:49:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.716+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901a67c0 con 0x7fa290077df0 2024-09-05T11:49:36.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.717+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2901105d0 con 0x7fa290077df0 2024-09-05T11:49:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.718+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa27c010ac0 con 0x7fa290077df0 2024-09-05T11:49:36.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.720+0000 7fa28cc00640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:49:36.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.720+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fa27c0cde30 con 0x7fa290077df0 2024-09-05T11:49:36.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.720+0000 7fa28fe00640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:49:36.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.721+0000 7fa28fe00640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 0x7fa2740a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa280031440 tx=0x7fa2800315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:49:36.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.724+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa27c09bcd0 con 0x7fa290077df0 2024-09-05T11:49:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:36.920+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa290062630 con 0x7fa2740a2b20 2024-09-05T11:49:37.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:37 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:49:37.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:37 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:49:37.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:37 smithi089 ceph-mon[34499]: osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] boot 2024-09-05T11:49:37.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:37 smithi089 ceph-mon[34499]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T11:49:37.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:37 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:49:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:37 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:49:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:37 smithi059 ceph-mon[30025]: osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] boot 2024-09-05T11:49:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:37 smithi059 ceph-mon[30025]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T11:49:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:37.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:37 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:49:37.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:37 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:49:37.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:37 smithi189 ceph-mon[34772]: osd.2 [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] boot 2024-09-05T11:49:37.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:37 smithi189 ceph-mon[34772]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T11:49:37.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T11:49:38.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:38.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:38.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T11:49:38.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T11:49:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T11:49:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T11:49:39.571 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:39.572 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:39 smithi059 ceph-mon[30025]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T11:49:39.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:39.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:39 smithi089 ceph-mon[34499]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T11:49:39.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T11:49:39.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:39 smithi189 ceph-mon[34772]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T11:49:40.402 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:40.402 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:40.402 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48305]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:40.403 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48305]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:40 smithi089 ceph-mon[34499]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:40.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:40 smithi059 ceph-mon[30025]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:40.708 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48334]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:40.708 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48334]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:40.708 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48334]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:40.709 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 11:49:40 smithi059 sudo[48334]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:40.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:40 smithi189 ceph-mon[34772]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:41.161 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:40 smithi089 sudo[42616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:41.161 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:40 smithi089 sudo[42616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:41.161 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:40 smithi089 sudo[42616]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:41.161 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:40 smithi089 sudo[42616]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:41.434 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:41 smithi089 sudo[42619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:41.434 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:41 smithi089 sudo[42619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:41.434 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:41 smithi089 sudo[42619]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:41.434 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 11:49:41 smithi089 sudo[42619]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1520255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]': finished 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4128192565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:41.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:49:41.669+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa27c098340 con 0x7fa290077df0 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1520255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]': finished 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4128192565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:41.672 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:41.690 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43324]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:41.690 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:41.691 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43324]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:41.691 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43324]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1520255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]: dispatch 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe023f0b-749c-4eed-ae29-f58180c46a1a"}]': finished 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4128192565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:49:41.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:42.088 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43327]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:42.089 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43327]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:42.089 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43327]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:42.089 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 11:49:41 smithi189 sudo[43327]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48414]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48414]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:43.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:42 smithi189 ceph-mon[34772]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T11:49:43.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:42 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:43.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:42 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:43.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43330]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:43.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43330]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:43.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:43.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43330]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:42 smithi089 ceph-mon[34499]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T11:49:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:42 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:42 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:43.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:43.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:43.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48788]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:43.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 sudo[48788]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:43.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 ceph-mon[30025]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T11:49:43.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:43.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:42 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43333]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43333]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43333]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 sudo[43333]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:43.851 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42622]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T11:49:43.851 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:43.852 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42622]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:43.852 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42622]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42625]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 sudo[42625]: pam_unix(sudo:session): session closed for user root 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T11:49:44.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T11:49:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:44 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:44 smithi089 ceph-mon[34499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:45.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:44 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:45.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:44 smithi059 ceph-mon[30025]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:45.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:44 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T11:49:45.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:44 smithi189 ceph-mon[34772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T11:49:45.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:45 smithi059 ceph-mon[30025]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:45 smithi089 ceph-mon[34499]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:45 smithi189 ceph-mon[34772]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:47.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:47 smithi059 ceph-mon[30025]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:47 smithi089 ceph-mon[34499]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:47 smithi189 ceph-mon[34772]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:49:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:49 smithi089 ceph-mon[34499]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-05T11:49:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T11:49:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:49 smithi089 ceph-mon[34499]: Deploying daemon osd.3 on smithi059 2024-09-05T11:49:50.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:49 smithi059 ceph-mon[30025]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-05T11:49:50.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T11:49:50.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:50.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:49 smithi059 ceph-mon[30025]: Deploying daemon osd.3 on smithi059 2024-09-05T11:49:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:49 smithi189 ceph-mon[34772]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-05T11:49:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T11:49:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:49:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:49 smithi189 ceph-mon[34772]: Deploying daemon osd.3 on smithi059 2024-09-05T11:49:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:51 smithi089 ceph-mon[34499]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-05T11:49:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:51 smithi059 ceph-mon[30025]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-05T11:49:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:51 smithi189 ceph-mon[34772]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-05T11:49:54.035 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:53 smithi059 ceph-mon[30025]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T11:49:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:53 smithi089 ceph-mon[34499]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T11:49:54.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:53 smithi189 ceph-mon[34772]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T11:49:56.026 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:55 smithi059 ceph-mon[30025]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:49:56.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:55 smithi189 ceph-mon[34772]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:49:56.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:55 smithi089 ceph-mon[34499]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:49:57.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:57.103 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:57.103 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:57.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:49:57.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:57.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:49:58.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:49:58 smithi059 ceph-mon[30025]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:49:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:49:58 smithi189 ceph-mon[34772]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:49:58.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:49:58 smithi089 ceph-mon[34499]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:50:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:50:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:00.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:00 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:00 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:00 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T11:50:00.590 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 3 on host 'smithi059' 2024-09-05T11:50:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.587+0000 7fa28cc00640 1 -- 172.21.15.59:0/3718980441 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa290062630 con 0x7fa2740a2b20 2024-09-05T11:50:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 msgr2=0x7fa2740a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 0x7fa2740a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa280031440 tx=0x7fa2800315b0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 msgr2=0x7fa29019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e9f0 tx=0x7fa27c00eeb0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 shutdown_connections 2024-09-05T11:50:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2740a2b20 0x7fa2740a4fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa290078790 0x7fa2901a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa290077df0 0x7fa29019e850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 --2- 172.21.15.59:0/3718980441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290076ba0 0x7fa29019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 >> 172.21.15.59:0/3718980441 conn(0x7fa2900ff3e0 msgr2=0x7fa290100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 shutdown_connections 2024-09-05T11:50:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:00.590+0000 7fa295600640 1 -- 172.21.15.59:0/3718980441 wait complete. 2024-09-05T11:50:01.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:01.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.432 DEBUG:teuthology.orchestra.run.smithi059:osd.3> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.3.service 2024-09-05T11:50:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:01 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:01 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:01 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.436 INFO:tasks.cephadm:Deploying osd.4 on smithi089 with /dev/vg_nvme/lv_3... 2024-09-05T11:50:01.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T11:50:01.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:01.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:01.691 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:50:01.958 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 11:50:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T11:50:01.700+0000 7f2d47254640 -1 osd.3 0 log_to_monitors true 2024-09-05T11:50:02.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:02 smithi059 ceph-mon[30025]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:02.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:02 smithi059 ceph-mon[30025]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:02.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:02 smithi059 ceph-mon[30025]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:02.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:02 smithi189 ceph-mon[34772]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:02.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:02 smithi189 ceph-mon[34772]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:02.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:02 smithi189 ceph-mon[34772]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:02 smithi089 ceph-mon[34499]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:02 smithi089 ceph-mon[34499]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:02 smithi089 ceph-mon[34499]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T11:50:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:03 smithi089 ceph-mon[34499]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T11:50:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:03 smithi089 ceph-mon[34499]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:03 smithi089 ceph-mon[34499]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T11:50:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:03 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:03 smithi089 ceph-mon[34499]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:03 smithi059 ceph-mon[30025]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T11:50:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:03 smithi059 ceph-mon[30025]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:03 smithi059 ceph-mon[30025]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T11:50:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:03 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:03 smithi059 ceph-mon[30025]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:03 smithi189 ceph-mon[34772]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T11:50:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:03 smithi189 ceph-mon[34772]: from='osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:03 smithi189 ceph-mon[34772]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T11:50:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:03 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:03 smithi189 ceph-mon[34772]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:50:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:50:04.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-09-05T11:50:04.581 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:50:04.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:04.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: from='osd.3 ' entity='osd.3' 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] boot 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T11:50:04.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.638 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T11:50:04.359+0000 7f2d41e00640 -1 osd.3 0 waiting for initial osdmap 2024-09-05T11:50:04.639 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 11:50:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T11:50:04.373+0000 7f2d37e00640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:50:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: from='osd.3 ' entity='osd.3' 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] boot 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T11:50:04.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:05.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 -- 172.21.15.89:0/2239580252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 msgr2=0x7fb978104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/2239580252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978104af0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb968009900 tx=0x7fb96802a410 comp rx=0 tx=0).stop 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 -- 172.21.15.89:0/2239580252 shutdown_connections 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/2239580252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978107960 0x7fb978109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/2239580252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978105030 0x7fb978107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/2239580252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978104af0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 -- 172.21.15.89:0/2239580252 >> 172.21.15.89:0/2239580252 conn(0x7fb9780fc280 msgr2=0x7fb9780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.042+0000 7fb97e6e5640 1 -- 172.21.15.89:0/2239580252 shutdown_connections 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 -- 172.21.15.89:0/2239580252 wait complete. 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 Processor -- start 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 -- start start 2024-09-05T11:50:05.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 0x7fb978198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 0x7fb97819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb97810d950 con 0x7fb978107960 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb97810d7d0 con 0x7fb978102700 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb97e6e5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb97810dad0 con 0x7fb978105030 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.043+0000 7fb977fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51386/0 (socket says 172.21.15.89:51386) 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 -- 172.21.15.89:0/725936430 learned_addr learned my addr 172.21.15.89:0/725936430 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb9777fe640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 0x7fb978198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 -- 172.21.15.89:0/725936430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 msgr2=0x7fb978198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 0x7fb978198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:05.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 -- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 msgr2=0x7fb97819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb97cc82640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 0x7fb97819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 0x7fb97819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 -- 172.21.15.89:0/725936430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97819fc70 con 0x7fb978102700 2024-09-05T11:50:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb9777fe640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 0x7fb978198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:50:05.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb97cc82640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 0x7fb97819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.044+0000 7fb977fff640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb968009230 tx=0x7fb968004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.045+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96803f030 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.045+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb968003ce0 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.045+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97819ff00 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.045+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968035070 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.045+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781a0380 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.046+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9780fdd70 con 0x7fb978102700 2024-09-05T11:50:05.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.047+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb968042030 con 0x7fb978102700 2024-09-05T11:50:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.048+0000 7fb9757fa640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 0x7fb9640a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.048+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7fb9680ee600 con 0x7fb978102700 2024-09-05T11:50:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.048+0000 7fb9777fe640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 0x7fb9640a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:05.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.049+0000 7fb9777fe640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 0x7fb9640a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb960006fd0 tx=0x7fb960008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:05.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.053+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9680bc760 con 0x7fb978102700 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: from='osd.3 ' entity='osd.3' 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: osd.3 [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] boot 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T11:50:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T11:50:05.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:05.232+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb978062630 con 0x7fb9640a2bb0 2024-09-05T11:50:05.932 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:50:05.933 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:50:05.933 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:05.933 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:05.933 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:05.933 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:05 smithi089 ceph-mon[34499]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:05 smithi059 ceph-mon[30025]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:05 smithi189 ceph-mon[34772]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi059 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 8547M 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:06.998 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:06.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:06.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:06 smithi089 ceph-mon[34499]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi059 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 8547M 2024-09-05T11:50:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:06 smithi189 ceph-mon[34772]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T11:50:07.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:07.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:07.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi059 2024-09-05T11:50:07.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 8547M 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:07.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:06 smithi059 ceph-mon[30025]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/2555269619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]': finished 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:08 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/3824618542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T11:50:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/2555269619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]': finished 2024-09-05T11:50:09.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T11:50:09.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:09.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:08 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/3824618542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T11:50:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/2555269619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]: dispatch 2024-09-05T11:50:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ff44594-2df9-4bb2-8154-402f258fab7c"}]': finished 2024-09-05T11:50:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T11:50:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/3824618542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:10 smithi189 ceph-mon[34772]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T11:50:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:10 smithi089 ceph-mon[34499]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T11:50:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:10 smithi059 ceph-mon[30025]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T11:50:12.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:12 smithi089 ceph-mon[34499]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-05T11:50:13.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:12 smithi059 ceph-mon[30025]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-05T11:50:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:12 smithi189 ceph-mon[34772]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-05T11:50:15.061 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:14 smithi089 ceph-mon[34499]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:50:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:14 smithi189 ceph-mon[34772]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:50:15.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:14 smithi059 ceph-mon[30025]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:50:16.996 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:16 smithi089 ceph-mon[34499]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:50:16.996 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T11:50:16.996 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:16 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:16 smithi189 ceph-mon[34772]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:50:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T11:50:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:16 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:16 smithi059 ceph-mon[30025]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:50:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T11:50:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:16 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:18.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:17 smithi189 ceph-mon[34772]: Deploying daemon osd.4 on smithi089 2024-09-05T11:50:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:17 smithi189 ceph-mon[34772]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:18.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:17 smithi089 ceph-mon[34499]: Deploying daemon osd.4 on smithi089 2024-09-05T11:50:18.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:17 smithi089 ceph-mon[34499]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:17 smithi059 ceph-mon[30025]: Deploying daemon osd.4 on smithi089 2024-09-05T11:50:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:17 smithi059 ceph-mon[30025]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:20.108 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:19 smithi089 ceph-mon[34499]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:19 smithi059 ceph-mon[30025]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:20.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:19 smithi189 ceph-mon[34772]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:21.952 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:21 smithi089 ceph-mon[34499]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:21 smithi059 ceph-mon[30025]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:22.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:21 smithi189 ceph-mon[34772]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:23.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:23.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:23.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:24.273 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:24 smithi089 ceph-mon[34499]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:24.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:24 smithi189 ceph-mon[34772]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:24.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:24 smithi059 ceph-mon[30025]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:26.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.145 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:26.145 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.145 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.145 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:26.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:26.148 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.192+0000 7fb9757fa640 1 -- 172.21.15.89:0/725936430 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb978062630 con 0x7fb9640a2bb0 2024-09-05T11:50:26.195 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 4 on host 'smithi089' 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 msgr2=0x7fb9640a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 0x7fb9640a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb960006fd0 tx=0x7fb960008040 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 msgr2=0x7fb978198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb968009230 tx=0x7fb968004430 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 shutdown_connections 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb9640a2bb0 0x7fb9640a5070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb978107960 0x7fb97819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978105030 0x7fb978198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 --2- 172.21.15.89:0/725936430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb978102700 0x7fb978198040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:26.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 >> 172.21.15.89:0/725936430 conn(0x7fb9780fc280 msgr2=0x7fb9781031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 shutdown_connections 2024-09-05T11:50:26.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:50:26.194+0000 7fb97e6e5640 1 -- 172.21.15.89:0/725936430 wait complete. 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:26.861 DEBUG:teuthology.orchestra.run.smithi089:osd.4> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.4.service 2024-09-05T11:50:26.864 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-09-05T11:50:26.864 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T11:50:27.113 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:50:27.166 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:27.166 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:27.167 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:27.234 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:27.234 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:27.234 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:28.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:28 smithi089 ceph-mon[34499]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:28.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:28 smithi089 ceph-mon[34499]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:28.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:28 smithi089 ceph-mon[34499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:28.316 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 11:50:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T11:50:28.006+0000 7f2b7f318640 -1 osd.4 0 log_to_monitors true 2024-09-05T11:50:28.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:28 smithi189 ceph-mon[34772]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:28.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:28 smithi189 ceph-mon[34772]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:28.449 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:28 smithi189 ceph-mon[34772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:28 smithi059 ceph-mon[30025]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:28 smithi059 ceph-mon[30025]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:28 smithi059 ceph-mon[30025]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T11:50:29.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:29 smithi189 ceph-mon[34772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T11:50:29.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:29 smithi189 ceph-mon[34772]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:29 smithi189 ceph-mon[34772]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T11:50:29.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:29.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:29 smithi189 ceph-mon[34772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-mon[34499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T11:50:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-mon[34499]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-mon[34499]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T11:50:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-mon[34499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.667 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:50:29.694 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-09-05T11:50:29.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:29 smithi059 ceph-mon[30025]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T11:50:29.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:29 smithi059 ceph-mon[30025]: from='osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:29 smithi059 ceph-mon[30025]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T11:50:29.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:29.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:29 smithi059 ceph-mon[30025]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:50:29.942 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:50:30.184 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T11:50:29.845+0000 7f2b7b29b640 -1 osd.4 0 waiting for initial osdmap 2024-09-05T11:50:30.184 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 11:50:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T11:50:29.850+0000 7f2b730bb640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:50:30.326 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:30.326 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:50:30.326 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T11:50:30.326 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:30.326 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='osd.4 ' entity='osd.4' 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.327 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:30.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 -- 172.21.15.189:0/2653139814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68341037c0 msgr2=0x7f683410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:30.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 --2- 172.21.15.189:0/2653139814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68341037c0 0x7f683410a050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f682c009ca0 tx=0x7f682c02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 -- 172.21.15.189:0/2653139814 shutdown_connections 2024-09-05T11:50:30.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 --2- 172.21.15.189:0/2653139814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68341037c0 0x7f683410a050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 --2- 172.21.15.189:0/2653139814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6834102e00 0x7f6834103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 --2- 172.21.15.189:0/2653139814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6834101c00 0x7f6834102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 -- 172.21.15.189:0/2653139814 >> 172.21.15.189:0/2653139814 conn(0x7f68340fd390 msgr2=0x7f68340ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 -- 172.21.15.189:0/2653139814 shutdown_connections 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.408+0000 7f683969f640 1 -- 172.21.15.189:0/2653139814 wait complete. 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 Processor -- start 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 -- start start 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6834101c00 0x7f6834075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68341037c0 0x7f6834074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:30.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f683410d930 con 0x7f6834101c00 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f683410d7b0 con 0x7f68341037c0 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f683969f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f683410dab0 con 0x7f6834102e00 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f68327fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f68327fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56584/0 (socket says 172.21.15.189:56584) 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f68327fc640 1 -- 172.21.15.189:0/4103580534 learned_addr learned my addr 172.21.15.189:0/4103580534 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.409+0000 7f68337fe640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68341037c0 0x7f6834074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 -- 172.21.15.189:0/4103580534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68341037c0 msgr2=0x7f6834074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68341037c0 0x7f6834074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 -- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6834101c00 msgr2=0x7f6834075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6834101c00 0x7f6834075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:30.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 -- 172.21.15.189:0/4103580534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68340753e0 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f68327fc640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f682800d770 tx=0x7f682800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828017070 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828005500 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828004e00 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68341a2080 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.410+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a25c0 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.411+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6834108fb0 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.412+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6828004fa0 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.413+0000 7f681bfff640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 0x7f681c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.413+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f68280cd910 con 0x7f6834102e00 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.413+0000 7f6832ffd640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 0x7f681c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.414+0000 7f6832ffd640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 0x7f681c0a5040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6834102c60 tx=0x7f6824006c90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:30.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.417+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f682809ba00 con 0x7f6834102e00 2024-09-05T11:50:30.595 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:30.594+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6834062630 con 0x7f681c0a2b80 2024-09-05T11:50:30.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:30.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='osd.4 ' entity='osd.4' 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='osd.4 ' entity='osd.4' 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:30.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:31.261 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi089 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] boot 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T11:50:31.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi089 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] boot 2024-09-05T11:50:31.587 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T11:50:31.588 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi089 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:31.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:31.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:31.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:31.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: osd.4 [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] boot 2024-09-05T11:50:31.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T11:50:31.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T11:50:32.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:32 smithi189 ceph-mon[34772]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:32.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:32 smithi189 ceph-mon[34772]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T11:50:32.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:32 smithi089 ceph-mon[34499]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:32.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:32 smithi089 ceph-mon[34499]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T11:50:32.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:32 smithi059 ceph-mon[30025]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-05T11:50:32.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:32 smithi059 ceph-mon[30025]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T11:50:33.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:33 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/3629490084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:33 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:33 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]': finished 2024-09-05T11:50:33.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:33 smithi189 ceph-mon[34772]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T11:50:33.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:33.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:33 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/3629490084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:33 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:33 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]': finished 2024-09-05T11:50:33.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:33 smithi089 ceph-mon[34499]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T11:50:33.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:33 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/3629490084' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:33 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]: dispatch 2024-09-05T11:50:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:33 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d7954d-f1d1-4ac8-b1df-e6c542fe0104"}]': finished 2024-09-05T11:50:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:33 smithi059 ceph-mon[30025]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T11:50:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:34.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:34 smithi189 ceph-mon[34772]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:34.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/3281873178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:34.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:34 smithi089 ceph-mon[34499]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:34.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/3281873178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:34.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:34 smithi059 ceph-mon[30025]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:34.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/3281873178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:50:36.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:36 smithi189 ceph-mon[34772]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:36.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:36 smithi089 ceph-mon[34499]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:36.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:36 smithi059 ceph-mon[30025]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:38 smithi189 ceph-mon[34772]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:38.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:38 smithi089 ceph-mon[34499]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:38 smithi059 ceph-mon[30025]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:40 smithi189 ceph-mon[34772]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:40 smithi089 ceph-mon[34499]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:40.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:40 smithi059 ceph-mon[30025]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T11:50:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:41.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T11:50:41.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:41.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T11:50:41.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:42.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:42 smithi189 ceph-mon[34772]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:42.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:42 smithi189 ceph-mon[34772]: Deploying daemon osd.5 on smithi189 2024-09-05T11:50:42.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:42 smithi089 ceph-mon[34499]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:42.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:42 smithi089 ceph-mon[34499]: Deploying daemon osd.5 on smithi189 2024-09-05T11:50:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:42 smithi059 ceph-mon[30025]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:42 smithi059 ceph-mon[30025]: Deploying daemon osd.5 on smithi189 2024-09-05T11:50:44.524 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:44 smithi059 ceph-mon[30025]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:44.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:44 smithi189 ceph-mon[34772]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:44.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:44 smithi089 ceph-mon[34499]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:46 smithi189 ceph-mon[34772]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:46 smithi059 ceph-mon[30025]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:46.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:46 smithi089 ceph-mon[34499]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:47.325 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:47 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:47.325 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:47 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:47.325 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:47 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:47.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:47 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:47.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:47 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:47.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:47 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:47.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:47.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:47.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:47 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:48 smithi089 ceph-mon[34499]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:48 smithi059 ceph-mon[30025]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:49.021 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:48 smithi189 ceph-mon[34772]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:50.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:50.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:50.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:50.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.581 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-09-05T11:50:50.581 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.578+0000 7f681bfff640 1 -- 172.21.15.189:0/4103580534 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6834062630 con 0x7f681c0a2b80 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 msgr2=0x7f681c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 0x7f681c0a5040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6834102c60 tx=0x7f6824006c90 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 msgr2=0x7f68340745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f682800d770 tx=0x7f682800dc30 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 shutdown_connections 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f681c0a2b80 0x7f681c0a5040 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68341037c0 0x7f6834074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6834102e00 0x7f68340745e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 --2- 172.21.15.189:0/4103580534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6834101c00 0x7f6834075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:50.583 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 >> 172.21.15.189:0/4103580534 conn(0x7f68340fd390 msgr2=0x7f6834106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:50.584 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 shutdown_connections 2024-09-05T11:50:50.584 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:50:50.580+0000 7f683969f640 1 -- 172.21.15.189:0/4103580534 wait complete. 2024-09-05T11:50:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:50.735 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.735 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:50.735 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:50.736 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:50.736 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.297 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.5.service 2024-09-05T11:50:51.299 INFO:tasks.cephadm:Deploying osd.6 on smithi059 with /dev/vg_nvme/lv_2... 2024-09-05T11:50:51.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T11:50:51.541 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:51.541 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.541 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.677 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:50:51.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:51.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:50:51.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:51.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:52.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:52 smithi189 ceph-mon[34772]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:52 smithi089 ceph-mon[34499]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:52.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:52 smithi059 ceph-mon[30025]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:53.089 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 11:50:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T11:50:52.946+0000 7fbd3f964640 -1 osd.5 0 log_to_monitors true 2024-09-05T11:50:53.622 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:53 smithi059 ceph-mon[30025]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:53.623 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:53 smithi059 ceph-mon[30025]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:53.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:53 smithi189 ceph-mon[34772]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:53 smithi189 ceph-mon[34772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:53.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:53 smithi089 ceph-mon[34499]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:53.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:53 smithi089 ceph-mon[34499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T11:50:54.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:54.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:50:54.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:54.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:50:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:50:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:50:55.303 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:50:55.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T11:50:55.250+0000 7fbd3c0fb640 -1 osd.5 0 waiting for initial osdmap 2024-09-05T11:50:55.589 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T11:50:55.258+0000 7fbd33707640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi189 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:55.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:55 smithi059 ceph-mon[30025]: from='osd.5 ' entity='osd.5' 2024-09-05T11:50:55.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 -- 172.21.15.59:0/271256074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 msgr2=0x7f3bd410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:55.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/271256074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd410c0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009ca0 tx=0x7f3bcc02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 -- 172.21.15.59:0/271256074 shutdown_connections 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/271256074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd410c0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/271256074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4104e50 0x7f3bd41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/271256074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4103c50 0x7f3bd4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 -- 172.21.15.59:0/271256074 >> 172.21.15.59:0/271256074 conn(0x7f3bd40ff3e0 msgr2=0x7f3bd4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 -- 172.21.15.59:0/271256074 shutdown_connections 2024-09-05T11:50:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.871+0000 7f3bd9e00640 1 -- 172.21.15.59:0/271256074 wait complete. 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.872+0000 7f3bd9e00640 1 Processor -- start 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.872+0000 7f3bd9e00640 1 -- start start 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4104e50 0x7f3bd419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd41a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410f8b0 con 0x7f3bd4105810 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410f730 con 0x7f3bd4103c50 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410fa30 con 0x7f3bd4104e50 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd41a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd41a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44758/0 (socket says 172.21.15.59:44758) 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:32788/0 (socket says 172.21.15.59:32788) 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd9400640 1 -- 172.21.15.59:0/1414113362 learned_addr learned my addr 172.21.15.59:0/1414113362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd3e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4104e50 0x7f3bd419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 -- 172.21.15.59:0/1414113362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4104e50 msgr2=0x7f3bd419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4104e50 0x7f3bd419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 -- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 msgr2=0x7f3bd41a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:50:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd41a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.873+0000 7f3bd8a00640 1 -- 172.21.15.59:0/1414113362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd41a1ab0 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd8a00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4104cb0 tx=0x7f3bc800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8017070 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bc8009c30 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41a1da0 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800f620 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.874+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd41a2200 con 0x7f3bd4103c50 2024-09-05T11:50:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.875+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd410af80 con 0x7f3bd4103c50 2024-09-05T11:50:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.877+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3bc801a070 con 0x7f3bd4103c50 2024-09-05T11:50:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.878+0000 7f3bd1600640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 0x7f3bbc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:50:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.879+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f3bc80cdb80 con 0x7f3bd4103c50 2024-09-05T11:50:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.879+0000 7f3bd3e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 0x7f3bbc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:50:55.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.879+0000 7f3bd3e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 0x7f3bbc0a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3bd419b3a0 tx=0x7f3bc0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:50:55.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:55.881+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bc809b290 con 0x7f3bd4103c50 2024-09-05T11:50:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi189 2024-09-05T11:50:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-05T11:50:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:50:55.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T11:50:55.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:55.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:55.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:55 smithi089 ceph-mon[34499]: from='osd.5 ' entity='osd.5' 2024-09-05T11:50:56.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:50:56.075+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3bd4062630 con 0x7f3bbc0a2bb0 2024-09-05T11:50:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi189 2024-09-05T11:50:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-05T11:50:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:50:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T11:50:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:55 smithi189 ceph-mon[34772]: from='osd.5 ' entity='osd.5' 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] boot 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:56.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:56.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:56 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:50:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:50:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] boot 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:56.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:56 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: osd.5 [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] boot 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:50:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:56 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:50:57.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:57 smithi089 ceph-mon[34499]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:57.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:57 smithi089 ceph-mon[34499]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T11:50:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:57 smithi059 ceph-mon[30025]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:57 smithi059 ceph-mon[30025]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T11:50:58.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:57 smithi189 ceph-mon[34772]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:50:58.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:57 smithi189 ceph-mon[34772]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T11:50:58.866 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:58 smithi059 ceph-mon[30025]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:50:58.866 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:58 smithi059 ceph-mon[30025]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T11:50:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:58 smithi089 ceph-mon[34499]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:50:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:58 smithi089 ceph-mon[34499]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T11:50:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:58 smithi189 ceph-mon[34772]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:50:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:58 smithi189 ceph-mon[34772]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T11:50:59.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:59 smithi089 ceph-mon[34499]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T11:50:59.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:59 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]: dispatch 2024-09-05T11:50:59.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:59 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]': finished 2024-09-05T11:50:59.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:59 smithi089 ceph-mon[34499]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T11:50:59.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:50:59 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:50:59.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:59 smithi059 ceph-mon[30025]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T11:50:59.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:59 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]: dispatch 2024-09-05T11:50:59.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:59 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]': finished 2024-09-05T11:50:59.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:59 smithi059 ceph-mon[30025]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T11:50:59.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:50:59 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:00.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:59 smithi189 ceph-mon[34772]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T11:51:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:59 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]: dispatch 2024-09-05T11:51:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:59 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4105436595' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3748a7e4-df25-42ae-b0cf-a50a43bb3ed6"}]': finished 2024-09-05T11:51:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:59 smithi189 ceph-mon[34772]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T11:51:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:50:59 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:00 smithi089 ceph-mon[34499]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:00 smithi089 ceph-mon[34499]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:00 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2600633511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:00 smithi059 ceph-mon[30025]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:00 smithi059 ceph-mon[30025]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:00 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2600633511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:00 smithi189 ceph-mon[34772]: pgmap v141: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:00 smithi189 ceph-mon[34772]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:00 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2600633511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:02 smithi089 ceph-mon[34499]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:02 smithi059 ceph-mon[30025]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:02 smithi189 ceph-mon[34772]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:03 smithi089 ceph-mon[34499]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:03 smithi089 ceph-mon[34499]: Cluster is now healthy 2024-09-05T11:51:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:03 smithi059 ceph-mon[30025]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:03 smithi059 ceph-mon[30025]: Cluster is now healthy 2024-09-05T11:51:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:03 smithi189 ceph-mon[34772]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:03 smithi189 ceph-mon[34772]: Cluster is now healthy 2024-09-05T11:51:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:04 smithi089 ceph-mon[34499]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:51:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:04 smithi059 ceph-mon[30025]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:51:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:04 smithi189 ceph-mon[34772]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:51:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:06 smithi059 ceph-mon[30025]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T11:51:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:06 smithi189 ceph-mon[34772]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T11:51:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:06 smithi089 ceph-mon[34499]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-05T11:51:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:08 smithi059 ceph-mon[30025]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T11:51:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T11:51:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:08 smithi189 ceph-mon[34772]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T11:51:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T11:51:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:08 smithi089 ceph-mon[34499]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T11:51:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T11:51:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:09.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:09 smithi059 ceph-mon[30025]: Deploying daemon osd.6 on smithi059 2024-09-05T11:51:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:09 smithi189 ceph-mon[34772]: Deploying daemon osd.6 on smithi059 2024-09-05T11:51:10.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:09 smithi089 ceph-mon[34499]: Deploying daemon osd.6 on smithi059 2024-09-05T11:51:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:10 smithi059 ceph-mon[30025]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T11:51:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:10 smithi189 ceph-mon[34772]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T11:51:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:10 smithi089 ceph-mon[34499]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T11:51:12.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:12 smithi059 ceph-mon[30025]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-05T11:51:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:12 smithi189 ceph-mon[34772]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-05T11:51:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:12 smithi089 ceph-mon[34499]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-05T11:51:14.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:14 smithi059 ceph-mon[30025]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:51:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:14 smithi189 ceph-mon[34772]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:51:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:14 smithi089 ceph-mon[34499]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:51:15.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:15.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:15.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:16.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:16.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:16.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:16.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:16 smithi059 ceph-mon[30025]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:16 smithi189 ceph-mon[34772]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:16 smithi089 ceph-mon[34499]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:19.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.966+0000 7f3bd1600640 1 -- 172.21.15.59:0/1414113362 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bd4062630 con 0x7f3bbc0a2bb0 2024-09-05T11:51:19.968 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 6 on host 'smithi059' 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 msgr2=0x7f3bbc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 0x7f3bbc0a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3bd419b3a0 tx=0x7f3bc0005f90 comp rx=0 tx=0).stop 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 msgr2=0x7f3bd4199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4104cb0 tx=0x7f3bc800dde0 comp rx=0 tx=0).stop 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 shutdown_connections 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3bbc0a2bb0 0x7f3bbc0a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bd4105810 0x7f3bd41a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd4104e50 0x7f3bd419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:19.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 --2- 172.21.15.59:0/1414113362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bd4103c50 0x7f3bd4199e80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 >> 172.21.15.59:0/1414113362 conn(0x7f3bd40ff3e0 msgr2=0x7f3bd4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:51:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 shutdown_connections 2024-09-05T11:51:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:51:19.969+0000 7f3bd9e00640 1 -- 172.21.15.59:0/1414113362 wait complete. 2024-09-05T11:51:20.816 DEBUG:teuthology.orchestra.run.smithi059:osd.6> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.6.service 2024-09-05T11:51:20.821 INFO:tasks.cephadm:Deploying osd.7 on smithi089 with /dev/vg_nvme/lv_2... 2024-09-05T11:51:20.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T11:51:21.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:20 smithi059 ceph-mon[30025]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:21.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:21.023 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.023 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.074 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:51:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:20 smithi189 ceph-mon[34772]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:20 smithi089 ceph-mon[34499]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:21.353 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 11:51:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T11:51:21.020+0000 7ff28e63f640 -1 osd.6 0 log_to_monitors true 2024-09-05T11:51:21.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:21 smithi089 ceph-mon[34499]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T11:51:22.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:21 smithi189 ceph-mon[34772]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T11:51:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:21 smithi059 ceph-mon[30025]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:22 smithi089 ceph-mon[34499]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:22 smithi089 ceph-mon[34499]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T11:51:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:22 smithi089 ceph-mon[34499]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T11:51:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:22 smithi089 ceph-mon[34499]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:51:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:22 smithi189 ceph-mon[34772]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:22 smithi189 ceph-mon[34772]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T11:51:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:22 smithi189 ceph-mon[34772]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T11:51:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:22 smithi189 ceph-mon[34772]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:51:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-mon[30025]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-mon[30025]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T11:51:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-mon[30025]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T11:51:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-mon[30025]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:51:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.209 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T11:51:22.754+0000 7ff289200640 -1 osd.6 0 waiting for initial osdmap 2024-09-05T11:51:23.209 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 11:51:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T11:51:22.764+0000 7ff284200640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:51:23.837 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:23 smithi089 ceph-mon[34499]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:51:23.838 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:23 smithi089 ceph-mon[34499]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T11:51:23.838 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.838 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.841 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:51:23.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-09-05T11:51:23.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:23 smithi059 ceph-mon[30025]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:51:23.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:23 smithi059 ceph-mon[30025]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T11:51:23.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:23.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:24.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:23 smithi189 ceph-mon[34772]: from='osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:51:24.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:23 smithi189 ceph-mon[34772]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T11:51:24.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:24.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:24.116 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- 172.21.15.89:0/632437808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc101d00 msgr2=0x7f00bc102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- 172.21.15.89:0/632437808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc101d00 0x7f00bc102180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02a410 comp rx=0 tx=0).stop 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- 172.21.15.89:0/632437808 shutdown_connections 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- 172.21.15.89:0/632437808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 0x7f00bc108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- 172.21.15.89:0/632437808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc101d00 0x7f00bc102180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- 172.21.15.89:0/632437808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc100b00 0x7f00bc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- 172.21.15.89:0/632437808 >> 172.21.15.89:0/632437808 conn(0x7f00bc0fc290 msgr2=0x7f00bc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:51:24.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- 172.21.15.89:0/632437808 shutdown_connections 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- 172.21.15.89:0/632437808 wait complete. 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 Processor -- start 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- start start 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc101d00 0x7f00bc109ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 0x7f00bc10a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10d8b0 con 0x7f00bc1026c0 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10d730 con 0x7f00bc100b00 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.624+0000 7f00c4007640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10da30 con 0x7f00bc101d00 2024-09-05T11:51:24.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56232/0 (socket says 172.21.15.89:56232) 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 -- 172.21.15.89:0/3450070930 learned_addr learned my addr 172.21.15.89:0/3450070930 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c15a2640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc101d00 0x7f00bc109ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c25a4640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 0x7f00bc10a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 -- 172.21.15.89:0/3450070930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc101d00 msgr2=0x7f00bc109ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc101d00 0x7f00bc109ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 -- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 msgr2=0x7f00bc10a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 0x7f00bc10a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 -- 172.21.15.89:0/3450070930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc10a8d0 con 0x7f00bc100b00 2024-09-05T11:51:24.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c1da3640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00b800d770 tx=0x7f00b800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8017070 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b8009c30 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a20b0 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8003dd0 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.625+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a25c0 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.626+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc107f50 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.627+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00b8003fb0 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.628+0000 7f00b2ffd640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 0x7f00a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.628+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f00b80cde10 con 0x7f00bc100b00 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.628+0000 7f00c15a2640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 0x7f00a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.629+0000 7f00c15a2640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 0x7f00a80a5040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac0315d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:51:24.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.632+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00b80d1030 con 0x7f00bc100b00 2024-09-05T11:51:24.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:24.811+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f00bc062630 con 0x7f00a80a2b80 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] boot 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T11:51:24.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:51:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:51:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] boot 2024-09-05T11:51:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T11:51:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: osd.6 [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] boot 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T11:51:25.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:26.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:26.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:25 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:25 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:26.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:26.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:26.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:26.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:25 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:26.844 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:26 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi059 2024-09-05T11:51:26.845 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:26 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 5698M 2024-09-05T11:51:26.845 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:26 smithi089 ceph-mon[34499]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:26.845 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:26 smithi089 ceph-mon[34499]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:26 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi059 2024-09-05T11:51:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:26 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 5698M 2024-09-05T11:51:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:26 smithi189 ceph-mon[34772]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:26 smithi189 ceph-mon[34772]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:26 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi059 2024-09-05T11:51:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:26 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 5698M 2024-09-05T11:51:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:26 smithi059 ceph-mon[30025]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:26 smithi059 ceph-mon[30025]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:27.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:27 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/2717355561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:27.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:27 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:27.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:27 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]': finished 2024-09-05T11:51:27.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:27 smithi089 ceph-mon[34499]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T11:51:27.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:27 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/2717355561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:27 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:27 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]': finished 2024-09-05T11:51:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:27 smithi189 ceph-mon[34772]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T11:51:28.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:27 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/2717355561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:27 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]: dispatch 2024-09-05T11:51:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:27 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20387607-714b-47c7-af78-daf2be8da827"}]': finished 2024-09-05T11:51:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:27 smithi059 ceph-mon[30025]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T11:51:28.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:28 smithi189 ceph-mon[34772]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:28 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/2315209540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:28 smithi089 ceph-mon[34499]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:28 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/2315209540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:28 smithi059 ceph-mon[30025]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:28 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/2315209540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:30 smithi189 ceph-mon[34772]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:30 smithi089 ceph-mon[34499]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:30 smithi059 ceph-mon[30025]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:32.085 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:31 smithi089 ceph-mon[34499]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:32.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:31 smithi189 ceph-mon[34772]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:31 smithi059 ceph-mon[30025]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:33 smithi089 ceph-mon[34499]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:33 smithi059 ceph-mon[30025]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:34.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:33 smithi189 ceph-mon[34772]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:35 smithi089 ceph-mon[34499]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:35 smithi059 ceph-mon[30025]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:36.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:35 smithi189 ceph-mon[34772]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:36 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T11:51:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:36 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:36 smithi089 ceph-mon[34499]: Deploying daemon osd.7 on smithi089 2024-09-05T11:51:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:36 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T11:51:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:36 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:36 smithi059 ceph-mon[30025]: Deploying daemon osd.7 on smithi089 2024-09-05T11:51:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:36 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T11:51:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:36 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:36 smithi189 ceph-mon[34772]: Deploying daemon osd.7 on smithi089 2024-09-05T11:51:38.143 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:37 smithi089 ceph-mon[34499]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:37 smithi059 ceph-mon[30025]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:38.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:37 smithi189 ceph-mon[34772]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:39 smithi089 ceph-mon[34499]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:39 smithi059 ceph-mon[30025]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:39 smithi189 ceph-mon[34772]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:42.161 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:41 smithi089 ceph-mon[34499]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:42.161 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:42.161 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:42.161 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:41 smithi059 ceph-mon[30025]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:41 smithi189 ceph-mon[34772]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:42.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:44.058 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:43 smithi089 ceph-mon[34499]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:43 smithi059 ceph-mon[30025]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:44.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:43 smithi189 ceph-mon[34772]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:45.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.175+0000 7f00b2ffd640 1 -- 172.21.15.89:0/3450070930 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f00bc062630 con 0x7f00a80a2b80 2024-09-05T11:51:45.179 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 7 on host 'smithi089' 2024-09-05T11:51:45.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 msgr2=0x7f00a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:45.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 0x7f00a80a5040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac0315d0 comp rx=0 tx=0).stop 2024-09-05T11:51:45.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 msgr2=0x7f00bc10b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00b800d770 tx=0x7f00b800dc30 comp rx=0 tx=0).stop 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 shutdown_connections 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00a80a2b80 0x7f00a80a5040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00bc1026c0 0x7f00bc10a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00bc101d00 0x7f00bc109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 --2- 172.21.15.89:0/3450070930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00bc100b00 0x7f00bc10b480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:45.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 >> 172.21.15.89:0/3450070930 conn(0x7f00bc0fc290 msgr2=0x7f00bc105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:51:45.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 shutdown_connections 2024-09-05T11:51:45.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:51:45.178+0000 7f00c4007640 1 -- 172.21.15.89:0/3450070930 wait complete. 2024-09-05T11:51:45.864 DEBUG:teuthology.orchestra.run.smithi089:osd.7> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.7.service 2024-09-05T11:51:45.866 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-09-05T11:51:45.867 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T11:51:46.126 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:51:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:46 smithi089 ceph-mon[34499]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:46 smithi059 ceph-mon[30025]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:46 smithi189 ceph-mon[34772]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:46.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:46.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:47.795 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 11:51:47 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T11:51:47.696+0000 7fe25d0f1640 -1 osd.7 0 log_to_monitors true 2024-09-05T11:51:48.433 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:48 smithi189 ceph-mon[34772]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:48.433 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:48 smithi189 ceph-mon[34772]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.433 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:48 smithi189 ceph-mon[34772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:48 smithi089 ceph-mon[34499]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:48.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:48 smithi089 ceph-mon[34499]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:48 smithi089 ceph-mon[34499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:48 smithi059 ceph-mon[30025]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:48 smithi059 ceph-mon[30025]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:48 smithi059 ceph-mon[30025]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T11:51:48.841 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:51:48.868 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-09-05T11:51:49.120 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:51:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T11:51:49.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.532 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T11:51:49.532 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T11:51:49.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 -- 172.21.15.189:0/1044264305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180102eb0 msgr2=0x7f9180103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:49.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 --2- 172.21.15.189:0/1044264305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180102eb0 0x7f9180103330 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9170009930 tx=0x7f917002a420 comp rx=0 tx=0).stop 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 -- 172.21.15.189:0/1044264305 shutdown_connections 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 --2- 172.21.15.189:0/1044264305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9180103870 0x7f918010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 --2- 172.21.15.189:0/1044264305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180102eb0 0x7f9180103330 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 --2- 172.21.15.189:0/1044264305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91800ffed0 0x7f91801002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.614+0000 7f9187dc3640 1 -- 172.21.15.189:0/1044264305 >> 172.21.15.189:0/1044264305 conn(0x7f91800fba40 msgr2=0x7f91800fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- 172.21.15.189:0/1044264305 shutdown_connections 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- 172.21.15.189:0/1044264305 wait complete. 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 Processor -- start 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- start start 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91800ffed0 0x7f9180198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:49.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9180102eb0 0x7f9180198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d890 con 0x7f9180102eb0 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d710 con 0x7f91800ffed0 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9187dc3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f918010da10 con 0x7f9180103870 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9186360640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9186360640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47074/0 (socket says 172.21.15.189:47074) 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.615+0000 7f9186360640 1 -- 172.21.15.189:0/649007396 learned_addr learned my addr 172.21.15.189:0/649007396 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9185b5f640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91800ffed0 0x7f9180198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f918535e640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9180102eb0 0x7f9180198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 -- 172.21.15.189:0/649007396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91800ffed0 msgr2=0x7f9180198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91800ffed0 0x7f9180198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 -- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9180102eb0 msgr2=0x7f9180198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9180102eb0 0x7f9180198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:51:49.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 -- 172.21.15.189:0/649007396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918019fc30 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9186360640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9178009fd0 tx=0x7f917800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178017070 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9178003e50 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780106c0 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918019ff20 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.616+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a03d0 con 0x7f9180103870 2024-09-05T11:51:49.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.617+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9180109170 con 0x7f9180103870 2024-09-05T11:51:49.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.618+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9178003970 con 0x7f9180103870 2024-09-05T11:51:49.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.619+0000 7f9176ffd640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 0x7f916c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:51:49.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.619+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7f91780ce1d0 con 0x7f9180103870 2024-09-05T11:51:49.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.619+0000 7f9185b5f640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 0x7f916c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:51:49.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.620+0000 7f9185b5f640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 0x7f916c0a50f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f918006a590 tx=0x7f917c00b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:51:49.624 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.623+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f917809b600 con 0x7f9180103870 2024-09-05T11:51:49.684 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T11:51:49.188+0000 7fe259074640 -1 osd.7 0 waiting for initial osdmap 2024-09-05T11:51:49.684 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T11:51:49.197+0000 7fe254e9c640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T11:51:49.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:49.798 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:51:49.796+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9180062630 con 0x7f916c0a2c30 2024-09-05T11:51:50.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi089 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] boot 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi089 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] boot 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi089 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: osd.7 [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] boot 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:51:50.686 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:51:51.511 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:51 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:51:51.512 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:51 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:51:51.512 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:51 smithi189 ceph-mon[34772]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:51.512 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:51 smithi189 ceph-mon[34772]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T11:51:51.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:51 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:51:51.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:51 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:51:51.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:51 smithi089 ceph-mon[34499]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:51.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:51 smithi089 ceph-mon[34499]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T11:51:51.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:51 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:51:51.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:51 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:51:51.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:51 smithi059 ceph-mon[30025]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:51:51.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:51 smithi059 ceph-mon[30025]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T11:51:52.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:51:52.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T11:51:52.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/1069471618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]': finished 2024-09-05T11:51:52.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T11:51:52.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:52 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:51:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T11:51:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/1069471618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]': finished 2024-09-05T11:51:52.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T11:51:52.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:52 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/1069471618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]: dispatch 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7405b92c-3969-427a-b954-31e466aef539"}]': finished 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T11:51:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:52 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:53.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:53 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/1838610877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:53.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:53 smithi189 ceph-mon[34772]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T11:51:53.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:53.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:53 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/1838610877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:53.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:53 smithi089 ceph-mon[34499]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T11:51:53.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:53.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:53 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/1838610877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:51:53.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:53 smithi059 ceph-mon[30025]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T11:51:53.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:51:54.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:54 smithi189 ceph-mon[34772]: pgmap v180: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:54.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:54 smithi189 ceph-mon[34772]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:54 smithi089 ceph-mon[34499]: pgmap v180: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:54 smithi089 ceph-mon[34499]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:54 smithi059 ceph-mon[30025]: pgmap v180: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:54 smithi059 ceph-mon[30025]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-05T11:51:55.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:55 smithi189 ceph-mon[34772]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-05T11:51:55.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:55 smithi089 ceph-mon[34499]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-05T11:51:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:55 smithi059 ceph-mon[30025]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-05T11:51:56.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:56 smithi189 ceph-mon[34772]: pgmap v182: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:56 smithi089 ceph-mon[34499]: pgmap v182: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:56.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:56 smithi059 ceph-mon[30025]: pgmap v182: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2024-09-05T11:51:57.816 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:57 smithi189 ceph-mon[34772]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-05T11:51:57.816 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:57 smithi189 ceph-mon[34772]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:57.816 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:57 smithi189 ceph-mon[34772]: Cluster is now healthy 2024-09-05T11:51:57.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:57 smithi089 ceph-mon[34499]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-05T11:51:57.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:57 smithi089 ceph-mon[34499]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:57.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:57 smithi089 ceph-mon[34499]: Cluster is now healthy 2024-09-05T11:51:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:57 smithi059 ceph-mon[30025]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-05T11:51:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:57 smithi059 ceph-mon[30025]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-05T11:51:57.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:57 smithi059 ceph-mon[30025]: Cluster is now healthy 2024-09-05T11:51:58.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:51:58 smithi189 ceph-mon[34772]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:51:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:51:58 smithi089 ceph-mon[34499]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:51:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:51:58 smithi059 ceph-mon[30025]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:00.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:00 smithi189 ceph-mon[34772]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:00 smithi089 ceph-mon[34499]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:00 smithi059 ceph-mon[30025]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:01 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T11:52:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:01 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:01 smithi089 ceph-mon[34499]: Deploying daemon osd.8 on smithi189 2024-09-05T11:52:01.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T11:52:01.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:01 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:01.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:01 smithi059 ceph-mon[30025]: Deploying daemon osd.8 on smithi189 2024-09-05T11:52:02.055 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T11:52:02.055 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:01 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:02.055 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:01 smithi189 ceph-mon[34772]: Deploying daemon osd.8 on smithi189 2024-09-05T11:52:02.666 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:02 smithi189 ceph-mon[34772]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:02 smithi089 ceph-mon[34499]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:02 smithi059 ceph-mon[30025]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:04.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:04 smithi189 ceph-mon[34772]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:04 smithi089 ceph-mon[34499]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:04 smithi059 ceph-mon[30025]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:06 smithi189 ceph-mon[34772]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:06 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:06 smithi089 ceph-mon[34499]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:06.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:06 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:06 smithi059 ceph-mon[30025]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:06 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:08 smithi089 ceph-mon[34499]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:08 smithi059 ceph-mon[30025]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:08 smithi189 ceph-mon[34772]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:09.891 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.889+0000 7f9176ffd640 1 -- 172.21.15.189:0/649007396 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9180062630 con 0x7f916c0a2c30 2024-09-05T11:52:09.891 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-09-05T11:52:09.893 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 msgr2=0x7f916c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:09.893 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 0x7f916c0a50f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f918006a590 tx=0x7f917c00b390 comp rx=0 tx=0).stop 2024-09-05T11:52:09.893 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 msgr2=0x7f918019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:09.893 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9178009fd0 tx=0x7f917800b260 comp rx=0 tx=0).stop 2024-09-05T11:52:09.893 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 shutdown_connections 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f916c0a2c30 0x7f916c0a50f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180103870 0x7f918019f5c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9180102eb0 0x7f9180198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 --2- 172.21.15.189:0/649007396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91800ffed0 0x7f9180198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 >> 172.21.15.189:0/649007396 conn(0x7f91800fba40 msgr2=0x7f9180106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 shutdown_connections 2024-09-05T11:52:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:52:09.892+0000 7f9187dc3640 1 -- 172.21.15.189:0/649007396 wait complete. 2024-09-05T11:52:10.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:10.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:10.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:10.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:10.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:10.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:10.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:10.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:10.537 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.8.service 2024-09-05T11:52:10.539 INFO:tasks.cephadm:Deploying osd.9 on smithi059 with /dev/vg_nvme/lv_1... 2024-09-05T11:52:10.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T11:52:10.924 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:52:11.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:11.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:12.036 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:11 smithi189 ceph-mon[34772]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:12.037 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 11:52:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T11:52:11.963+0000 7fd6db4d0640 -1 osd.8 0 log_to_monitors true 2024-09-05T11:52:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:11 smithi089 ceph-mon[34499]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:11 smithi059 ceph-mon[30025]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:12 smithi189 ceph-mon[34772]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:12 smithi189 ceph-mon[34772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:12 smithi089 ceph-mon[34499]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:12 smithi089 ceph-mon[34499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:12 smithi059 ceph-mon[30025]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:12 smithi059 ceph-mon[30025]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T11:52:14.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:14.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T11:52:14.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T11:52:14.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi189 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:14.178 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:13 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi189 2024-09-05T11:52:14.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-05T11:52:14.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.187 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:14.187 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:14.187 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:13 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:52:14.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi189 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:13 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:14.670 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:52:14.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T11:52:14.571+0000 7fd6d7c67640 -1 osd.8 0 waiting for initial osdmap 2024-09-05T11:52:14.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T11:52:14.583+0000 7fd6cf273640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:52:15.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:52:15.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T11:52:15.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T11:52:15.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:14 smithi059 ceph-mon[30025]: from='osd.8 ' entity='osd.8' 2024-09-05T11:52:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:52:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T11:52:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T11:52:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:14 smithi089 ceph-mon[34499]: from='osd.8 ' entity='osd.8' 2024-09-05T11:52:15.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.249+0000 7f2fc8400640 1 -- 172.21.15.59:0/3899130190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0105920 msgr2=0x7f2fc010c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:15.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 --2- 172.21.15.59:0/3899130190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0105920 0x7f2fc010c240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009ca0 tx=0x7f2fb802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 -- 172.21.15.59:0/3899130190 shutdown_connections 2024-09-05T11:52:15.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 --2- 172.21.15.59:0/3899130190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0105920 0x7f2fc010c240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 --2- 172.21.15.59:0/3899130190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0104f60 0x7f2fc01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 --2- 172.21.15.59:0/3899130190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0103e00 0x7f2fc0104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 -- 172.21.15.59:0/3899130190 >> 172.21.15.59:0/3899130190 conn(0x7f2fc00ff5b0 msgr2=0x7f2fc01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 -- 172.21.15.59:0/3899130190 shutdown_connections 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.250+0000 7f2fc8400640 1 -- 172.21.15.59:0/3899130190 wait complete. 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 Processor -- start 2024-09-05T11:52:15.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 -- start start 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 0x7f2fc019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0105920 0x7f2fc01a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010fae0 con 0x7f2fc0104f60 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010f960 con 0x7f2fc0105920 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc8400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010fc60 con 0x7f2fc0103e00 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51388/0 (socket says 172.21.15.59:51388) 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc5000640 1 -- 172.21.15.59:0/82430666 learned_addr learned my addr 172.21.15.59:0/82430666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.251+0000 7f2fc5a00640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 0x7f2fc019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc6400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0105920 0x7f2fc01a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 -- 172.21.15.59:0/82430666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0105920 msgr2=0x7f2fc01a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0105920 0x7f2fc01a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 -- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 msgr2=0x7f2fc019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 -- 172.21.15.59:0/82430666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc01a1c80 con 0x7f2fc0103e00 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5000640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.252+0000 7f2fc5a00640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 0x7f2fc019a050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400e9f0 tx=0x7f2fb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.253+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400cd80 con 0x7f2fc0103e00 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.253+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01a1f70 con 0x7f2fc0103e00 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.253+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb40035f0 con 0x7f2fc0103e00 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.253+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4010670 con 0x7f2fc0103e00 2024-09-05T11:52:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.253+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01a2420 con 0x7f2fc0103e00 2024-09-05T11:52:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.254+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc010b120 con 0x7f2fc0103e00 2024-09-05T11:52:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.256+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2fb40037a0 con 0x7f2fc0103e00 2024-09-05T11:52:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.257+0000 7f2fbe000640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 0x7f2fa80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.257+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5582+0+0 (secure 0 0 0) 0x7f2fb40ce880 con 0x7f2fc0103e00 2024-09-05T11:52:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.257+0000 7f2fc5000640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 0x7f2fa80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.258+0000 7f2fc5000640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 0x7f2fa80a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fc019b570 tx=0x7f2fac005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:52:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.259+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fb409c1f0 con 0x7f2fc0103e00 2024-09-05T11:52:15.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:52:15.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T11:52:15.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T11:52:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:14 smithi189 ceph-mon[34772]: from='osd.8 ' entity='osd.8' 2024-09-05T11:52:15.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:15.454+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2fc0062630 con 0x7f2fa80a2bb0 2024-09-05T11:52:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:52:16.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:52:16.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:16.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:16.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:16.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:16.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:16.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:16.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] boot 2024-09-05T11:52:16.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T11:52:16.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:15 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:16.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] boot 2024-09-05T11:52:16.194 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T11:52:16.194 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:15 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: osd.8 [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] boot 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T11:52:16.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:15 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T11:52:17.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:17 smithi059 ceph-mon[30025]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T11:52:17.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:17 smithi189 ceph-mon[34772]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T11:52:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:17 smithi089 ceph-mon[34499]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T11:52:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:18 smithi059 ceph-mon[30025]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:18.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:18 smithi059 ceph-mon[30025]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T11:52:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:18 smithi189 ceph-mon[34772]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:18 smithi189 ceph-mon[34772]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T11:52:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:18 smithi089 ceph-mon[34499]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:18 smithi089 ceph-mon[34499]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T11:52:19.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T11:52:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]: dispatch 2024-09-05T11:52:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]': finished 2024-09-05T11:52:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T11:52:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:19 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1108531185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T11:52:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]: dispatch 2024-09-05T11:52:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]': finished 2024-09-05T11:52:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T11:52:19.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:19.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:19 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1108531185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]: dispatch 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2560687661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85bf4014-3af1-4004-ac79-add151bbd6df"}]': finished 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:19 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1108531185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:20.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:20 smithi189 ceph-mon[34772]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:20 smithi089 ceph-mon[34499]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:20.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:20 smithi059 ceph-mon[30025]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:22.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:22 smithi189 ceph-mon[34772]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-05T11:52:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:22 smithi089 ceph-mon[34499]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-05T11:52:22.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:22 smithi059 ceph-mon[30025]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-05T11:52:24.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:24 smithi189 ceph-mon[34772]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:52:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:24 smithi089 ceph-mon[34499]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:52:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:24 smithi059 ceph-mon[30025]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T11:52:26.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:26 smithi059 ceph-mon[30025]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T11:52:26.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:26 smithi189 ceph-mon[34772]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T11:52:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:26 smithi089 ceph-mon[34499]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T11:52:28.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:28 smithi059 ceph-mon[30025]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T11:52:28.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T11:52:28.705 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:28.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:28 smithi189 ceph-mon[34772]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T11:52:28.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T11:52:28.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:28 smithi089 ceph-mon[34499]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-05T11:52:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T11:52:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:29.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:29 smithi189 ceph-mon[34772]: Deploying daemon osd.9 on smithi059 2024-09-05T11:52:29.934 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:29 smithi059 ceph-mon[30025]: Deploying daemon osd.9 on smithi059 2024-09-05T11:52:29.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:29 smithi089 ceph-mon[34499]: Deploying daemon osd.9 on smithi059 2024-09-05T11:52:30.789 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:30 smithi059 ceph-mon[30025]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T11:52:30.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:30 smithi189 ceph-mon[34772]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T11:52:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:30 smithi089 ceph-mon[34499]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-05T11:52:32.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:32 smithi189 ceph-mon[34772]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:52:32.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:32 smithi059 ceph-mon[30025]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:52:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:32 smithi089 ceph-mon[34499]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:52:34.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:34 smithi059 ceph-mon[30025]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:34.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:34 smithi189 ceph-mon[34772]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:34 smithi089 ceph-mon[34499]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:35.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:35.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:35.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:36.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:36 smithi059 ceph-mon[30025]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:36.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:36 smithi089 ceph-mon[34499]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:36 smithi189 ceph-mon[34772]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:38.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.936+0000 7f2fbe000640 1 -- 172.21.15.59:0/82430666 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fc0062630 con 0x7f2fa80a2bb0 2024-09-05T11:52:38.938 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 9 on host 'smithi059' 2024-09-05T11:52:38.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 msgr2=0x7f2fa80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:38.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 0x7f2fa80a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fc019b570 tx=0x7f2fac005f90 comp rx=0 tx=0).stop 2024-09-05T11:52:38.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 msgr2=0x7f2fc019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:38.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 0x7f2fc019a050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400e9f0 tx=0x7f2fb400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 shutdown_connections 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fa80a2bb0 0x7f2fa80a5070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc0105920 0x7f2fc01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fc0104f60 0x7f2fc019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 --2- 172.21.15.59:0/82430666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fc0103e00 0x7f2fc019a050 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.939+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 >> 172.21.15.59:0/82430666 conn(0x7f2fc00ff5b0 msgr2=0x7f2fc01082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.940+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 shutdown_connections 2024-09-05T11:52:38.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:52:38.940+0000 7f2fc8400640 1 -- 172.21.15.59:0/82430666 wait complete. 2024-09-05T11:52:39.714 DEBUG:teuthology.orchestra.run.smithi059:osd.9> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.9.service 2024-09-05T11:52:39.718 INFO:tasks.cephadm:Deploying osd.10 on smithi089 with /dev/vg_nvme/lv_1... 2024-09-05T11:52:39.718 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T11:52:39.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:39.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:39.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:39.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:39.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:39.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:52:40.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:52:40.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:40.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:40.205 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 11:52:40 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T11:52:40.008+0000 7fc43fe24640 -1 osd.9 0 log_to_monitors true 2024-09-05T11:52:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:40 smithi089 ceph-mon[34499]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:40 smithi089 ceph-mon[34499]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T11:52:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:40 smithi059 ceph-mon[30025]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:40 smithi059 ceph-mon[30025]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T11:52:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:40 smithi189 ceph-mon[34772]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:40 smithi189 ceph-mon[34772]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T11:52:41.680 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T11:52:41.680 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: osdmap e62: 10 total, 9 up, 10 in 2024-09-05T11:52:41.680 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:52:41.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:41.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:52:41.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T11:52:41.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: osdmap e62: 10 total, 9 up, 10 in 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T11:52:41.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: osdmap e62: 10 total, 9 up, 10 in 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T11:52:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:42.633 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:42 smithi089 ceph-mon[34499]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:42.633 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:42 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:42 smithi059 ceph-mon[30025]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:42 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:42.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:42 smithi059 ceph-mon[30025]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' 2024-09-05T11:52:42.709 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 11:52:42 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T11:52:42.394+0000 7fc43b400640 -1 osd.9 0 waiting for initial osdmap 2024-09-05T11:52:42.709 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 11:52:42 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T11:52:42.427+0000 7fc430a00640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:52:42.738 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-05T11:52:42.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-09-05T11:52:42.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:42 smithi089 ceph-mon[34499]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' 2024-09-05T11:52:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T11:52:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:42 smithi189 ceph-mon[34772]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:42 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:42 smithi189 ceph-mon[34772]: from='osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877]' entity='osd.9' 2024-09-05T11:52:43.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 -- 172.21.15.89:0/3720127170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c1026c0 msgr2=0x7f753c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 --2- 172.21.15.89:0/3720127170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c1026c0 0x7f753c108f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ca0 tx=0x7f753402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 -- 172.21.15.89:0/3720127170 shutdown_connections 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 --2- 172.21.15.89:0/3720127170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c1026c0 0x7f753c108f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 --2- 172.21.15.89:0/3720127170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c101d00 0x7f753c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 --2- 172.21.15.89:0/3720127170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f753c100b00 0x7f753c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 -- 172.21.15.89:0/3720127170 >> 172.21.15.89:0/3720127170 conn(0x7f753c0fc290 msgr2=0x7f753c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 -- 172.21.15.89:0/3720127170 shutdown_connections 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.538+0000 7f7540d36640 1 -- 172.21.15.89:0/3720127170 wait complete. 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 Processor -- start 2024-09-05T11:52:43.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 -- start start 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f753c100b00 0x7f753c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c1026c0 0x7f753c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c880 con 0x7f753c100b00 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c700 con 0x7f753c101d00 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7540d36640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10ca00 con 0x7f753c1026c0 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7539d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7539d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44540/0 (socket says 172.21.15.89:44540) 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.539+0000 7f7539d9b640 1 -- 172.21.15.89:0/1251314254 learned_addr learned my addr 172.21.15.89:0/1251314254 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-05T11:52:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f753ad9d640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c1026c0 0x7f753c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 -- 172.21.15.89:0/1251314254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c1026c0 msgr2=0x7f753c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c1026c0 0x7f753c1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 -- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f753c100b00 msgr2=0x7f753c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f753c100b00 0x7f753c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 -- 172.21.15.89:0/1251314254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c1a1f20 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7539d9b640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b4a0 tx=0x7f752c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c014070 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c010af0 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c1a2210 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c017d10 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.540+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a2780 con 0x7f753c101d00 2024-09-05T11:52:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.541+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c1a2a40 con 0x7f753c101d00 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.542+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f752c017070 con 0x7f753c101d00 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.543+0000 7f752b7fe640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.543+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7f752c0cdcd0 con 0x7f753c101d00 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.543+0000 7f753a59c640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.544+0000 7f753a59c640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 0x7f75240a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f753c101b60 tx=0x7f7530006c90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:52:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.547+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f752c09b430 con 0x7f753c101d00 2024-09-05T11:52:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:52:43.725+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f753c1a2c10 con 0x7f75240a2bb0 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] boot 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: osdmap e64: 10 total, 10 up, 10 in 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:43.727 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:43.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:52:43.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:52:43.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:43.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] boot 2024-09-05T11:52:43.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: osdmap e64: 10 total, 10 up, 10 in 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:43.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: osd.9 [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] boot 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: osdmap e64: 10 total, 10 up, 10 in 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:52:44.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:52:44.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:44.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi059 2024-09-05T11:52:44.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-05T11:52:44.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:44.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:44.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:44.685 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:44.859 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi059 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:44.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi059 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:52:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:45.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:45 smithi089 ceph-mon[34499]: osdmap e65: 10 total, 10 up, 10 in 2024-09-05T11:52:45.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:45 smithi059 ceph-mon[30025]: osdmap e65: 10 total, 10 up, 10 in 2024-09-05T11:52:46.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:45 smithi189 ceph-mon[34772]: osdmap e65: 10 total, 10 up, 10 in 2024-09-05T11:52:46.913 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:46.913 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/3240419366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:46.913 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:46.914 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]': finished 2024-09-05T11:52:46.914 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T11:52:46.914 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/3240419366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]': finished 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T11:52:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:52:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/3240419366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]: dispatch 2024-09-05T11:52:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f83d4779-7bf1-47e6-93a7-e54379bc692f"}]': finished 2024-09-05T11:52:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T11:52:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:52:47.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:47 smithi089 ceph-mon[34499]: from='client.? 172.21.15.89:0/3381565861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:47.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.89:0/3381565861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:48.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:47 smithi189 ceph-mon[34772]: from='client.? 172.21.15.89:0/3381565861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:52:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:48 smithi089 ceph-mon[34499]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:48 smithi059 ceph-mon[30025]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:48 smithi189 ceph-mon[34772]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:50 smithi089 ceph-mon[34499]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:50.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:50 smithi059 ceph-mon[30025]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:50 smithi189 ceph-mon[34772]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:52 smithi089 ceph-mon[34499]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:52 smithi059 ceph-mon[30025]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:52 smithi189 ceph-mon[34772]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:54.825 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:54 smithi089 ceph-mon[34499]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:54 smithi059 ceph-mon[30025]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:54 smithi189 ceph-mon[34772]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:55.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T11:52:55.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T11:52:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T11:52:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:52:56.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:56 smithi089 ceph-mon[34499]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:56.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:56 smithi089 ceph-mon[34499]: Deploying daemon osd.10 on smithi089 2024-09-05T11:52:56.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:56 smithi059 ceph-mon[30025]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:56.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:56 smithi059 ceph-mon[30025]: Deploying daemon osd.10 on smithi089 2024-09-05T11:52:56.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:56 smithi189 ceph-mon[34772]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:56.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:56 smithi189 ceph-mon[34772]: Deploying daemon osd.10 on smithi089 2024-09-05T11:52:58.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:52:58 smithi089 ceph-mon[34499]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:52:58 smithi059 ceph-mon[30025]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:52:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:52:58 smithi189 ceph-mon[34772]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:00 smithi059 ceph-mon[30025]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:01.053 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:00 smithi089 ceph-mon[34499]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:01.053 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:01.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:01.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:00 smithi189 ceph-mon[34772]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:02.817 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:02 smithi089 ceph-mon[34499]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:02 smithi059 ceph-mon[30025]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:02 smithi189 ceph-mon[34772]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:03.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.597+0000 7f752b7fe640 1 -- 172.21.15.89:0/1251314254 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f753c1a2c10 con 0x7f75240a2bb0 2024-09-05T11:53:03.598 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 10 on host 'smithi089' 2024-09-05T11:53:03.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 msgr2=0x7f75240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:03.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 0x7f75240a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f753c101b60 tx=0x7f7530006c90 comp rx=0 tx=0).stop 2024-09-05T11:53:03.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 msgr2=0x7f753c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b4a0 tx=0x7f752c00b960 comp rx=0 tx=0).stop 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 shutdown_connections 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753c1026c0 0x7f753c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f753c101d00 0x7f753c19a830 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 --2- 172.21.15.89:0/1251314254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f753c100b00 0x7f753c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 >> 172.21.15.89:0/1251314254 conn(0x7f753c0fc290 msgr2=0x7f753c06a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 shutdown_connections 2024-09-05T11:53:03.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-05T11:53:03.599+0000 7f7540d36640 1 -- 172.21.15.89:0/1251314254 wait complete. 2024-09-05T11:53:04.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:04 smithi089 ceph-mon[34499]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:04.681 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:04.682 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:04 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:04.712 DEBUG:teuthology.orchestra.run.smithi089:osd.10> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.10.service 2024-09-05T11:53:04.714 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-09-05T11:53:04.715 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T11:53:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:04 smithi059 ceph-mon[30025]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:04 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:04.971 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:53:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:04 smithi189 ceph-mon[34772]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:04 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:05 smithi089 ceph-mon[34499]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:06.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:05 smithi059 ceph-mon[30025]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:06.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.239 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:05 smithi189 ceph-mon[34772]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:06.239 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.239 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:06.550 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 11:53:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T11:53:06.268+0000 7fd1c06c4640 -1 osd.10 0 log_to_monitors true 2024-09-05T11:53:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:06 smithi089 ceph-mon[34499]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:06 smithi089 ceph-mon[34499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:06 smithi059 ceph-mon[30025]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:06 smithi059 ceph-mon[30025]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:06 smithi189 ceph-mon[34772]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:06 smithi189 ceph-mon[34772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T11:53:07.709 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-05T11:53:07.736 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-09-05T11:53:07.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:07.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T11:53:07.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:07.987 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:53:08.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T11:53:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: osdmap e67: 11 total, 10 up, 11 in 2024-09-05T11:53:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:08.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:07 smithi189 ceph-mon[34772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: from='osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: osdmap e67: 11 total, 10 up, 11 in 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:08.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:07 smithi089 ceph-mon[34499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:08.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: osdmap e67: 11 total, 10 up, 11 in 2024-09-05T11:53:08.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:08.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:07 smithi059 ceph-mon[30025]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-05T11:53:08.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 -- 172.21.15.189:0/3588302621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101cd0 msgr2=0x7fc734102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:08.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 --2- 172.21.15.189:0/3588302621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101cd0 0x7fc734102150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc728009900 tx=0x7fc72802a410 comp rx=0 tx=0).stop 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 -- 172.21.15.189:0/3588302621 shutdown_connections 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 --2- 172.21.15.189:0/3588302621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734102690 0x7fc734108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 --2- 172.21.15.189:0/3588302621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101cd0 0x7fc734102150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 --2- 172.21.15.189:0/3588302621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc734100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 -- 172.21.15.189:0/3588302621 >> 172.21.15.189:0/3588302621 conn(0x7fc7340fc280 msgr2=0x7fc7340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 -- 172.21.15.189:0/3588302621 shutdown_connections 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.503+0000 7fc73933e640 1 -- 172.21.15.189:0/3588302621 wait complete. 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 Processor -- start 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 -- start start 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc73406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734101cd0 0x7fc73419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:08.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410c750 con 0x7fc734100ad0 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410c5d0 con 0x7fc734101cd0 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.504+0000 7fc73933e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410c8d0 con 0x7fc734102690 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:46466/0 (socket says 172.21.15.189:46466) 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 -- 172.21.15.189:0/2258220618 learned_addr learned my addr 172.21.15.189:0/2258220618 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7327fc640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734101cd0 0x7fc73419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc732ffd640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc73406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 -- 172.21.15.189:0/2258220618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734101cd0 msgr2=0x7fc73419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734101cd0 0x7fc73419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:08.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 -- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 msgr2=0x7fc73406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc73406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 -- 172.21.15.189:0/2258220618 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419fab0 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc732ffd640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc73406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc7337fe640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00eb10 tx=0x7fc72c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c004b20 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c004cc0 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010760 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73419fd40 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.505+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a0250 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.506+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734107e00 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.507+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc72c005020 con 0x7fc734102690 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.508+0000 7fc71bfff640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 0x7fc71c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:08.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.508+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7fc72c0cf020 con 0x7fc734102690 2024-09-05T11:53:08.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.508+0000 7fc732ffd640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 0x7fc71c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:08.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.509+0000 7fc732ffd640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 0x7fc71c0a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc734101b30 tx=0x7fc72400a5e0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:08.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.512+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc72c09c6f0 con 0x7fc734102690 2024-09-05T11:53:08.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:08.689+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc734062630 con 0x7fc71c0a2b80 2024-09-05T11:53:08.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T11:53:08.600+0000 7fd1bc647640 -1 osd.10 0 waiting for initial osdmap 2024-09-05T11:53:08.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T11:53:08.615+0000 7fd1b4467640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:53:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:53:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi089 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='osd.10 ' entity='osd.10' 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:53:09.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi089 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='osd.10 ' entity='osd.10' 2024-09-05T11:53:09.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:09.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:53:09.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:53:09.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi089 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='osd.10 ' entity='osd.10' 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T11:53:09.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] boot 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:09 smithi189 ceph-mon[34772]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T11:53:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:53:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:53:10.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:10.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] boot 2024-09-05T11:53:10.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T11:53:10.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:10.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:09 smithi089 ceph-mon[34499]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: osd.10 [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] boot 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T11:53:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:09 smithi059 ceph-mon[30025]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/1920594859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]': finished 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T11:53:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:11.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T11:53:11.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/1920594859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]': finished 2024-09-05T11:53:11.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T11:53:11.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:11.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T11:53:11.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/1920594859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]: dispatch 2024-09-05T11:53:11.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedf8ed4-fc52-4603-b9cd-19837038c324"}]': finished 2024-09-05T11:53:11.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T11:53:11.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:12.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:12 smithi189 ceph-mon[34772]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:12.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:12 smithi189 ceph-mon[34772]: from='client.? 172.21.15.189:0/2828469843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:53:12.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:12 smithi189 ceph-mon[34772]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T11:53:12.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:12 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:12 smithi089 ceph-mon[34499]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:12 smithi089 ceph-mon[34499]: from='client.? 172.21.15.189:0/2828469843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:53:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:12 smithi089 ceph-mon[34499]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T11:53:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:12 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:12 smithi059 ceph-mon[30025]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:12.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:12 smithi059 ceph-mon[30025]: from='client.? 172.21.15.189:0/2828469843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T11:53:12.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:12 smithi059 ceph-mon[30025]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T11:53:12.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:12 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:14.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:14 smithi089 ceph-mon[34499]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:14.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:14 smithi059 ceph-mon[30025]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:14.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:14 smithi189 ceph-mon[34772]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:16 smithi189 ceph-mon[34772]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:16 smithi059 ceph-mon[30025]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:16.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:16 smithi089 ceph-mon[34499]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:18.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:18 smithi189 ceph-mon[34772]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:53:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:18 smithi089 ceph-mon[34499]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:53:18.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:18 smithi059 ceph-mon[30025]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-05T11:53:20.724 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:20 smithi189 ceph-mon[34772]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:53:20.724 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T11:53:20.724 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:20 smithi089 ceph-mon[34499]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:53:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T11:53:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:20.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:20 smithi059 ceph-mon[30025]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T11:53:20.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T11:53:20.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:21.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:21 smithi189 ceph-mon[34772]: Deploying daemon osd.11 on smithi189 2024-09-05T11:53:21.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:21 smithi089 ceph-mon[34499]: Deploying daemon osd.11 on smithi189 2024-09-05T11:53:21.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:21 smithi059 ceph-mon[30025]: Deploying daemon osd.11 on smithi189 2024-09-05T11:53:22.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:22 smithi189 ceph-mon[34772]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-05T11:53:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:22 smithi089 ceph-mon[34499]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-05T11:53:22.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:22 smithi059 ceph-mon[30025]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-05T11:53:24.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:24 smithi189 ceph-mon[34772]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-05T11:53:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:24 smithi089 ceph-mon[34499]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-05T11:53:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:24 smithi059 ceph-mon[30025]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-05T11:53:26.777 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:26 smithi189 ceph-mon[34772]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:26.777 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:26.778 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:26.778 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:26 smithi089 ceph-mon[34499]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:26.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:26 smithi059 ceph-mon[30025]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:28.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:28 smithi189 ceph-mon[34772]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:28 smithi089 ceph-mon[34499]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:28 smithi059 ceph-mon[30025]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T11:53:29.470 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.469+0000 7fc71bfff640 1 -- 172.21.15.189:0/2258220618 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc734062630 con 0x7fc71c0a2b80 2024-09-05T11:53:29.470 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-09-05T11:53:29.473 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.471+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 msgr2=0x7fc71c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:29.473 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.471+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 0x7fc71c0a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc734101b30 tx=0x7fc72400a5e0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.473 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.471+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 msgr2=0x7fc73419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:29.473 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.471+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00eb10 tx=0x7fc72c00c2d0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.473 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 shutdown_connections 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc71c0a2b80 0x7fc71c0a5040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734102690 0x7fc73419f1a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc734101cd0 0x7fc73419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 --2- 172.21.15.189:0/2258220618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc734100ad0 0x7fc73406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 >> 172.21.15.189:0/2258220618 conn(0x7fc7340fc280 msgr2=0x7fc734104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 shutdown_connections 2024-09-05T11:53:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:29.472+0000 7fc73933e640 1 -- 172.21.15.189:0/2258220618 wait complete. 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:29 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:29 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:29 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:30.210 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.11.service 2024-09-05T11:53:30.212 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-05T11:53:30.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T11:53:30.600 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:30.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:30 smithi189 ceph-mon[34772]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:31.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:30 smithi059 ceph-mon[30025]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 -- 172.21.15.59:0/1971107928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c104f60 msgr2=0x7ff42c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 --2- 172.21.15.59:0/1971107928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c104f60 0x7ff42c1053e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009860 tx=0x7ff41c02a340 comp rx=0 tx=0).stop 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 -- 172.21.15.59:0/1971107928 shutdown_connections 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 --2- 172.21.15.59:0/1971107928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c105920 0x7ff42c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 --2- 172.21.15.59:0/1971107928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c104f60 0x7ff42c1053e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 --2- 172.21.15.59:0/1971107928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.171+0000 7ff434800640 1 -- 172.21.15.59:0/1971107928 >> 172.21.15.59:0/1971107928 conn(0x7ff42c0ff5b0 msgr2=0x7ff42c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.172+0000 7ff434800640 1 -- 172.21.15.59:0/1971107928 shutdown_connections 2024-09-05T11:53:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.172+0000 7ff434800640 1 -- 172.21.15.59:0/1971107928 wait complete. 2024-09-05T11:53:31.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.172+0000 7ff434800640 1 Processor -- start 2024-09-05T11:53:31.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 -- start start 2024-09-05T11:53:31.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c104f60 0x7ff42c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c105920 0x7ff42c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10fab0 con 0x7ff42c103e00 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10f930 con 0x7ff42c104f60 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff434800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10fc30 con 0x7ff42c105920 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff431e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff431e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46434/0 (socket says 172.21.15.59:46434) 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff431e00640 1 -- 172.21.15.59:0/3234998823 learned_addr learned my addr 172.21.15.59:0/3234998823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff432800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c105920 0x7ff42c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.173+0000 7ff431400640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c104f60 0x7ff42c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:31.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 -- 172.21.15.59:0/3234998823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c105920 msgr2=0x7ff42c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c105920 0x7ff42c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 -- 172.21.15.59:0/3234998823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c104f60 msgr2=0x7ff42c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c104f60 0x7ff42c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 -- 172.21.15.59:0/3234998823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c1a1d20 con 0x7ff42c103e00 2024-09-05T11:53:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff431e00640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9c0 tx=0x7ff42000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42000cde0 con 0x7ff42c103e00 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4200035f0 con 0x7ff42c103e00 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c1a1fb0 con 0x7ff42c103e00 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.174+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420010690 con 0x7ff42c103e00 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.175+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a2460 con 0x7ff42c103e00 2024-09-05T11:53:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.176+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c10b190 con 0x7ff42c103e00 2024-09-05T11:53:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.178+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff4200037a0 con 0x7ff42c103e00 2024-09-05T11:53:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.180+0000 7ff426a00640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 0x7ff4140a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.180+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7ff4200cee80 con 0x7ff42c103e00 2024-09-05T11:53:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.180+0000 7ff431400640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 0x7ff4140a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.181+0000 7ff431400640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 0x7ff4140a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff41c030410 tx=0x7ff41c030550 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:30 smithi089 ceph-mon[34499]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:31.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.183+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff42009c3b0 con 0x7ff42c103e00 2024-09-05T11:53:31.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.369+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff42c02d7c0 con 0x7ff42c103e00 2024-09-05T11:53:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.369+0000 7ff426a00640 1 -- 172.21.15.59:0/3234998823 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7ff42009bcb0 con 0x7ff42c103e00 2024-09-05T11:53:31.371 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 msgr2=0x7ff4140a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 0x7ff4140a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff41c030410 tx=0x7ff41c030550 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 msgr2=0x7ff42c19a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9c0 tx=0x7ff42000ee80 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 shutdown_connections 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff4140a2bb0 0x7ff4140a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff42c105920 0x7ff42c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff42c104f60 0x7ff42c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 --2- 172.21.15.59:0/3234998823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff42c103e00 0x7ff42c19a0f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 >> 172.21.15.59:0/3234998823 conn(0x7ff42c0ff5b0 msgr2=0x7ff42c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 shutdown_connections 2024-09-05T11:53:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:31.372+0000 7ff434800640 1 -- 172.21.15.59:0/3234998823 wait complete. 2024-09-05T11:53:32.085 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:31 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3234998823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:32.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:31 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3234998823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:32.089 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 11:53:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T11:53:31.842+0000 7fa41e9ce640 -1 osd.11 0 log_to_monitors true 2024-09-05T11:53:32.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:31 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3234998823' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:32.334 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725537188,"num_in_osds":12,"osd_in_since":1725537191,"num_remapped_pgs":0} 2024-09-05T11:53:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:32 smithi189 ceph-mon[34772]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:32 smithi189 ceph-mon[34772]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:32 smithi189 ceph-mon[34772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.097 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:32 smithi089 ceph-mon[34499]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:33.098 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:32 smithi089 ceph-mon[34499]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.098 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:32 smithi089 ceph-mon[34499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:32 smithi059 ceph-mon[30025]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:32 smithi059 ceph-mon[30025]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:32 smithi059 ceph-mon[30025]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T11:53:33.335 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T11:53:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:34.089 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T11:53:33.765+0000 7fa41a951640 -1 osd.11 0 waiting for initial osdmap 2024-09-05T11:53:34.089 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T11:53:33.779+0000 7fa416779640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:53:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T11:53:34.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:33 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T11:53:34.186 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:33 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-05T11:53:34.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:34.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-05T11:53:34.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T11:53:34.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:33 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1846637306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0103e00 msgr2=0x7f7cc0104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1846637306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0103e00 0x7f7cc0104200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009900 tx=0x7f7cb002a410 comp rx=0 tx=0).stop 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1846637306 shutdown_connections 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1846637306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0105920 0x7f7cc010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1846637306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0104f60 0x7f7cc01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1846637306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0103e00 0x7f7cc0104200 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1846637306 >> 172.21.15.59:0/1846637306 conn(0x7f7cc00ff5b0 msgr2=0x7f7cc01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1846637306 shutdown_connections 2024-09-05T11:53:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.348+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1846637306 wait complete. 2024-09-05T11:53:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 Processor -- start 2024-09-05T11:53:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 -- start start 2024-09-05T11:53:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 0x7f7cc019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 0x7f7cc01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010fa90 con 0x7f7cc0104f60 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010f910 con 0x7f7cc0103e00 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.349+0000 7f7cc4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc010fc10 con 0x7f7cc0105920 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 0x7f7cc019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:37842/0 (socket says 172.21.15.59:37842) 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbde00640 1 -- 172.21.15.59:0/1767952763 learned_addr learned my addr 172.21.15.59:0/1767952763 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbe800640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 0x7f7cc01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 -- 172.21.15.59:0/1767952763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 msgr2=0x7f7cc01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 0x7f7cc01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 -- 172.21.15.59:0/1767952763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 msgr2=0x7f7cc019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbd400640 1 -- 172.21.15.59:0/1767952763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc01a1d30 con 0x7f7cc0104f60 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbe800640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 0x7f7cc01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.350+0000 7f7cbde00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cbd400640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 0x7f7cc019a640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00e9c0 tx=0x7f7cac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00ccc0 con 0x7f7cc0104f60 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac0035f0 con 0x7f7cc0104f60 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc01a1fc0 con 0x7f7cc0104f60 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00ce60 con 0x7f7cc0104f60 2024-09-05T11:53:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.351+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01a2520 con 0x7f7cc0104f60 2024-09-05T11:53:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.352+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc010b1f0 con 0x7f7cc0104f60 2024-09-05T11:53:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.354+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7cac0037a0 con 0x7f7cc0104f60 2024-09-05T11:53:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.355+0000 7f7cb6a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 0x7f7ca40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.355+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7f7cac0cedf0 con 0x7f7cc0104f60 2024-09-05T11:53:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.356+0000 7f7cbde00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 0x7f7ca40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.356+0000 7f7cbde00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 0x7f7ca40a50f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0031440 tx=0x7f7cb00315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.359+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cac09c270 con 0x7f7cc0104f60 2024-09-05T11:53:34.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.546+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7cc0062e90 con 0x7f7cc0104f60 2024-09-05T11:53:34.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.546+0000 7f7cb6a00640 1 -- 172.21.15.59:0/1767952763 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f7cac09bb70 con 0x7f7cc0104f60 2024-09-05T11:53:34.548 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 msgr2=0x7f7ca40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 0x7f7ca40a50f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0031440 tx=0x7f7cb00315b0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 msgr2=0x7f7cc019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 0x7f7cc019a640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00e9c0 tx=0x7f7cac00ee80 comp rx=0 tx=0).stop 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 shutdown_connections 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ca40a2c30 0x7f7ca40a50f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cc0105920 0x7f7cc01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cc0104f60 0x7f7cc019a640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 --2- 172.21.15.59:0/1767952763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cc0103e00 0x7f7cc019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:34.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 >> 172.21.15.59:0/1767952763 conn(0x7f7cc00ff5b0 msgr2=0x7f7cc01083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:34.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 shutdown_connections 2024-09-05T11:53:34.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:34.549+0000 7f7cc4a00640 1 -- 172.21.15.59:0/1767952763 wait complete. 2024-09-05T11:53:34.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:34.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: Detected new or changed devices on smithi189 2024-09-05T11:53:34.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-05T11:53:34.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:34.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1767952763' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:34.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:34 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: Detected new or changed devices on smithi189 2024-09-05T11:53:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-05T11:53:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1767952763' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:34 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: Detected new or changed devices on smithi189 2024-09-05T11:53:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-05T11:53:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1767952763' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:35.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:34 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:35.345 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1725537188,"num_in_osds":12,"osd_in_since":1725537191,"num_remapped_pgs":0} 2024-09-05T11:53:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:35 smithi189 ceph-mon[34772]: purged_snaps scrub starts 2024-09-05T11:53:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:35 smithi189 ceph-mon[34772]: purged_snaps scrub ok 2024-09-05T11:53:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:35 smithi189 ceph-mon[34772]: osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] boot 2024-09-05T11:53:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:35 smithi189 ceph-mon[34772]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T11:53:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:35 smithi089 ceph-mon[34499]: purged_snaps scrub starts 2024-09-05T11:53:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:35 smithi089 ceph-mon[34499]: purged_snaps scrub ok 2024-09-05T11:53:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:35 smithi089 ceph-mon[34499]: osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] boot 2024-09-05T11:53:36.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:35 smithi089 ceph-mon[34499]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T11:53:36.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:36.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:35 smithi059 ceph-mon[30025]: purged_snaps scrub starts 2024-09-05T11:53:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:35 smithi059 ceph-mon[30025]: purged_snaps scrub ok 2024-09-05T11:53:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:35 smithi059 ceph-mon[30025]: osd.11 [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] boot 2024-09-05T11:53:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:35 smithi059 ceph-mon[30025]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T11:53:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T11:53:36.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T11:53:36.729 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:37.050 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:36 smithi059 ceph-mon[30025]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:37.050 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:36 smithi059 ceph-mon[30025]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T11:53:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:36 smithi189 ceph-mon[34772]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:36 smithi189 ceph-mon[34772]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T11:53:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:36 smithi089 ceph-mon[34499]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-05T11:53:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:36 smithi089 ceph-mon[34499]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T11:53:37.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 -- 172.21.15.59:0/1333508396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c105810 msgr2=0x7f417c10c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:37.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 --2- 172.21.15.59:0/1333508396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c105810 0x7f417c10c0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4174009ca0 tx=0x7f417402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 -- 172.21.15.59:0/1333508396 shutdown_connections 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 --2- 172.21.15.59:0/1333508396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c105810 0x7f417c10c0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 --2- 172.21.15.59:0/1333508396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 0x7f417c1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 --2- 172.21.15.59:0/1333508396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c103c50 0x7f417c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 -- 172.21.15.59:0/1333508396 >> 172.21.15.59:0/1333508396 conn(0x7f417c0ff3e0 msgr2=0x7f417c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 -- 172.21.15.59:0/1333508396 shutdown_connections 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.263+0000 7f4180a00640 1 -- 172.21.15.59:0/1333508396 wait complete. 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.264+0000 7f4180a00640 1 Processor -- start 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.264+0000 7f4180a00640 1 -- start start 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:37.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 0x7f417c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c105810 0x7f417c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10f930 con 0x7f417c103c50 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10f7b0 con 0x7f417c105810 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4180a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10fab0 con 0x7f417c104e50 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46470/0 (socket says 172.21.15.59:46470) 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 -- 172.21.15.59:0/1402969833 learned_addr learned my addr 172.21.15.59:0/1402969833 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f4173e00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 0x7f417c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417be00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c105810 0x7f417c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 -- 172.21.15.59:0/1402969833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 msgr2=0x7f417c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 0x7f417c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 -- 172.21.15.59:0/1402969833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c105810 msgr2=0x7f417c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c105810 0x7f417c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.265+0000 7f417b400640 1 -- 172.21.15.59:0/1402969833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c1a1b00 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f417b400640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e9c0 tx=0x7f416c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c00ccc0 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416c0035f0 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c1a1df0 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c00ce60 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.266+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a2320 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.267+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c10af80 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.269+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f416c0037a0 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.270+0000 7f4178c00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 0x7f41600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.271+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f416c0cef40 con 0x7f417c103c50 2024-09-05T11:53:37.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.271+0000 7f4173e00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 0x7f41600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:37.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.271+0000 7f4173e00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 0x7f41600a50f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f417c19b3f0 tx=0x7f4164009110 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:37.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.275+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f416c09c2a0 con 0x7f417c103c50 2024-09-05T11:53:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.458+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f417c062e90 con 0x7f417c103c50 2024-09-05T11:53:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.459+0000 7f4178c00640 1 -- 172.21.15.59:0/1402969833 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f416c09bba0 con 0x7f417c103c50 2024-09-05T11:53:37.461 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 msgr2=0x7f41600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 0x7f41600a50f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f417c19b3f0 tx=0x7f4164009110 comp rx=0 tx=0).stop 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 msgr2=0x7f417c199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e9c0 tx=0x7f416c00edc0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 shutdown_connections 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41600a2c30 0x7f41600a50f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f417c105810 0x7f417c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f417c104e50 0x7f417c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 --2- 172.21.15.59:0/1402969833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c103c50 0x7f417c199ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 >> 172.21.15.59:0/1402969833 conn(0x7f417c0ff3e0 msgr2=0x7f417c108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 shutdown_connections 2024-09-05T11:53:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:37.461+0000 7f4180a00640 1 -- 172.21.15.59:0/1402969833 wait complete. 2024-09-05T11:53:38.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:37 smithi189 ceph-mon[34772]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:38.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:37 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1402969833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:37 smithi089 ceph-mon[34499]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:37 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1402969833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:37 smithi059 ceph-mon[30025]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:37 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1402969833' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T11:53:38.294 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1725537214,"num_in_osds":12,"osd_in_since":1725537191,"num_remapped_pgs":0} 2024-09-05T11:53:38.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:53:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.172+0000 7f6d01400640 1 -- 172.21.15.59:0/702235618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc1049a0 msgr2=0x7f6cfc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.172+0000 7f6d01400640 1 --2- 172.21.15.59:0/702235618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc1089f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009900 tx=0x7f6ce402a410 comp rx=0 tx=0).stop 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 -- 172.21.15.59:0/702235618 shutdown_connections 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 --2- 172.21.15.59:0/702235618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 --2- 172.21.15.59:0/702235618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc1089f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 --2- 172.21.15.59:0/702235618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 -- 172.21.15.59:0/702235618 >> 172.21.15.59:0/702235618 conn(0x7f6cfc0ffa90 msgr2=0x7f6cfc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 -- 172.21.15.59:0/702235618 shutdown_connections 2024-09-05T11:53:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.173+0000 7f6d01400640 1 -- 172.21.15.59:0/702235618 wait complete. 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 Processor -- start 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 -- start start 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc114190 con 0x7f6cfc103ff0 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc114010 con 0x7f6cfc109120 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6d01400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc114310 con 0x7f6cfc1049a0 2024-09-05T11:53:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.174+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46486/0 (socket says 172.21.15.59:46486) 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 -- 172.21.15.59:0/1136818743 learned_addr learned my addr 172.21.15.59:0/1136818743 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cf3e00640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfb200640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 -- 172.21.15.59:0/1136818743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc1049a0 msgr2=0x7f6cfc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 -- 172.21.15.59:0/1136818743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 msgr2=0x7f6cfc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 -- 172.21.15.59:0/1136818743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc19f5c0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfb200640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.175+0000 7f6cfa800640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00e9f0 tx=0x7f6cec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.176+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00cd80 con 0x7f6cfc103ff0 2024-09-05T11:53:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.176+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cec005500 con 0x7f6cfc103ff0 2024-09-05T11:53:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.176+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec010690 con 0x7f6cfc103ff0 2024-09-05T11:53:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.176+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc19f8b0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.176+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1a7190 con 0x7f6cfc103ff0 2024-09-05T11:53:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.177+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc1043d0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.179+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6cec0046c0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.180+0000 7f6cf3400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 0x7f6ce00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.180+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6cec0cefc0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.180+0000 7f6cf3e00640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 0x7f6ce00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.181+0000 7f6cf3e00640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 0x7f6ce00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.184+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cec09c320 con 0x7f6cfc103ff0 2024-09-05T11:53:39.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.369+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cfc10c3c0 con 0x7f6cfc103ff0 2024-09-05T11:53:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.371+0000 7f6cf3400640 1 -- 172.21.15.59:0/1136818743 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18458 (secure 0 0 0) 0x7f6cec0b4e20 con 0x7f6cfc103ff0 2024-09-05T11:53:39.373 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:39.373 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":77,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:53:35.777363+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 msgr2=0x7f6ce00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 0x7f6ce00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4005b50 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 msgr2=0x7f6cfc19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00e9f0 tx=0x7f6cec00edf0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 shutdown_connections 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6ce00a2c00 0x7f6ce00a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6cfc109120 0x7f6cfc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cfc1049a0 0x7f6cfc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 --2- 172.21.15.59:0/1136818743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cfc103ff0 0x7f6cfc19a610 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 >> 172.21.15.59:0/1136818743 conn(0x7f6cfc0ffa90 msgr2=0x7f6cfc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:39.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 shutdown_connections 2024-09-05T11:53:39.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:39.373+0000 7f6d01400640 1 -- 172.21.15.59:0/1136818743 wait complete. 2024-09-05T11:53:40.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:39 smithi059 ceph-mon[30025]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:40.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:39 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1136818743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:53:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:39 smithi089 ceph-mon[34499]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:39 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1136818743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:53:40.200 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T11:49:37.375375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T11:53:40.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-05T11:53:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:39 smithi189 ceph-mon[34772]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:39 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1136818743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:53:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:41.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 -- 172.21.15.59:0/1975931405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 msgr2=0x7f445c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:41.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 --2- 172.21.15.59:0/1975931405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c1043d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445002a410 comp rx=0 tx=0).stop 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 -- 172.21.15.59:0/1975931405 shutdown_connections 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 --2- 172.21.15.59:0/1975931405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 --2- 172.21.15.59:0/1975931405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 0x7f445c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 --2- 172.21.15.59:0/1975931405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c1043d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 -- 172.21.15.59:0/1975931405 >> 172.21.15.59:0/1975931405 conn(0x7f445c0ffa90 msgr2=0x7f445c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 -- 172.21.15.59:0/1975931405 shutdown_connections 2024-09-05T11:53:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.127+0000 7f4463600640 1 -- 172.21.15.59:0/1975931405 wait complete. 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 Processor -- start 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 -- start start 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 0x7f445c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f445c114150 con 0x7f445c103ff0 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f445c113fd0 con 0x7f445c1049a0 2024-09-05T11:53:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.128+0000 7f4463600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f445c1142d0 con 0x7f445c109120 2024-09-05T11:53:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4460c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4460c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55830/0 (socket says 172.21.15.59:55830) 2024-09-05T11:53:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4460c00640 1 -- 172.21.15.59:0/2335260698 learned_addr learned my addr 172.21.15.59:0/2335260698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f445be00640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 0x7f445c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 -- 172.21.15.59:0/2335260698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 msgr2=0x7f445c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 0x7f445c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 -- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 msgr2=0x7f445c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 -- 172.21.15.59:0/2335260698 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c19f5a0 con 0x7f445c109120 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.129+0000 7f4461600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c19eec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4454009fd0 tx=0x7f445400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.130+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445400ce20 con 0x7f445c109120 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.130+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4454005020 con 0x7f445c109120 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.130+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c19f890 con 0x7f445c109120 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.130+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454010720 con 0x7f445c109120 2024-09-05T11:53:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.130+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c1a7170 con 0x7f445c109120 2024-09-05T11:53:41.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.131+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c1043d0 con 0x7f445c109120 2024-09-05T11:53:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.133+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4454005540 con 0x7f445c109120 2024-09-05T11:53:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.134+0000 7f4459600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 0x7f44440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.134+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f44540cf1a0 con 0x7f445c109120 2024-09-05T11:53:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.134+0000 7f4460c00640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 0x7f44440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.135+0000 7f4460c00640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 0x7f44440a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44500096c0 tx=0x7f4450005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:41.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.137+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f445409c500 con 0x7f445c109120 2024-09-05T11:53:41.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.321+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f445c02d7c0 con 0x7f445c109120 2024-09-05T11:53:41.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.322+0000 7f4459600640 1 -- 172.21.15.59:0/2335260698 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f445409be00 con 0x7f445c109120 2024-09-05T11:53:41.323 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 msgr2=0x7f44440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 0x7f44440a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44500096c0 tx=0x7f4450005a20 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 msgr2=0x7f445c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c19eec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4454009fd0 tx=0x7f445400ef10 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 shutdown_connections 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44440a2b80 0x7f44440a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f445c109120 0x7f445c19eec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f445c1049a0 0x7f445c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 --2- 172.21.15.59:0/2335260698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f445c103ff0 0x7f445c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 >> 172.21.15.59:0/2335260698 conn(0x7f445c0ffa90 msgr2=0x7f445c100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 shutdown_connections 2024-09-05T11:53:41.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:41.324+0000 7f4463600640 1 -- 172.21.15.59:0/2335260698 wait complete. 2024-09-05T11:53:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:41 smithi059 ceph-mon[30025]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2335260698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.a on smithi059 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.d on smithi059 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.g on smithi059 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.j on smithi059 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.b on smithi089 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.e on smithi089 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.h on smithi089 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.k on smithi089 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-09-05T11:53:42.063 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-09-05T11:53:42.064 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph orch apply mds all '12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-09-05T11:53:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:41 smithi089 ceph-mon[34499]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2335260698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:53:42.324 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T11:53:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:41 smithi189 ceph-mon[34772]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2335260698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 -- 172.21.15.189:0/293146221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068100ad0 msgr2=0x7f8068100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 --2- 172.21.15.189:0/293146221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068100ad0 0x7f8068100ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8064009900 tx=0x7f806402a410 comp rx=0 tx=0).stop 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 -- 172.21.15.189:0/293146221 shutdown_connections 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 --2- 172.21.15.189:0/293146221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068102690 0x7f8068108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 --2- 172.21.15.189:0/293146221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068101cd0 0x7f8068102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 --2- 172.21.15.189:0/293146221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068100ad0 0x7f8068100ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:42.856 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.841+0000 7f8070114640 1 -- 172.21.15.189:0/293146221 >> 172.21.15.189:0/293146221 conn(0x7f80680fc280 msgr2=0x7f80680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- 172.21.15.189:0/293146221 shutdown_connections 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- 172.21.15.189:0/293146221 wait complete. 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 Processor -- start 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- start start 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 0x7f8068196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068101cd0 0x7f8068197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f806810c730 con 0x7f8068100ad0 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f806810c5b0 con 0x7f8068101cd0 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.842+0000 7f8070114640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f806810c8b0 con 0x7f8068102690 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:33152/0 (socket says 172.21.15.189:33152) 2024-09-05T11:53:42.857 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 -- 172.21.15.189:0/2604700521 learned_addr learned my addr 172.21.15.189:0/2604700521 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806d6af640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068101cd0 0x7f8068197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806deb0640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 0x7f8068196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 -- 172.21.15.189:0/2604700521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068101cd0 msgr2=0x7f8068197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068101cd0 0x7f8068197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 -- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 msgr2=0x7f8068196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 0x7f8068196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 -- 172.21.15.189:0/2604700521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806819ea10 con 0x7f8068102690 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806deb0640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 0x7f8068196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f806e6b1640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f806000cad0 tx=0x7f806000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806000ae20 con 0x7f8068102690 2024-09-05T11:53:42.858 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8068069fd0 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8060005020 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806000f6f0 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.843+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806806a4c0 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.844+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068100ed0 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.846+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80600051d0 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.846+0000 7f805effd640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 0x7f80540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.847+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f80600cec30 con 0x7f8068102690 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.847+0000 7f806deb0640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 0x7f80540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.847+0000 7f806deb0640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 0x7f80540a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f8064032dd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:42.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:42.850+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8060014030 con 0x7f8068102690 2024-09-05T11:53:43.038 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.036+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7f806806aa40 con 0x7f80540a2bb0 2024-09-05T11:53:43.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.046+0000 7f805effd640 1 -- 172.21.15.189:0/2604700521 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f806806aa40 con 0x7f80540a2bb0 2024-09-05T11:53:43.047 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 msgr2=0x7f80540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 0x7f80540a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f8064032dd0 comp rx=0 tx=0).stop 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 msgr2=0x7f806819e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f806000cad0 tx=0x7f806000cf90 comp rx=0 tx=0).stop 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 shutdown_connections 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f80540a2bb0 0x7f80540a5070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8068102690 0x7f806819e3a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:43.050 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8068101cd0 0x7f8068197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:43.051 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 --2- 172.21.15.189:0/2604700521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8068100ad0 0x7f8068196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:43.051 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 >> 172.21.15.189:0/2604700521 conn(0x7f80680fc280 msgr2=0x7f80681050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:43.051 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 shutdown_connections 2024-09-05T11:53:43.051 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-05T11:53:43.048+0000 7f8070114640 1 -- 172.21.15.189:0/2604700521 wait complete. 2024-09-05T11:53:43.747 DEBUG:teuthology.orchestra.run.smithi059:mds.a> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.a.service 2024-09-05T11:53:43.752 DEBUG:teuthology.orchestra.run.smithi059:mds.d> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.d.service 2024-09-05T11:53:43.797 DEBUG:teuthology.orchestra.run.smithi059:mds.g> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.g.service 2024-09-05T11:53:43.841 DEBUG:teuthology.orchestra.run.smithi059:mds.j> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.j.service 2024-09-05T11:53:43.886 DEBUG:teuthology.orchestra.run.smithi089:mds.b> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.b.service 2024-09-05T11:53:43.889 DEBUG:teuthology.orchestra.run.smithi089:mds.e> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.e.service 2024-09-05T11:53:43.932 DEBUG:teuthology.orchestra.run.smithi089:mds.h> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.h.service 2024-09-05T11:53:43.975 DEBUG:teuthology.orchestra.run.smithi089:mds.k> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.k.service 2024-09-05T11:53:44.018 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.c.service 2024-09-05T11:53:44.021 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.f.service 2024-09-05T11:53:44.065 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.i.service 2024-09-05T11:53:44.108 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.l.service 2024-09-05T11:53:44.151 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-05T11:53:44.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:53:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-05T11:53:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:44 smithi089 ceph-mon[34499]: Deploying daemon mds.l on smithi189 2024-09-05T11:53:44.199 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:44 smithi059 ceph-mon[30025]: Deploying daemon mds.l on smithi189 2024-09-05T11:53:44.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:44.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:53:44.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:44.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:44 smithi189 ceph-mon[34772]: Deploying daemon mds.l on smithi189 2024-09-05T11:53:44.546 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:45.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 -- 172.21.15.59:0/3316165026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f01019c0 msgr2=0x7f00f0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:45.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3316165026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f01019c0 0x7f00f0101e20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009930 tx=0x7f00dc02a420 comp rx=0 tx=0).stop 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 -- 172.21.15.59:0/3316165026 shutdown_connections 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3316165026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0102360 0x7f00f010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3316165026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f01019c0 0x7f00f0101e20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3316165026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01079c0 0x7f00f0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 -- 172.21.15.59:0/3316165026 >> 172.21.15.59:0/3316165026 conn(0x7f00f00fd740 msgr2=0x7f00f00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 -- 172.21.15.59:0/3316165026 shutdown_connections 2024-09-05T11:53:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.052+0000 7f00f6a00640 1 -- 172.21.15.59:0/3316165026 wait complete. 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.053+0000 7f00f6a00640 1 Processor -- start 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 -- start start 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 0x7f00f019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 0x7f00f019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111e10 con 0x7f00f01079c0 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111c90 con 0x7f00f01019c0 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111f90 con 0x7f00f0102360 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 0x7f00f019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:45.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55768/0 (socket says 172.21.15.59:55768) 2024-09-05T11:53:45.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.054+0000 7f00f4c00640 1 -- 172.21.15.59:0/3459536459 learned_addr learned my addr 172.21.15.59:0/3459536459 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:45.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f6000640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 0x7f00f019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:45.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 -- 172.21.15.59:0/3459536459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 msgr2=0x7f00f019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 0x7f00f019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 -- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 msgr2=0x7f00f019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 0x7f00f019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 -- 172.21.15.59:0/3459536459 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f019f470 con 0x7f00f0102360 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f5600640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 0x7f00f019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f6000640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 0x7f00f019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.055+0000 7f00f4c00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02a940 tx=0x7f00dc003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.056+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc03f070 con 0x7f00f0102360 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.056+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f019f700 con 0x7f00f0102360 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.056+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc031c90 con 0x7f00f0102360 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.056+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc035030 con 0x7f00f0102360 2024-09-05T11:53:45.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.056+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01a7040 con 0x7f00f0102360 2024-09-05T11:53:45.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.057+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0101e20 con 0x7f00f0102360 2024-09-05T11:53:45.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.059+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00dc042070 con 0x7f00f0102360 2024-09-05T11:53:45.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.061+0000 7f00ee000640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 0x7f00d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:45.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.061+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f00dc06de00 con 0x7f00f0102360 2024-09-05T11:53:45.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.061+0000 7f00f5600640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 0x7f00d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:45.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.062+0000 7f00f5600640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 0x7f00d80a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009f40 tx=0x7f00e4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:45.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.062+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00dc0bc940 con 0x7f00f0102360 2024-09-05T11:53:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.286+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f01a7240 con 0x7f00f0102360 2024-09-05T11:53:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.287+0000 7f00ee000640 1 -- 172.21.15.59:0/3459536459 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f00dc0bc240 con 0x7f00f0102360 2024-09-05T11:53:45.288 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:45.289 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":1,"btime":"2024-09-05T11:46:24:063581+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-05T11:53:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 1 2024-09-05T11:53:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.289+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 msgr2=0x7f00d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.289+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 0x7f00d80a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009f40 tx=0x7f00e4005f90 comp rx=0 tx=0).stop 2024-09-05T11:53:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.289+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 msgr2=0x7f00f019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.289+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00dc02a940 tx=0x7f00dc003b20 comp rx=0 tx=0).stop 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 shutdown_connections 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00d80a2b80 0x7f00d80a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f01079c0 0x7f00f019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f0102360 0x7f00f019aa00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 --2- 172.21.15.59:0/3459536459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f01019c0 0x7f00f019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 >> 172.21.15.59:0/3459536459 conn(0x7f00f00fd740 msgr2=0x7f00f00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 shutdown_connections 2024-09-05T11:53:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:45.290+0000 7f00f6a00640 1 -- 172.21.15.59:0/3459536459 wait complete. 2024-09-05T11:53:45.589 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 systemd[1]: Starting Ceph mds.l for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:53:45.882 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 podman[63903]: 2024-09-05 11:53:45.530634118 +0000 UTC m=+0.015947535 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:53:45.882 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 podman[63903]: 2024-09-05 11:53:45.645919182 +0000 UTC m=+0.131232589 container create 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T11:53:45.882 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 podman[63903]: 2024-09-05 11:53:45.881122478 +0000 UTC m=+0.366435888 container init 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 podman[63903]: 2024-09-05 11:53:45.883970635 +0000 UTC m=+0.369284055 container start 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 ceph-mds[63922]: -- 172.21.15.189:0/1599634186 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1664bb860 con 0x55f166499400 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l[63918]: starting mds.l at 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 bash[63903]: 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 11:53:45 smithi189 systemd[1]: Started Ceph mds.l for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3459536459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.189:6832/3760489514,v1:172.21.15.189:6833/3760489514] up:boot 2024-09-05T11:53:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: fsmap 1 up:standby 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3459536459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.189:6832/3760489514,v1:172.21.15.189:6833/3760489514] up:boot 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: fsmap 1 up:standby 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:46.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3459536459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.189:6832/3760489514,v1:172.21.15.189:6833/3760489514] up:boot 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: fsmap 1 up:standby 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:46.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:46.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:47 smithi089 ceph-mon[34499]: Deploying daemon mds.k on smithi089 2024-09-05T11:53:47.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:47 smithi189 ceph-mon[34772]: Deploying daemon mds.k on smithi089 2024-09-05T11:53:47.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:47 smithi059 ceph-mon[30025]: Deploying daemon mds.k on smithi089 2024-09-05T11:53:48.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:53:48.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:48 smithi189 ceph-mon[34772]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:48.413 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:48 smithi089 ceph-mon[34499]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:48.414 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 systemd[1]: Starting Ceph mds.k for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:53:48.446 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:48.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:48 smithi059 ceph-mon[30025]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:48.680 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 podman[63066]: 2024-09-05 11:53:48.315114242 +0000 UTC m=+0.015529436 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:53:48.680 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 podman[63066]: 2024-09-05 11:53:48.434709956 +0000 UTC m=+0.135125139 container create e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T11:53:48.680 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 podman[63066]: 2024-09-05 11:53:48.679158197 +0000 UTC m=+0.379573380 container init e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T11:53:48.934 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 podman[63066]: 2024-09-05 11:53:48.682102831 +0000 UTC m=+0.382518024 container start e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T11:53:48.934 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 ceph-mds[63085]: -- 172.21.15.89:0/1731884581 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cf9781860 con 0x556cf975e800 2024-09-05T11:53:48.934 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k[63081]: starting mds.k at 2024-09-05T11:53:48.934 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 bash[63066]: e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 2024-09-05T11:53:48.935 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 11:53:48 smithi089 systemd[1]: Started Ceph mds.k for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:53:49.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 -- 172.21.15.59:0/3375688335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8076720 msgr2=0x7f72c8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:49.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 --2- 172.21.15.59:0/3375688335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8076720 0x7f72c8076b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b802a410 comp rx=0 tx=0).stop 2024-09-05T11:53:49.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 -- 172.21.15.59:0/3375688335 shutdown_connections 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 --2- 172.21.15.59:0/3375688335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8112a30 0x7f72c8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 --2- 172.21.15.59:0/3375688335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 --2- 172.21.15.59:0/3375688335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8076720 0x7f72c8076b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.010+0000 7f72cd400640 1 -- 172.21.15.59:0/3375688335 >> 172.21.15.59:0/3375688335 conn(0x7f72c80ffa90 msgr2=0x7f72c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.011+0000 7f72cd400640 1 -- 172.21.15.59:0/3375688335 shutdown_connections 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.011+0000 7f72cd400640 1 -- 172.21.15.59:0/3375688335 wait complete. 2024-09-05T11:53:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.012+0000 7f72cd400640 1 Processor -- start 2024-09-05T11:53:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.012+0000 7f72cd400640 1 -- start start 2024-09-05T11:53:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.012+0000 7f72cd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 0x7f72c81a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72cd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 0x7f72c819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72cd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8118610 con 0x7f72c8112a30 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72cd400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8118490 con 0x7f72c8077040 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72cd400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8118790 con 0x7f72c8076720 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 0x7f72c819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 0x7f72c81a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 0x7f72c81a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55786/0 (socket says 172.21.15.59:55786) 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 0x7f72c819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55870/0 (socket says 172.21.15.59:55870) 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40618/0 (socket says 172.21.15.59:40618) 2024-09-05T11:53:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c6800640 1 -- 172.21.15.59:0/2094374170 learned_addr learned my addr 172.21.15.59:0/2094374170 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 -- 172.21.15.59:0/2094374170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 msgr2=0x7f72c81a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 0x7f72c81a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 -- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 msgr2=0x7f72c819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 0x7f72c819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.013+0000 7f72c5e00640 1 -- 172.21.15.59:0/2094374170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c819efd0 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72c5e00640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f72b400e9f0 tx=0x7f72b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b400ccc0 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c819f2c0 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b4010910 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4018960 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.014+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81ab5b0 con 0x7f72c8077040 2024-09-05T11:53:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.015+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c8076b00 con 0x7f72c8077040 2024-09-05T11:53:49.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.018+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f72b4010ac0 con 0x7f72c8077040 2024-09-05T11:53:49.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.019+0000 7f72bf400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 0x7f72ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.019+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f72b40cefd0 con 0x7f72c8077040 2024-09-05T11:53:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.019+0000 7f72c6800640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 0x7f72ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.020+0000 7f72c6800640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 0x7f72ac0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f72b80096c0 tx=0x7f72b80046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:49.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.023+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72b409bce0 con 0x7f72c8077040 2024-09-05T11:53:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.253+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72c81ab870 con 0x7f72c8077040 2024-09-05T11:53:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.253+0000 7f72bf400640 1 -- 172.21.15.59:0/2094374170 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f72b409ffe0 con 0x7f72c8077040 2024-09-05T11:53:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-09-05T11:53:49.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:49.256 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"btime":"2024-09-05T11:53:49:061401+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 msgr2=0x7f72ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 0x7f72ac0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f72b80096c0 tx=0x7f72b80046c0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 msgr2=0x7f72c81a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f72b400e9f0 tx=0x7f72b400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 shutdown_connections 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72ac0a2b80 0x7f72ac0a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72c8112a30 0x7f72c819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72c8077040 0x7f72c81a4c20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 --2- 172.21.15.59:0/2094374170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8076720 0x7f72c81a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 >> 172.21.15.59:0/2094374170 conn(0x7f72c80ffa90 msgr2=0x7f72c8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.256+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 shutdown_connections 2024-09-05T11:53:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:49.257+0000 7f72cd400640 1 -- 172.21.15.59:0/2094374170 wait complete. 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: Deploying daemon mds.j on smithi059 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6832/2738174591,v1:172.21.15.89:6833/2738174591] up:boot 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: fsmap 2 up:standby 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T11:53:49.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:49 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2094374170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: Deploying daemon mds.j on smithi059 2024-09-05T11:53:50.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6832/2738174591,v1:172.21.15.89:6833/2738174591] up:boot 2024-09-05T11:53:50.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: fsmap 2 up:standby 2024-09-05T11:53:50.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T11:53:50.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:49 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2094374170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: Deploying daemon mds.j on smithi059 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6832/2738174591,v1:172.21.15.89:6833/2738174591] up:boot 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: fsmap 2 up:standby 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T11:53:50.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:49 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2094374170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:52.009 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:51 smithi059 ceph-mon[30025]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:52.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:53:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:51 smithi089 ceph-mon[34499]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:51 smithi189 ceph-mon[34772]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:52.614 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 podman[69342]: 2024-09-05 11:53:52.351981709 +0000 UTC m=+0.146497867 container create ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T11:53:52.614 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 podman[69342]: 2024-09-05 11:53:52.613153304 +0000 UTC m=+0.407669466 container init ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-05T11:53:52.781 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:52.925 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 podman[69342]: 2024-09-05 11:53:52.619858005 +0000 UTC m=+0.414374177 container start ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T11:53:52.925 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 ceph-mds[69385]: -- 172.21.15.59:0/1018315052 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55740c31f860 con 0x55740c2fc800 2024-09-05T11:53:52.926 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j[69371]: starting mds.j at 2024-09-05T11:53:52.926 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 bash[69342]: ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd 2024-09-05T11:53:52.926 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 11:53:52 smithi059 systemd[1]: Started Ceph mds.j for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:53:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 -- 172.21.15.59:0/1146570364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc8102360 msgr2=0x7f5cc810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 --2- 172.21.15.59:0/1146570364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc8102360 0x7f5cc810e890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009cd0 tx=0x7f5cc002a8f0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 -- 172.21.15.59:0/1146570364 shutdown_connections 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 --2- 172.21.15.59:0/1146570364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc8102360 0x7f5cc810e890 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 --2- 172.21.15.59:0/1146570364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc81019c0 0x7f5cc8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 --2- 172.21.15.59:0/1146570364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 0x7f5cc8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 -- 172.21.15.59:0/1146570364 >> 172.21.15.59:0/1146570364 conn(0x7f5cc80fd740 msgr2=0x7f5cc80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 -- 172.21.15.59:0/1146570364 shutdown_connections 2024-09-05T11:53:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.364+0000 7f5ccee00640 1 -- 172.21.15.59:0/1146570364 wait complete. 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.365+0000 7f5ccee00640 1 Processor -- start 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 -- start start 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 0x7f5cc8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 0x7f5cc8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8111dc0 con 0x7f5cc81019c0 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8111c40 con 0x7f5cc8102360 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8111f40 con 0x7f5cc81079c0 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52666/0 (socket says 172.21.15.59:52666) 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccda00640 1 -- 172.21.15.59:0/3192507911 learned_addr learned my addr 172.21.15.59:0/3192507911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5ccd000640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 0x7f5cc8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.366+0000 7f5cce400640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 0x7f5cc8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 -- 172.21.15.59:0/3192507911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 msgr2=0x7f5cc8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 0x7f5cc8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 -- 172.21.15.59:0/3192507911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 msgr2=0x7f5cc8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 0x7f5cc8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 -- 172.21.15.59:0/3192507911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81a6740 con 0x7f5cc81019c0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccd000640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 0x7f5cc8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5ccda00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d770 tx=0x7f5cbc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.367+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc017070 con 0x7f5cc81019c0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.368+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc005500 con 0x7f5cc81019c0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.368+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81a6a30 con 0x7f5cc81019c0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.368+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc004e00 con 0x7f5cc81019c0 2024-09-05T11:53:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.368+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a6f70 con 0x7f5cc81019c0 2024-09-05T11:53:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.369+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc8101e20 con 0x7f5cc81019c0 2024-09-05T11:53:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.371+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5cbc004fa0 con 0x7f5cc81019c0 2024-09-05T11:53:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.372+0000 7f5cc6000640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 0x7f5cb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.372+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f5cbc0ce4d0 con 0x7f5cc81019c0 2024-09-05T11:53:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.372+0000 7f5ccd000640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 0x7f5cb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.373+0000 7f5ccd000640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 0x7f5cb00a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8075f80 tx=0x7f5cb4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:53.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.377+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cbc09b1e0 con 0x7f5cc81019c0 2024-09-05T11:53:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.622+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc8109f30 con 0x7f5cc81019c0 2024-09-05T11:53:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.622+0000 7f5cc6000640 1 -- 172.21.15.59:0/3192507911 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f5cbc09f4e0 con 0x7f5cc81019c0 2024-09-05T11:53:53.624 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:53.624 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":4,"btime":"2024-09-05T11:53:52:894117+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T11:53:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 4 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 msgr2=0x7f5cb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 0x7f5cb00a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8075f80 tx=0x7f5cb4005f90 comp rx=0 tx=0).stop 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 msgr2=0x7f5cc80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d770 tx=0x7f5cbc00dc30 comp rx=0 tx=0).stop 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 shutdown_connections 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb00a2c00 0x7f5cb00a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc81079c0 0x7f5cc8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cc8102360 0x7f5cc8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 --2- 172.21.15.59:0/3192507911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc81019c0 0x7f5cc80744f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:53.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 >> 172.21.15.59:0/3192507911 conn(0x7f5cc80fd740 msgr2=0x7f5cc80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:53.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 shutdown_connections 2024-09-05T11:53:53.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:53.625+0000 7f5ccee00640 1 -- 172.21.15.59:0/3192507911 wait complete. 2024-09-05T11:53:53.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:53.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: Deploying daemon mds.d on smithi059 2024-09-05T11:53:53.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:53.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.59:6834/1152129733,v1:172.21.15.59:6835/1152129733] up:boot 2024-09-05T11:53:53.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: fsmap 3 up:standby 2024-09-05T11:53:53.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T11:53:53.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:53 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3192507911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: Deploying daemon mds.d on smithi059 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.59:6834/1152129733,v1:172.21.15.59:6835/1152129733] up:boot 2024-09-05T11:53:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: fsmap 3 up:standby 2024-09-05T11:53:54.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T11:53:54.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:53 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3192507911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: Deploying daemon mds.d on smithi059 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.59:6834/1152129733,v1:172.21.15.59:6835/1152129733] up:boot 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: fsmap 3 up:standby 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T11:53:54.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:53 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3192507911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:56.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:55 smithi059 ceph-mon[30025]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:56.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:55 smithi089 ceph-mon[34499]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:55 smithi189 ceph-mon[34772]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:56.395 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 systemd[1]: Starting Ceph mds.d for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:53:56.704 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 podman[69738]: 2024-09-05 11:53:56.298311175 +0000 UTC m=+0.027501665 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:53:56.704 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 podman[69738]: 2024-09-05 11:53:56.409481068 +0000 UTC m=+0.138671535 container create f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-05T11:53:56.705 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 podman[69738]: 2024-09-05 11:53:56.595251211 +0000 UTC m=+0.324441678 container init f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-05T11:53:56.705 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 podman[69738]: 2024-09-05 11:53:56.60124512 +0000 UTC m=+0.330435588 container start f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-05T11:53:56.705 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 ceph-mds[69757]: -- 172.21.15.59:0/1165788366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565305f6d860 con 0x565305f4b000 2024-09-05T11:53:56.705 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d[69753]: starting mds.d at 2024-09-05T11:53:56.705 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 bash[69738]: f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 2024-09-05T11:53:56.950 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:53:56.957 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 11:53:56 smithi059 systemd[1]: Started Ceph mds.d for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:53:57.326 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.59:6836/1365446032,v1:172.21.15.59:6837/1365446032] up:boot 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: fsmap 4 up:standby 2024-09-05T11:53:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:57 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T11:53:57.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.830+0000 7fa61be00640 1 -- 172.21.15.59:0/574869953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 msgr2=0x7fa6141067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:57.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.830+0000 7fa61be00640 1 --2- 172.21.15.59:0/574869953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa6141067f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 -- 172.21.15.59:0/574869953 shutdown_connections 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 --2- 172.21.15.59:0/574869953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 0x7fa61410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 --2- 172.21.15.59:0/574869953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa6141067f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 --2- 172.21.15.59:0/574869953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 0x7fa6141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 -- 172.21.15.59:0/574869953 >> 172.21.15.59:0/574869953 conn(0x7fa6140fd8b0 msgr2=0x7fa6140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 -- 172.21.15.59:0/574869953 shutdown_connections 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.831+0000 7fa61be00640 1 -- 172.21.15.59:0/574869953 wait complete. 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 Processor -- start 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 -- start start 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 0x7fa61419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:57.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 0x7fa61419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa614111fd0 con 0x7fa614106f20 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa614111e50 con 0x7fa6141027a0 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.832+0000 7fa61be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112150 con 0x7fa614101df0 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36684/0 (socket says 172.21.15.59:36684) 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 -- 172.21.15.59:0/3843979938 learned_addr learned my addr 172.21.15.59:0/3843979938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa619e00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 0x7fa61419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa619400640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 0x7fa61419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 -- 172.21.15.59:0/3843979938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 msgr2=0x7fa61419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 0x7fa61419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 -- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 msgr2=0x7fa61419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:57.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 0x7fa61419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 -- 172.21.15.59:0/3843979938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61419f5b0 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.833+0000 7fa618a00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa600004770 tx=0x7fa600003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.834+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60003f070 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.834+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa600031ca0 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.834+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419f840 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.834+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600035030 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.834+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6141a7180 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.835+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6141021d0 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.836+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa600042070 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.838+0000 7fa612000640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 0x7fa5fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.838+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa6000ef210 con 0x7fa6141027a0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.839+0000 7fa619400640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 0x7fa5fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:53:57.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.839+0000 7fa619400640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 0x7fa5fc0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa608009bd0 tx=0x7fa608005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:53:57.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:57.842+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6000bbf20 con 0x7fa6141027a0 2024-09-05T11:53:58.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.063+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6141a7350 con 0x7fa6141027a0 2024-09-05T11:53:58.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.59:6836/1365446032,v1:172.21.15.59:6837/1365446032] up:boot 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: fsmap 4 up:standby 2024-09-05T11:53:58.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:57 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T11:53:58.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.064+0000 7fa612000640 1 -- 172.21.15.59:0/3843979938 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fa6000c0220 con 0x7fa6141027a0 2024-09-05T11:53:58.067 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:53:58.067 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"btime":"2024-09-05T11:53:56:894830+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T11:53:58.067 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 5 2024-09-05T11:53:58.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.066+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 msgr2=0x7fa5fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:58.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.066+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 0x7fa5fc0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa608009bd0 tx=0x7fa608005f90 comp rx=0 tx=0).stop 2024-09-05T11:53:58.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.066+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 msgr2=0x7fa61419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:53:58.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.066+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa600004770 tx=0x7fa600003c10 comp rx=0 tx=0).stop 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 shutdown_connections 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5fc0a2c00 0x7fa5fc0a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa614106f20 0x7fa61419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6141027a0 0x7fa61419ab40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 --2- 172.21.15.59:0/3843979938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa614101df0 0x7fa61419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 >> 172.21.15.59:0/3843979938 conn(0x7fa6140fd8b0 msgr2=0x7fa6140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 shutdown_connections 2024-09-05T11:53:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:53:58.067+0000 7fa61be00640 1 -- 172.21.15.59:0/3843979938 wait complete. 2024-09-05T11:53:58.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.59:6836/1365446032,v1:172.21.15.59:6837/1365446032] up:boot 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: fsmap 4 up:standby 2024-09-05T11:53:58.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:57 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T11:53:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:58 smithi089 ceph-mon[34499]: Deploying daemon mds.e on smithi089 2024-09-05T11:53:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:58 smithi089 ceph-mon[34499]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:58.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:53:58 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3843979938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:58.935 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:58 smithi089 systemd[1]: Starting Ceph mds.e for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:53:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:58 smithi189 ceph-mon[34772]: Deploying daemon mds.e on smithi089 2024-09-05T11:53:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:58 smithi189 ceph-mon[34772]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:53:58 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3843979938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:58 smithi059 ceph-mon[30025]: Deploying daemon mds.e on smithi089 2024-09-05T11:53:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:58 smithi059 ceph-mon[30025]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:53:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:53:58 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3843979938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:53:59.327 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 podman[63379]: 2024-09-05 11:53:58.958966834 +0000 UTC m=+0.015433642 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:53:59.327 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 podman[63379]: 2024-09-05 11:53:59.056998256 +0000 UTC m=+0.113465062 container create f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-05T11:53:59.596 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 podman[63379]: 2024-09-05 11:53:59.477308794 +0000 UTC m=+0.533775596 container init f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-05T11:53:59.597 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 podman[63379]: 2024-09-05 11:53:59.480148825 +0000 UTC m=+0.536615630 container start f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T11:53:59.597 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 ceph-mds[63398]: -- 172.21.15.89:0/3166285289 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9f694b860 con 0x55d9f6929000 2024-09-05T11:53:59.597 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e[63394]: starting mds.e at 2024-09-05T11:53:59.597 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 bash[63379]: f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 2024-09-05T11:53:59.934 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 11:53:59 smithi089 systemd[1]: Started Ceph mds.e for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:00.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:00.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: Deploying daemon mds.f on smithi189 2024-09-05T11:54:00.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6834/318370648,v1:172.21.15.89:6835/318370648] up:boot 2024-09-05T11:54:00.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: fsmap 5 up:standby 2024-09-05T11:54:00.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T11:54:00.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: Deploying daemon mds.f on smithi189 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6834/318370648,v1:172.21.15.89:6835/318370648] up:boot 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: fsmap 5 up:standby 2024-09-05T11:54:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:00.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: Deploying daemon mds.f on smithi189 2024-09-05T11:54:00.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6834/318370648,v1:172.21.15.89:6835/318370648] up:boot 2024-09-05T11:54:00.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: fsmap 5 up:standby 2024-09-05T11:54:00.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T11:54:01.180 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 -- 172.21.15.59:0/3497554335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 msgr2=0x7f99a010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 --2- 172.21.15.59:0/3497554335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a010bb60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f998c009900 tx=0x7f998c02a480 comp rx=0 tx=0).stop 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 -- 172.21.15.59:0/3497554335 shutdown_connections 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 --2- 172.21.15.59:0/3497554335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a010c180 0x7f99a010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 --2- 172.21.15.59:0/3497554335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a010bb60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 --2- 172.21.15.59:0/3497554335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a0102d50 0x7f99a0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 -- 172.21.15.59:0/3497554335 >> 172.21.15.59:0/3497554335 conn(0x7f99a00fc7d0 msgr2=0x7f99a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 -- 172.21.15.59:0/3497554335 shutdown_connections 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.735+0000 7f99a5800640 1 -- 172.21.15.59:0/3497554335 wait complete. 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.736+0000 7f99a5800640 1 Processor -- start 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.736+0000 7f99a5800640 1 -- start start 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a0102d50 0x7f99a01009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 0x7f99a0101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0111fd0 con 0x7f99a0103670 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0111e50 con 0x7f99a0102d50 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f99a5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0112150 con 0x7f99a010c180 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f999e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f999e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39992/0 (socket says 172.21.15.59:39992) 2024-09-05T11:54:01.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f999e000640 1 -- 172.21.15.59:0/3990519284 learned_addr learned my addr 172.21.15.59:0/3990519284 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f999ea00640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a0102d50 0x7f99a01009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.737+0000 7f999f400640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 0x7f99a0101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 -- 172.21.15.59:0/3990519284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 msgr2=0x7f99a0101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 0x7f99a0101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 -- 172.21.15.59:0/3990519284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a0102d50 msgr2=0x7f99a01009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a0102d50 0x7f99a01009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 -- 172.21.15.59:0/3990519284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a01a6970 con 0x7f99a0103670 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999f400640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 0x7f99a0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f999e000640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f998c004770 tx=0x7f998c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c03f070 con 0x7f99a0103670 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f998c031ca0 con 0x7f99a0103670 2024-09-05T11:54:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c035030 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.738+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a01a6c00 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.739+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a01a70e0 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.740+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a0103130 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.741+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f998c042070 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.743+0000 7f9997400640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 0x7f99840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.743+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f998c0ef200 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.743+0000 7f999ea00640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 0x7f99840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.744+0000 7f999ea00640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 0x7f99840a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9990009bd0 tx=0x7f9990005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.747+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f998c0bbfd0 con 0x7f99a0103670 2024-09-05T11:54:01.761 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:01 smithi189 systemd[1]: Starting Ceph mds.f for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:54:01.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.969+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a01071f0 con 0x7f99a0103670 2024-09-05T11:54:01.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.969+0000 7f9997400640 1 -- 172.21.15.59:0/3990519284 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f998c0c02d0 con 0x7f99a0103670 2024-09-05T11:54:01.971 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:01.971 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":6,"btime":"2024-09-05T11:53:59:780870+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T11:54:01.972 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 6 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 msgr2=0x7f99840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 0x7f99840a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9990009bd0 tx=0x7f9990005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 msgr2=0x7f99a0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f998c004770 tx=0x7f998c003c10 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 shutdown_connections 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f99840a2b80 0x7f99840a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a010c180 0x7f99a0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99a0103670 0x7f99a0100ef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 --2- 172.21.15.59:0/3990519284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99a0102d50 0x7f99a01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 >> 172.21.15.59:0/3990519284 conn(0x7f99a00fc7d0 msgr2=0x7f99a00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:01.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 shutdown_connections 2024-09-05T11:54:01.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:01.972+0000 7f99a5800640 1 -- 172.21.15.59:0/3990519284 wait complete. 2024-09-05T11:54:02.037 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:01 smithi189 podman[64216]: 2024-09-05 11:54:01.660267787 +0000 UTC m=+0.016220782 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:54:02.037 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:01 smithi189 podman[64216]: 2024-09-05 11:54:01.791336283 +0000 UTC m=+0.147289258 container create 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-05T11:54:02.037 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 podman[64216]: 2024-09-05 11:54:02.036184245 +0000 UTC m=+0.392137227 container init 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T11:54:02.338 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 podman[64216]: 2024-09-05 11:54:02.039377463 +0000 UTC m=+0.395330447 container start 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-05T11:54:02.339 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mds[64235]: -- 172.21.15.189:0/3357544245 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561426cf7860 con 0x561426cd5400 2024-09-05T11:54:02.339 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f[64230]: starting mds.f at 2024-09-05T11:54:02.339 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 bash[64216]: 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a 2024-09-05T11:54:02.339 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 11:54:02 smithi189 systemd[1]: Started Ceph mds.f for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3990519284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.706 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:02.707 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:02.707 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:02 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:02.880 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:02.880 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3990519284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:02.880 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:02.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:02.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:02 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3990519284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:02 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:03 smithi089 ceph-mon[34499]: Deploying daemon mds.c on smithi189 2024-09-05T11:54:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:03 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.189:6834/4161006032,v1:172.21.15.189:6835/4161006032] up:boot 2024-09-05T11:54:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:03 smithi089 ceph-mon[34499]: fsmap 6 up:standby 2024-09-05T11:54:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:03 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T11:54:03.943 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:03 smithi189 ceph-mon[34772]: Deploying daemon mds.c on smithi189 2024-09-05T11:54:03.944 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:03 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.189:6834/4161006032,v1:172.21.15.189:6835/4161006032] up:boot 2024-09-05T11:54:03.944 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:03 smithi189 ceph-mon[34772]: fsmap 6 up:standby 2024-09-05T11:54:03.944 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:03 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T11:54:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:03 smithi059 ceph-mon[30025]: Deploying daemon mds.c on smithi189 2024-09-05T11:54:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:03 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.189:6834/4161006032,v1:172.21.15.189:6835/4161006032] up:boot 2024-09-05T11:54:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:03 smithi059 ceph-mon[30025]: fsmap 6 up:standby 2024-09-05T11:54:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:03 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T11:54:04.339 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 systemd[1]: Starting Ceph mds.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:54:04.609 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 podman[64519]: 2024-09-05 11:54:04.245286036 +0000 UTC m=+0.016140222 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:54:04.609 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 podman[64519]: 2024-09-05 11:54:04.364075165 +0000 UTC m=+0.134929347 container create b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T11:54:04.609 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 podman[64519]: 2024-09-05 11:54:04.608463192 +0000 UTC m=+0.379317375 container init b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T11:54:04.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:04 smithi089 ceph-mon[34499]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:04 smithi059 ceph-mon[30025]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:04 smithi189 ceph-mon[34772]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:05.089 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 podman[64519]: 2024-09-05 11:54:04.611234275 +0000 UTC m=+0.382088462 container start b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T11:54:05.089 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 ceph-mds[64538]: -- 172.21.15.189:0/75590296 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a973441860 con 0x55a97341e800 2024-09-05T11:54:05.089 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c[64534]: starting mds.c at 2024-09-05T11:54:05.089 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 bash[64519]: b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 2024-09-05T11:54:05.089 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 11:54:04 smithi189 systemd[1]: Started Ceph mds.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:05.098 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:05.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.620+0000 7fe754800640 1 -- 172.21.15.59:0/2833346705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c101de0 msgr2=0x7fe74c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:05.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.620+0000 7fe754800640 1 --2- 172.21.15.59:0/2833346705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c101de0 0x7fe74c1021c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe740009900 tx=0x7fe74002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 -- 172.21.15.59:0/2833346705 shutdown_connections 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 --2- 172.21.15.59:0/2833346705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 --2- 172.21.15.59:0/2833346705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c102790 0x7fe74c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 --2- 172.21.15.59:0/2833346705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c101de0 0x7fe74c1021c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 -- 172.21.15.59:0/2833346705 >> 172.21.15.59:0/2833346705 conn(0x7fe74c0fd8c0 msgr2=0x7fe74c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 -- 172.21.15.59:0/2833346705 shutdown_connections 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.621+0000 7fe754800640 1 -- 172.21.15.59:0/2833346705 wait complete. 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 Processor -- start 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 -- start start 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c101de0 0x7fe74c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:05.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c111fc0 con 0x7fe74c101de0 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c111e40 con 0x7fe74c102790 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.622+0000 7fe754800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c112140 con 0x7fe74c106f10 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c101de0 0x7fe74c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59974/0 (socket says 172.21.15.59:59974) 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 -- 172.21.15.59:0/1310184542 learned_addr learned my addr 172.21.15.59:0/1310184542 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe752800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 -- 172.21.15.59:0/1310184542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 msgr2=0x7fe74c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 -- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c101de0 msgr2=0x7fe74c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c101de0 0x7fe74c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe751400640 1 -- 172.21.15.59:0/1310184542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c1a68e0 con 0x7fe74c102790 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.623+0000 7fe752800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe751400640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9f0 tx=0x7fe73c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00cd80 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73c010910 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c018990 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c1a6bd0 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.624+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1a7160 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.626+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c1021c0 con 0x7fe74c102790 2024-09-05T11:54:05.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.627+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe73c010ac0 con 0x7fe74c102790 2024-09-05T11:54:05.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.628+0000 7fe746a00640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 0x7fe7340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:05.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.628+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe73c0ceeb0 con 0x7fe74c102790 2024-09-05T11:54:05.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.628+0000 7fe751e00640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 0x7fe7340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:05.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.629+0000 7fe751e00640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 0x7fe7340a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe740000c00 tx=0x7fe740005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:05.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.632+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe73c09bbc0 con 0x7fe74c102790 2024-09-05T11:54:05.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.859+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe74c10a0f0 con 0x7fe74c102790 2024-09-05T11:54:05.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.860+0000 7fe746a00640 1 -- 172.21.15.59:0/1310184542 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fe73c09fec0 con 0x7fe74c102790 2024-09-05T11:54:05.861 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:05.862 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"btime":"2024-09-05T11:54:04:662402+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T11:54:05.862 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 msgr2=0x7fe7340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 0x7fe7340a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe740000c00 tx=0x7fe740005b50 comp rx=0 tx=0).stop 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 msgr2=0x7fe74c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9f0 tx=0x7fe73c00eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 shutdown_connections 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7340a2c00 0x7fe7340a50c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe74c106f10 0x7fe74c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe74c102790 0x7fe74c074a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 --2- 172.21.15.59:0/1310184542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe74c101de0 0x7fe74c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:05.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 >> 172.21.15.59:0/1310184542 conn(0x7fe74c0fd8c0 msgr2=0x7fe74c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:05.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 shutdown_connections 2024-09-05T11:54:05.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:05.862+0000 7fe754800640 1 -- 172.21.15.59:0/1310184542 wait complete. 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.189:6836/827690495,v1:172.21.15.189:6837/827690495] up:boot 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: fsmap 7 up:standby 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:05.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:05.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:05 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.189:6836/827690495,v1:172.21.15.189:6837/827690495] up:boot 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: fsmap 7 up:standby 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:05.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:05.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:05.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:05 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.189:6836/827690495,v1:172.21.15.189:6837/827690495] up:boot 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: fsmap 7 up:standby 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:06.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:05 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:06.880 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:06 smithi089 ceph-mon[34499]: Deploying daemon mds.h on smithi089 2024-09-05T11:54:06.881 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:06 smithi089 ceph-mon[34499]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:06.881 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:06 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1310184542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:06 smithi059 ceph-mon[30025]: Deploying daemon mds.h on smithi089 2024-09-05T11:54:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:06 smithi059 ceph-mon[30025]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:06 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1310184542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:07.070 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:06 smithi189 ceph-mon[34772]: Deploying daemon mds.h on smithi089 2024-09-05T11:54:07.070 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:06 smithi189 ceph-mon[34772]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:07.070 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:06 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1310184542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:07.184 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 podman[63698]: 2024-09-05 11:54:07.114097028 +0000 UTC m=+0.138748417 container create 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, ceph=True) 2024-09-05T11:54:07.684 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 podman[63698]: 2024-09-05 11:54:07.384061213 +0000 UTC m=+0.408712614 container init 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T11:54:07.684 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 podman[63698]: 2024-09-05 11:54:07.386755342 +0000 UTC m=+0.411406735 container start 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T11:54:07.684 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 ceph-mds[63718]: -- 172.21.15.89:0/54453527 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b433ba1860 con 0x55b433b7f000 2024-09-05T11:54:07.685 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: starting mds.h at 2024-09-05T11:54:07.685 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 bash[63698]: 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 2024-09-05T11:54:07.685 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 11:54:07 smithi089 systemd[1]: Started Ceph mds.h for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:08.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: Deploying daemon mds.i on smithi189 2024-09-05T11:54:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6836/2748476304,v1:172.21.15.89:6837/2748476304] up:boot 2024-09-05T11:54:08.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: fsmap 8 up:standby 2024-09-05T11:54:08.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:08 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T11:54:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: Deploying daemon mds.i on smithi189 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6836/2748476304,v1:172.21.15.89:6837/2748476304] up:boot 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: fsmap 8 up:standby 2024-09-05T11:54:08.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:08 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: Deploying daemon mds.i on smithi189 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6836/2748476304,v1:172.21.15.89:6837/2748476304] up:boot 2024-09-05T11:54:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: fsmap 8 up:standby 2024-09-05T11:54:08.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:08 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T11:54:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:09.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3744011463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 msgr2=0x7fd3f4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:09.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3744011463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f4110ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009ca0 tx=0x7fd3ec02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3744011463 shutdown_connections 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3744011463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f4110ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3744011463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3744011463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 0x7fd3f41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3744011463 >> 172.21.15.59:0/3744011463 conn(0x7fd3f40ffa90 msgr2=0x7fd3f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3744011463 shutdown_connections 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.603+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3744011463 wait complete. 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 Processor -- start 2024-09-05T11:54:09.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 -- start start 2024-09-05T11:54:09.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 0x7fd3f419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114180 con 0x7fd3f41049a0 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114000 con 0x7fd3f4103ff0 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.604+0000 7fd3fbc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114300 con 0x7fd3f4109120 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40016/0 (socket says 172.21.15.59:40016) 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 -- 172.21.15.59:0/3152361886 learned_addr learned my addr 172.21.15.59:0/3152361886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f9200640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 0x7fd3f419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f9c00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 -- 172.21.15.59:0/3152361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 msgr2=0x7fd3f419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:09.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 -- 172.21.15.59:0/3152361886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 msgr2=0x7fd3f419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 0x7fd3f419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 -- 172.21.15.59:0/3152361886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f419f570 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.605+0000 7fd3f3e00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e9f0 tx=0x7fd3e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.606+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000cd80 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.606+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f419f860 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.606+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e0010910 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.606+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00188f0 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.606+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a7140 con 0x7fd3f41049a0 2024-09-05T11:54:09.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.607+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f41043d0 con 0x7fd3f41049a0 2024-09-05T11:54:09.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.610+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3e0010ac0 con 0x7fd3f41049a0 2024-09-05T11:54:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.611+0000 7fd3f1600640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 0x7fd3dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.611+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd3e00ceec0 con 0x7fd3f41049a0 2024-09-05T11:54:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.611+0000 7fd3f9200640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 0x7fd3dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.612+0000 7fd3f9200640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 0x7fd3dc0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005d00 tx=0x7fd3e800b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.616+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3e009bbd0 con 0x7fd3f41049a0 2024-09-05T11:54:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.836+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3f41a7470 con 0x7fd3f41049a0 2024-09-05T11:54:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.837+0000 7fd3f1600640 1 -- 172.21.15.59:0/3152361886 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7fd3e009fed0 con 0x7fd3f41049a0 2024-09-05T11:54:09.839 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:09.839 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":9,"btime":"2024-09-05T11:54:07:673082+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T11:54:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 9 2024-09-05T11:54:09.840 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:09 smithi189 systemd[1]: Starting Ceph mds.i for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 msgr2=0x7fd3dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 0x7fd3dc0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005d00 tx=0x7fd3e800b410 comp rx=0 tx=0).stop 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 msgr2=0x7fd3f419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e9f0 tx=0x7fd3e000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 shutdown_connections 2024-09-05T11:54:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3dc0a2c00 0x7fd3dc0a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3f4109120 0x7fd3f419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f41049a0 0x7fd3f419ab00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 --2- 172.21.15.59:0/3152361886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3f4103ff0 0x7fd3f419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.839+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 >> 172.21.15.59:0/3152361886 conn(0x7fd3f40ffa90 msgr2=0x7fd3f41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.840+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 shutdown_connections 2024-09-05T11:54:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:09.840+0000 7fd3fbc00640 1 -- 172.21.15.59:0/3152361886 wait complete. 2024-09-05T11:54:10.133 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:09 smithi189 podman[64821]: 2024-09-05 11:54:09.745649595 +0000 UTC m=+0.015962693 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:54:10.133 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:09 smithi189 podman[64821]: 2024-09-05 11:54:09.904392974 +0000 UTC m=+0.174706061 container create 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-05T11:54:10.133 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 podman[64821]: 2024-09-05 11:54:10.132134822 +0000 UTC m=+0.402447910 container init 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-05T11:54:10.542 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 podman[64821]: 2024-09-05 11:54:10.135236317 +0000 UTC m=+0.405549405 container start 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T11:54:10.542 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mds[64840]: -- 172.21.15.189:0/2607458861 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56022b6c1860 con 0x56022b69e800 2024-09-05T11:54:10.542 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i[64836]: starting mds.i at 2024-09-05T11:54:10.542 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 bash[64821]: 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 2024-09-05T11:54:10.542 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 11:54:10 smithi189 systemd[1]: Started Ceph mds.i for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:10.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3152361886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:10.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:10 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3152361886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:10.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:10.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:10 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3152361886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:10.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:10 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:11 smithi059 ceph-mon[30025]: Deploying daemon mds.g on smithi059 2024-09-05T11:54:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:11 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.189:6838/3241376158,v1:172.21.15.189:6839/3241376158] up:boot 2024-09-05T11:54:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:11 smithi059 ceph-mon[30025]: fsmap 9 up:standby 2024-09-05T11:54:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:11 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T11:54:11.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:11 smithi189 ceph-mon[34772]: Deploying daemon mds.g on smithi059 2024-09-05T11:54:11.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:11 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.189:6838/3241376158,v1:172.21.15.189:6839/3241376158] up:boot 2024-09-05T11:54:11.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:11 smithi189 ceph-mon[34772]: fsmap 9 up:standby 2024-09-05T11:54:11.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:11 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T11:54:11.936 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:11 smithi089 ceph-mon[34499]: Deploying daemon mds.g on smithi059 2024-09-05T11:54:11.937 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:11 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.189:6838/3241376158,v1:172.21.15.189:6839/3241376158] up:boot 2024-09-05T11:54:11.937 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:11 smithi089 ceph-mon[34499]: fsmap 9 up:standby 2024-09-05T11:54:11.937 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:11 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T11:54:12.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:12.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:12 smithi059 ceph-mon[30025]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:12.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:12 smithi189 ceph-mon[34772]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:12 smithi089 ceph-mon[34499]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:13.178 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:12 smithi059 systemd[1]: Starting Ceph mds.g for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T11:54:13.450 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 podman[70523]: 2024-09-05 11:54:13.080869376 +0000 UTC m=+0.030841256 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T11:54:13.450 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 podman[70523]: 2024-09-05 11:54:13.22137145 +0000 UTC m=+0.171343319 container create 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-05T11:54:13.450 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 podman[70523]: 2024-09-05 11:54:13.449166034 +0000 UTC m=+0.399137919 container init 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-05T11:54:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:13.708 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 podman[70523]: 2024-09-05 11:54:13.455760579 +0000 UTC m=+0.405732448 container start 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T11:54:13.708 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 ceph-mds[70568]: -- 172.21.15.59:0/2415279602 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2f41bf860 con 0x55b2f419c800 2024-09-05T11:54:13.708 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g[70553]: starting mds.g at 2024-09-05T11:54:13.708 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 bash[70523]: 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 2024-09-05T11:54:13.708 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 11:54:13 smithi059 systemd[1]: Started Ceph mds.g for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 -- 172.21.15.59:0/1741002082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 msgr2=0x7f654810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 --2- 172.21.15.59:0/1741002082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654810b9a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6534009900 tx=0x7f653402a480 comp rx=0 tx=0).stop 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 -- 172.21.15.59:0/1741002082 shutdown_connections 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 --2- 172.21.15.59:0/1741002082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 --2- 172.21.15.59:0/1741002082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654810b9a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 --2- 172.21.15.59:0/1741002082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f6548102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 -- 172.21.15.59:0/1741002082 >> 172.21.15.59:0/1741002082 conn(0x7f65480fc610 msgr2=0x7f65480fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 -- 172.21.15.59:0/1741002082 shutdown_connections 2024-09-05T11:54:14.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.173+0000 7f654e400640 1 -- 172.21.15.59:0/1741002082 wait complete. 2024-09-05T11:54:14.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.174+0000 7f654e400640 1 Processor -- start 2024-09-05T11:54:14.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 -- start start 2024-09-05T11:54:14.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f654819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:14.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:14.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:14.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6548111e10 con 0x7f65481034b0 2024-09-05T11:54:14.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6548111c90 con 0x7f654810bfc0 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6548111f90 con 0x7f6548102b90 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f6547e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f6547e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38670/0 (socket says 172.21.15.59:38670) 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f6547e00640 1 -- 172.21.15.59:0/3913995974 learned_addr learned my addr 172.21.15.59:0/3913995974 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.175+0000 7f654d000640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f654819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654da00640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 -- 172.21.15.59:0/3913995974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 msgr2=0x7f654819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 -- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 msgr2=0x7f654819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 -- 172.21.15.59:0/3913995974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654819f480 con 0x7f6548102b90 2024-09-05T11:54:14.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f6547e00640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654da00640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.176+0000 7f654d000640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f654819a4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f653c005fd0 tx=0x7f653c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.177+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c009c30 con 0x7f6548102b90 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.177+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f653c009dd0 con 0x7f6548102b90 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.177+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654819f770 con 0x7f6548102b90 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.177+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c010460 con 0x7f6548102b90 2024-09-05T11:54:14.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.177+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a7050 con 0x7f6548102b90 2024-09-05T11:54:14.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.178+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6548102f70 con 0x7f6548102b90 2024-09-05T11:54:14.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.180+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f653c003ef0 con 0x7f6548102b90 2024-09-05T11:54:14.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.181+0000 7f6545600640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:14.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.182+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f653c0ce700 con 0x7f6548102b90 2024-09-05T11:54:14.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.182+0000 7f6547e00640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:14.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.182+0000 7f6547e00640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 0x7f65300a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65340096c0 tx=0x7f65340046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:14.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.184+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f653c017020 con 0x7f6548102b90 2024-09-05T11:54:14.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.419+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65481a72f0 con 0x7f6548102b90 2024-09-05T11:54:14.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.420+0000 7f6545600640 1 -- 172.21.15.59:0/3913995974 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f653c09b410 con 0x7f6548102b90 2024-09-05T11:54:14.422 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:14.422 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":11,"btime":"2024-09-05T11:54:13:548380+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T11:54:14.422 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 11 2024-09-05T11:54:14.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.422+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 msgr2=0x7f65300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:14.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.422+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 0x7f65300a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65340096c0 tx=0x7f65340046c0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.422+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 msgr2=0x7f654819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:14.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.422+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f654819a4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f653c005fd0 tx=0x7f653c00dde0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 shutdown_connections 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65300a2b80 0x7f65300a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f654810bfc0 0x7f654819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481034b0 0x7f654819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 --2- 172.21.15.59:0/3913995974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548102b90 0x7f654819a4d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 >> 172.21.15.59:0/3913995974 conn(0x7f65480fc610 msgr2=0x7f65480fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 shutdown_connections 2024-09-05T11:54:14.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:14.423+0000 7f654e400640 1 -- 172.21.15.59:0/3913995974 wait complete. 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.59:6838/3325685690,v1:172.21.15.59:6839/3325685690] up:boot 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: fsmap 10 up:standby 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: Deploying daemon mds.a on smithi059 2024-09-05T11:54:14.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:14 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3913995974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.59:6838/3325685690,v1:172.21.15.59:6839/3325685690] up:boot 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: fsmap 10 up:standby 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:14.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: Deploying daemon mds.a on smithi059 2024-09-05T11:54:14.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:14 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3913995974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.59:6838/3325685690,v1:172.21.15.59:6839/3325685690] up:boot 2024-09-05T11:54:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: fsmap 10 up:standby 2024-09-05T11:54:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: Deploying daemon mds.a on smithi059 2024-09-05T11:54:14.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:14 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3913995974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:16.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:16 smithi059 ceph-mon[30025]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:16.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:16 smithi189 ceph-mon[34772]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:16.920 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:16 smithi059 podman[70903]: 2024-09-05 11:54:16.912845428 +0000 UTC m=+0.373727411 container init 7acf1b0de9afdf6a9314c376d446c475b41aae7f243ae6d31f6b34ad677c0783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-05T11:54:16.920 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:16 smithi059 podman[70903]: 2024-09-05 11:54:16.919091439 +0000 UTC m=+0.379973412 container start 7acf1b0de9afdf6a9314c376d446c475b41aae7f243ae6d31f6b34ad677c0783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T11:54:16.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:16 smithi089 ceph-mon[34499]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:17.208 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:16 smithi059 ceph-mds[70922]: -- 172.21.15.59:0/3014061927 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633e004bc20 con 0x5633e0028800 2024-09-05T11:54:17.208 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a[70918]: starting mds.a at 2024-09-05T11:54:17.208 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:17 smithi059 bash[70903]: 7acf1b0de9afdf6a9314c376d446c475b41aae7f243ae6d31f6b34ad677c0783 2024-09-05T11:54:17.208 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 11:54:17 smithi059 systemd[1]: Started Ceph mds.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:18.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: Deploying daemon mds.b on smithi089 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.59:6840/2374945780,v1:172.21.15.59:6841/2374945780] up:boot 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: fsmap 11 up:standby 2024-09-05T11:54:18.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T11:54:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: Deploying daemon mds.b on smithi089 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.59:6840/2374945780,v1:172.21.15.59:6841/2374945780] up:boot 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: fsmap 11 up:standby 2024-09-05T11:54:18.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T11:54:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: Deploying daemon mds.b on smithi089 2024-09-05T11:54:18.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.59:6840/2374945780,v1:172.21.15.59:6841/2374945780] up:boot 2024-09-05T11:54:18.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: fsmap 11 up:standby 2024-09-05T11:54:18.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T11:54:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.090+0000 7f7094000640 1 -- 172.21.15.59:0/743228340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c1049c0 msgr2=0x7f708c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.090+0000 7f7094000640 1 --2- 172.21.15.59:0/743228340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c1049c0 0x7f708c108a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7078009900 tx=0x7f707802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 -- 172.21.15.59:0/743228340 shutdown_connections 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 --2- 172.21.15.59:0/743228340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c109140 0x7f708c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 --2- 172.21.15.59:0/743228340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c1049c0 0x7f708c108a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 --2- 172.21.15.59:0/743228340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 0x7f708c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 -- 172.21.15.59:0/743228340 >> 172.21.15.59:0/743228340 conn(0x7f708c0ffa50 msgr2=0x7f708c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 -- 172.21.15.59:0/743228340 shutdown_connections 2024-09-05T11:54:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.091+0000 7f7094000640 1 -- 172.21.15.59:0/743228340 wait complete. 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 Processor -- start 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 -- start start 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 0x7f708c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c109140 0x7f708c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f708c114190 con 0x7f708c109140 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f708c114010 con 0x7f708c1049c0 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7094000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f708c114310 con 0x7f708c104010 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7090c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7092000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c109140 0x7f708c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7090c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43816/0 (socket says 172.21.15.59:43816) 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.092+0000 7f7090c00640 1 -- 172.21.15.59:0/1190939425 learned_addr learned my addr 172.21.15.59:0/1190939425 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7091600640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 0x7f708c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 -- 172.21.15.59:0/1190939425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 msgr2=0x7f708c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 0x7f708c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 -- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c109140 msgr2=0x7f708c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c109140 0x7f708c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 -- 172.21.15.59:0/1190939425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c19f600 con 0x7f708c1049c0 2024-09-05T11:54:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.093+0000 7f7090c00640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7078004770 tx=0x7f7078003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.094+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707803f070 con 0x7f708c1049c0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.094+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7078031ca0 con 0x7f708c1049c0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.094+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c19f890 con 0x7f708c1049c0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.094+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078035030 con 0x7f708c1049c0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.094+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1a71d0 con 0x7f708c1049c0 2024-09-05T11:54:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.095+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c1043f0 con 0x7f708c1049c0 2024-09-05T11:54:19.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.097+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7078042070 con 0x7f708c1049c0 2024-09-05T11:54:19.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.098+0000 7f708a000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 0x7f70740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:19.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.098+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f70780ef240 con 0x7f708c1049c0 2024-09-05T11:54:19.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.099+0000 7f7091600640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 0x7f70740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:19.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.099+0000 7f7091600640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 0x7f70740a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7080005d00 tx=0x7f708000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:19.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.103+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70780bbf50 con 0x7f708c1049c0 2024-09-05T11:54:19.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.324+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f708c10bdf0 con 0x7f708c1049c0 2024-09-05T11:54:19.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.324+0000 7f708a000640 1 -- 172.21.15.59:0/1190939425 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f70780c0250 con 0x7f708c1049c0 2024-09-05T11:54:19.326 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:19.326 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":12,"btime":"2024-09-05T11:54:17:559258+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T11:54:19.327 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 12 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 msgr2=0x7f70740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 0x7f70740a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7080005d00 tx=0x7f708000b410 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 msgr2=0x7f708c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7078004770 tx=0x7f7078003c10 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 shutdown_connections 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70740a2c00 0x7f70740a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c109140 0x7f708c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f708c1049c0 0x7f708c19ab90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 --2- 172.21.15.59:0/1190939425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f708c104010 0x7f708c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 >> 172.21.15.59:0/1190939425 conn(0x7f708c0ffa50 msgr2=0x7f708c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:19.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 shutdown_connections 2024-09-05T11:54:19.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:19.328+0000 7f7094000640 1 -- 172.21.15.59:0/1190939425 wait complete. 2024-09-05T11:54:19.934 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 11:54:19 smithi089 systemd[1]: Started Ceph mds.b for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1190939425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:20 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1190939425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:20 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1190939425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:20 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:21 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:boot 2024-09-05T11:54:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:21 smithi089 ceph-mon[34499]: fsmap 12 up:standby 2024-09-05T11:54:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:21 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T11:54:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:21 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:21 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:boot 2024-09-05T11:54:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:21 smithi059 ceph-mon[30025]: fsmap 12 up:standby 2024-09-05T11:54:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T11:54:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:21 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:21 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:boot 2024-09-05T11:54:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:21 smithi189 ceph-mon[34772]: fsmap 12 up:standby 2024-09-05T11:54:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:21 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T11:54:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:21 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:21.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:22.357 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:22.820 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:22 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:22 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 -- 172.21.15.59:0/3878382876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 msgr2=0x7f5568108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 --2- 172.21.15.59:0/3878382876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f5568108830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5554009900 tx=0x7f555402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 -- 172.21.15.59:0/3878382876 shutdown_connections 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 --2- 172.21.15.59:0/3878382876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568108f60 0x7f5568110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 --2- 172.21.15.59:0/3878382876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f5568108830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 --2- 172.21.15.59:0/3878382876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568103e30 0x7f5568104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 -- 172.21.15.59:0/3878382876 >> 172.21.15.59:0/3878382876 conn(0x7f55680ff8d0 msgr2=0x7f5568101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 -- 172.21.15.59:0/3878382876 shutdown_connections 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.937+0000 7f5567e00640 1 -- 172.21.15.59:0/3878382876 wait complete. 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 Processor -- start 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 -- start start 2024-09-05T11:54:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f556819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568108f60 0x7f556819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5568114010 con 0x7f5568103e30 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5568113e90 con 0x7f55681047e0 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.938+0000 7f5567e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5568114190 con 0x7f5568108f60 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f556819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54238/0 (socket says 172.21.15.59:54238) 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f556819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43916/0 (socket says 172.21.15.59:43916) 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 -- 172.21.15.59:0/701029612 learned_addr learned my addr 172.21.15.59:0/701029612 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5567400640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568108f60 0x7f556819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 -- 172.21.15.59:0/701029612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568108f60 msgr2=0x7f556819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568108f60 0x7f556819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 -- 172.21.15.59:0/701029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 msgr2=0x7f556819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f556819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 -- 172.21.15.59:0/701029612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819f400 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.939+0000 7f5566a00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f555800d950 tx=0x7f555800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.940+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558017030 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.940+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558004020 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.940+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819f6f0 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.940+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580047a0 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.940+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a6fd0 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.941+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568104210 con 0x7f5568103e30 2024-09-05T11:54:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.944+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55580041d0 con 0x7f5568103e30 2024-09-05T11:54:22.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.945+0000 7f555f400640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 0x7f554c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:22.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.946+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f55580ce500 con 0x7f5568103e30 2024-09-05T11:54:22.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.946+0000 7f5566000640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 0x7f554c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:22.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.947+0000 7f5566000640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 0x7f554c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5554000c00 tx=0x7f5554005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:22.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:22.950+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f555809b860 con 0x7f5568103e30 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:22 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.203+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f556802d7c0 con 0x7f5568103e30 2024-09-05T11:54:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.205+0000 7f555f400640 1 -- 172.21.15.59:0/701029612 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f555809b160 con 0x7f5568103e30 2024-09-05T11:54:23.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":13,"btime":"2024-09-05T11:54:20:096748+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-05T11:54:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 13 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 msgr2=0x7f554c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 0x7f554c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5554000c00 tx=0x7f5554005b50 comp rx=0 tx=0).stop 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 msgr2=0x7f556819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f555800d950 tx=0x7f555800de10 comp rx=0 tx=0).stop 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 shutdown_connections 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f554c0a2c00 0x7f554c0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568108f60 0x7f556819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55681047e0 0x7f556819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 --2- 172.21.15.59:0/701029612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5568103e30 0x7f556819a450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.207+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 >> 172.21.15.59:0/701029612 conn(0x7f55680ff8d0 msgr2=0x7f5568100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.208+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 shutdown_connections 2024-09-05T11:54:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:23.208+0000 7f5567e00640 1 -- 172.21.15.59:0/701029612 wait complete. 2024-09-05T11:54:23.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:23.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:23.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:23 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:23 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/701029612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:23.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:23.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:23 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:23 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/701029612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:23.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:23.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:23.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:23 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:23 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/701029612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:23.970 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-05T11:54:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:24 smithi059 ceph-mon[30025]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:24 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.928 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:24 smithi089 ceph-mon[34499]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:24 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:24 smithi189 ceph-mon[34772]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:24 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 -- 172.21.15.59:0/2866187457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc109120 msgr2=0x7f2fdc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/2866187457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc109120 0x7f2fdc110ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009ca0 tx=0x7f2fd402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 -- 172.21.15.59:0/2866187457 shutdown_connections 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/2866187457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc109120 0x7f2fdc110ca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/2866187457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/2866187457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 0x7f2fdc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 -- 172.21.15.59:0/2866187457 >> 172.21.15.59:0/2866187457 conn(0x7f2fdc0ffa90 msgr2=0x7f2fdc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.477+0000 7f2fe0a00640 1 -- 172.21.15.59:0/2866187457 shutdown_connections 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.478+0000 7f2fe0a00640 1 -- 172.21.15.59:0/2866187457 wait complete. 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.478+0000 7f2fe0a00640 1 Processor -- start 2024-09-05T11:54:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.478+0000 7f2fe0a00640 1 -- start start 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 0x7f2fdc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc109120 0x7f2fdc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc1141a0 con 0x7f2fdc1049a0 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc114020 con 0x7f2fdc103ff0 2024-09-05T11:54:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fe0a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc114320 con 0x7f2fdc109120 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54256/0 (socket says 172.21.15.59:54256) 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fd3e00640 1 -- 172.21.15.59:0/1922914109 learned_addr learned my addr 172.21.15.59:0/1922914109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fd9e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 0x7f2fdc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.479+0000 7f2fda800640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc109120 0x7f2fdc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 -- 172.21.15.59:0/1922914109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc109120 msgr2=0x7f2fdc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc109120 0x7f2fdc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 -- 172.21.15.59:0/1922914109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 msgr2=0x7f2fdc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 0x7f2fdc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 -- 172.21.15.59:0/1922914109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc19f590 con 0x7f2fdc1049a0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd3e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e9f0 tx=0x7f2fc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400cd80 con 0x7f2fdc1049a0 2024-09-05T11:54:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc4005500 con 0x7f2fdc1049a0 2024-09-05T11:54:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4010690 con 0x7f2fdc1049a0 2024-09-05T11:54:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc19f880 con 0x7f2fdc1049a0 2024-09-05T11:54:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.480+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1a7160 con 0x7f2fdc1049a0 2024-09-05T11:54:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.482+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc1043d0 con 0x7f2fdc1049a0 2024-09-05T11:54:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.483+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2fc40046c0 con 0x7f2fdc1049a0 2024-09-05T11:54:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.485+0000 7f2fd2a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 0x7f2fc00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.485+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2fc40cef90 con 0x7f2fdc1049a0 2024-09-05T11:54:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.485+0000 7f2fd9e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 0x7f2fc00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.486+0000 7f2fd9e00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 0x7f2fc00a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0095f0 tx=0x7f2fcc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.490+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fc409c2f0 con 0x7f2fdc1049a0 2024-09-05T11:54:25.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.691+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2fdc1a7390 con 0x7f2fdc1049a0 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.691+0000 7f2fd2a00640 1 -- 172.21.15.59:0/1922914109 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f2fc409bbf0 con 0x7f2fdc1049a0 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:54:25.693 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-09-05T11:54:25.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.694+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 msgr2=0x7f2fc00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:25.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.694+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 0x7f2fc00a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0095f0 tx=0x7f2fcc005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:25.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.694+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 msgr2=0x7f2fdc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:25.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.694+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e9f0 tx=0x7f2fc400edf0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 shutdown_connections 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2fc00a2c00 0x7f2fc00a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc109120 0x7f2fdc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fdc1049a0 0x7f2fdc19ab20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 --2- 172.21.15.59:0/1922914109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fdc103ff0 0x7f2fdc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 >> 172.21.15.59:0/1922914109 conn(0x7f2fdc0ffa90 msgr2=0x7f2fdc1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 shutdown_connections 2024-09-05T11:54:25.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:25.695+0000 7f2fe0a00640 1 -- 172.21.15.59:0/1922914109 wait complete. 2024-09-05T11:54:26.415 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-05T11:54:26.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-05T11:54:26.791 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1922914109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:26.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:26.931 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1922914109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1922914109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:54:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 -- 172.21.15.59:0/485293340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4101de0 msgr2=0x7ff1e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 --2- 172.21.15.59:0/485293340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4101de0 0x7ff1e41021c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009900 tx=0x7ff1d802a400 comp rx=0 tx=0).stop 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 -- 172.21.15.59:0/485293340 shutdown_connections 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 --2- 172.21.15.59:0/485293340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4106f10 0x7ff1e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 --2- 172.21.15.59:0/485293340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 0x7ff1e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 --2- 172.21.15.59:0/485293340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4101de0 0x7ff1e41021c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 -- 172.21.15.59:0/485293340 >> 172.21.15.59:0/485293340 conn(0x7ff1e40fd8c0 msgr2=0x7ff1e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 -- 172.21.15.59:0/485293340 shutdown_connections 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.299+0000 7ff1ec600640 1 -- 172.21.15.59:0/485293340 wait complete. 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.300+0000 7ff1ec600640 1 Processor -- start 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 -- start start 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4101de0 0x7ff1e4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 0x7ff1e4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4111fc0 con 0x7ff1e4106f10 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4111e40 con 0x7ff1e4102790 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ec600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e4112140 con 0x7ff1e4101de0 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ea600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1e9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4101de0 0x7ff1e4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ea600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54270/0 (socket says 172.21.15.59:54270) 2024-09-05T11:54:27.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1ea600640 1 -- 172.21.15.59:0/2042964116 learned_addr learned my addr 172.21.15.59:0/2042964116 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.301+0000 7ff1e9200640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 0x7ff1e4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 -- 172.21.15.59:0/2042964116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4101de0 msgr2=0x7ff1e4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4101de0 0x7ff1e4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 -- 172.21.15.59:0/2042964116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 msgr2=0x7ff1e4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 0x7ff1e4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 -- 172.21.15.59:0/2042964116 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e419d3b0 con 0x7ff1e4106f10 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ea600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00eb10 tx=0x7ff1dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004b20 con 0x7ff1e4106f10 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1dc004cc0 con 0x7ff1e4106f10 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc010760 con 0x7ff1e4106f10 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.302+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e419d6a0 con 0x7ff1e4106f10 2024-09-05T11:54:27.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.303+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a4f80 con 0x7ff1e4106f10 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.304+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e41021c0 con 0x7ff1e4106f10 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.306+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1dc005020 con 0x7ff1e4106f10 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.307+0000 7ff1e2000640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 0x7ff1cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.307+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff1dc0cf210 con 0x7ff1e4106f10 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.307+0000 7ff1e9c00640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 0x7ff1cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.308+0000 7ff1e9c00640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 0x7ff1cc0a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8000c00 tx=0x7ff1d8005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:27.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.311+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1dc09c570 con 0x7ff1e4106f10 2024-09-05T11:54:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:27.497+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff1e402d7c0 con 0x7ff1e4106f10 2024-09-05T11:54:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.083+0000 7ff1e2000640 1 -- 172.21.15.59:0/2042964116 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7ff1dc09be70 con 0x7ff1e4106f10 2024-09-05T11:54:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 msgr2=0x7ff1cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 0x7ff1cc0a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8000c00 tx=0x7ff1d8005b50 comp rx=0 tx=0).stop 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 msgr2=0x7ff1e419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00eb10 tx=0x7ff1dc00c2d0 comp rx=0 tx=0).stop 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 shutdown_connections 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff1cc0a2c00 0x7ff1cc0a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e4106f10 0x7ff1e419ccd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e4102790 0x7ff1e4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 --2- 172.21.15.59:0/2042964116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e4101de0 0x7ff1e4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 >> 172.21.15.59:0/2042964116 conn(0x7ff1e40fd8c0 msgr2=0x7ff1e40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 shutdown_connections 2024-09-05T11:54:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:28.086+0000 7ff1ec600640 1 -- 172.21.15.59:0/2042964116 wait complete. 2024-09-05T11:54:28.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:28 smithi189 ceph-mon[34772]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:28.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:28 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T11:54:28.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:28 smithi089 ceph-mon[34499]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:28.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:28 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T11:54:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:28 smithi059 ceph-mon[30025]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:28 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T11:54:28.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-05T11:54:29.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:29 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T11:54:29.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:29 smithi059 ceph-mon[30025]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T11:54:29.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:29 smithi059 ceph-mon[30025]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T11:54:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:29 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T11:54:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:29 smithi089 ceph-mon[34499]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T11:54:29.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:29 smithi089 ceph-mon[34499]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T11:54:29.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:29 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2042964116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T11:54:29.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:29 smithi189 ceph-mon[34772]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T11:54:29.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:29 smithi189 ceph-mon[34772]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T11:54:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 -- 172.21.15.59:0/1232857376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e60108f60 msgr2=0x7f7e60110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 --2- 172.21.15.59:0/1232857376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e60108f60 0x7f7e60110ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009ca0 tx=0x7f7e5802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 -- 172.21.15.59:0/1232857376 shutdown_connections 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 --2- 172.21.15.59:0/1232857376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e60108f60 0x7f7e60110ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 --2- 172.21.15.59:0/1232857376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e601047e0 0x7f7e60108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 --2- 172.21.15.59:0/1232857376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e60104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 -- 172.21.15.59:0/1232857376 >> 172.21.15.59:0/1232857376 conn(0x7f7e600ff8d0 msgr2=0x7f7e60101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 -- 172.21.15.59:0/1232857376 shutdown_connections 2024-09-05T11:54:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.758+0000 7f7e64c00640 1 -- 172.21.15.59:0/1232857376 wait complete. 2024-09-05T11:54:29.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 Processor -- start 2024-09-05T11:54:29.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 -- start start 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e6019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 0x7f7e6019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 0x7f7e6019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60114010 con 0x7f7e601047e0 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60113e90 con 0x7f7e60108f60 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.759+0000 7f7e64c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60114190 con 0x7f7e60103e30 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 0x7f7e6019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e6019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 0x7f7e6019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54280/0 (socket says 172.21.15.59:54280) 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e57e00640 1 -- 172.21.15.59:0/3905952194 learned_addr learned my addr 172.21.15.59:0/3905952194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:29.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5fe00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 0x7f7e6019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 -- 172.21.15.59:0/3905952194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 msgr2=0x7f7e6019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 0x7f7e6019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 -- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 msgr2=0x7f7e6019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 0x7f7e6019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 -- 172.21.15.59:0/3905952194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6019f3a0 con 0x7f7e60103e30 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5fe00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 0x7f7e6019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:54:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.760+0000 7f7e5f400640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e6019a3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9f0 tx=0x7f7e5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.761+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000ccc0 con 0x7f7e60103e30 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.761+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6019f690 con 0x7f7e60103e30 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.761+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50010910 con 0x7f7e60103e30 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.761+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50018960 con 0x7f7e60103e30 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.761+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601a6f70 con 0x7f7e60103e30 2024-09-05T11:54:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.762+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60104210 con 0x7f7e60103e30 2024-09-05T11:54:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.764+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7e50010ac0 con 0x7f7e60103e30 2024-09-05T11:54:29.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.765+0000 7f7e5cc00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 0x7f7e440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:29.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.765+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f7e500cf1f0 con 0x7f7e60103e30 2024-09-05T11:54:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.765+0000 7f7e57e00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 0x7f7e440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.766+0000 7f7e57e00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 0x7f7e440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e60108870 tx=0x7f7e4800b4a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:29.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.769+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e5009c3e0 con 0x7f7e60103e30 2024-09-05T11:54:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:29.957+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7e601a7260 con 0x7f7e60103e30 2024-09-05T11:54:30.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.507+0000 7f7e5cc00640 1 -- 172.21.15.59:0/3905952194 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f7e5009bce0 con 0x7f7e60103e30 2024-09-05T11:54:30.510 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-09-05T11:54:30.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.512+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 msgr2=0x7f7e440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:30.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.512+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 0x7f7e440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e60108870 tx=0x7f7e4800b4a0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.512+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 msgr2=0x7f7e6019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:30.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.512+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e6019a3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9f0 tx=0x7f7e5000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 shutdown_connections 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7e440a2b80 0x7f7e440a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e60108f60 0x7f7e6019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e601047e0 0x7f7e6019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 --2- 172.21.15.59:0/3905952194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e60103e30 0x7f7e6019a3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 >> 172.21.15.59:0/3905952194 conn(0x7f7e600ff8d0 msgr2=0x7f7e601003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 shutdown_connections 2024-09-05T11:54:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:30.513+0000 7f7e64c00640 1 -- 172.21.15.59:0/3905952194 wait complete. 2024-09-05T11:54:30.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:30 smithi189 ceph-mon[34772]: pgmap v284: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:30.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:30 smithi189 ceph-mon[34772]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:30.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:30 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3905952194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:30.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:30 smithi059 ceph-mon[30025]: pgmap v284: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:30.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:30 smithi059 ceph-mon[30025]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:30.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:30 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3905952194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:30 smithi089 ceph-mon[34499]: pgmap v284: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:30 smithi089 ceph-mon[34499]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:30 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3905952194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:31.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-05T11:54:31.665 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:31.804 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:31 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:31.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:31 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T11:54:31.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:31 smithi059 ceph-mon[30025]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T11:54:31.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:31 smithi059 ceph-mon[30025]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T11:54:31.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:31 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:31.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:31 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T11:54:31.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:31 smithi189 ceph-mon[34772]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T11:54:31.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:31 smithi189 ceph-mon[34772]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T11:54:31.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:31 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T11:54:31.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:31 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T11:54:31.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:31 smithi089 ceph-mon[34499]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T11:54:31.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:31 smithi089 ceph-mon[34499]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T11:54:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 -- 172.21.15.59:0/1970878345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc112b90 msgr2=0x7fa2cc114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/1970878345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc112b90 0x7fa2cc114f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009ca0 tx=0x7fa2c402a950 comp rx=0 tx=0).stop 2024-09-05T11:54:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 -- 172.21.15.59:0/1970878345 shutdown_connections 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/1970878345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc112b90 0x7fa2cc114f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/1970878345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/1970878345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc074590 0x7fa2cc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 -- 172.21.15.59:0/1970878345 >> 172.21.15.59:0/1970878345 conn(0x7fa2cc0fd8c0 msgr2=0x7fa2cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 -- 172.21.15.59:0/1970878345 shutdown_connections 2024-09-05T11:54:32.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.254+0000 7fa2d1a00640 1 -- 172.21.15.59:0/1970878345 wait complete. 2024-09-05T11:54:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.255+0000 7fa2d1a00640 1 Processor -- start 2024-09-05T11:54:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.255+0000 7fa2d1a00640 1 -- start start 2024-09-05T11:54:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc112b90 0x7fa2cc1a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1185a0 con 0x7fa2cc074590 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc118420 con 0x7fa2cc112b90 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc118720 con 0x7fa2cc074eb0 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40386/0 (socket says 172.21.15.59:40386) 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2caa00640 1 -- 172.21.15.59:0/793985295 learned_addr learned my addr 172.21.15.59:0/793985295 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2ca000640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.256+0000 7fa2cb400640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc112b90 0x7fa2cc1a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 -- 172.21.15.59:0/793985295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc112b90 msgr2=0x7fa2cc1a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc112b90 0x7fa2cc1a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 -- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 msgr2=0x7fa2cc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 -- 172.21.15.59:0/793985295 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc1a3b20 con 0x7fa2cc074eb0 2024-09-05T11:54:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2caa00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2ca000640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc19f030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800e9f0 tx=0x7fa2b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800ccc0 con 0x7fa2cc074eb0 2024-09-05T11:54:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc1a3e10 con 0x7fa2cc074eb0 2024-09-05T11:54:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8010910 con 0x7fa2cc074eb0 2024-09-05T11:54:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8018960 con 0x7fa2cc074eb0 2024-09-05T11:54:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.257+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1ab6a0 con 0x7fa2cc074eb0 2024-09-05T11:54:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.258+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc074970 con 0x7fa2cc074eb0 2024-09-05T11:54:32.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.260+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa2b8010ac0 con 0x7fa2cc074eb0 2024-09-05T11:54:32.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.262+0000 7fa2c3400640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 0x7fa2b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:32.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.262+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fa2b80cf6e0 con 0x7fa2cc074eb0 2024-09-05T11:54:32.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.262+0000 7fa2caa00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 0x7fa2b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:32.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.263+0000 7fa2caa00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 0x7fa2b00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011550 tx=0x7fa2bc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:32.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.264+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2b809c100 con 0x7fa2cc074eb0 2024-09-05T11:54:32.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.489+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2cc104a40 con 0x7fa2cc074eb0 2024-09-05T11:54:32.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.548+0000 7fa2c3400640 1 -- 172.21.15.59:0/793985295 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa2b80bc110 con 0x7fa2cc074eb0 2024-09-05T11:54:32.550 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.551+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 msgr2=0x7fa2b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.551+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 0x7fa2b00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011550 tx=0x7fa2bc005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.551+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 msgr2=0x7fa2cc19f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.551+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc19f030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800e9f0 tx=0x7fa2b800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 shutdown_connections 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2b00a2b80 0x7fa2b00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2cc112b90 0x7fa2cc1a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2cc074eb0 0x7fa2cc19f030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 --2- 172.21.15.59:0/793985295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc074590 0x7fa2cc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 >> 172.21.15.59:0/793985295 conn(0x7fa2cc0fd8c0 msgr2=0x7fa2cc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 shutdown_connections 2024-09-05T11:54:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:32.552+0000 7fa2d1a00640 1 -- 172.21.15.59:0/793985295 wait complete. 2024-09-05T11:54:32.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:32 smithi189 ceph-mon[34772]: pgmap v287: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:32.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:32 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/793985295' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:32 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:32 smithi189 ceph-mon[34772]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T11:54:32.896 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:32 smithi059 ceph-mon[30025]: pgmap v287: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:32.896 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:32 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/793985295' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.896 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:32 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.896 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:32 smithi059 ceph-mon[30025]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T11:54:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:32 smithi089 ceph-mon[34499]: pgmap v287: 97 pgs: 12 active+clean, 10 creating+peering, 75 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:32 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/793985295' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:32 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T11:54:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:32 smithi089 ceph-mon[34499]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T11:54:33.325 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-05T11:54:33.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-05T11:54:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:33.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:33 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T11:54:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:33 smithi189 ceph-mon[34772]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T11:54:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:33 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T11:54:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:33 smithi189 ceph-mon[34772]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T11:54:33.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:33 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T11:54:33.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:33 smithi059 ceph-mon[30025]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T11:54:33.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:33 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T11:54:33.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:33 smithi059 ceph-mon[30025]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T11:54:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:33 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T11:54:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:33 smithi089 ceph-mon[34499]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T11:54:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:33 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T11:54:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:33 smithi089 ceph-mon[34499]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T11:54:34.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.282+0000 7f313ce00640 1 -- 172.21.15.59:0/963550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313810e4e0 msgr2=0x7f31381108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:34.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.282+0000 7f313ce00640 1 --2- 172.21.15.59:0/963550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313810e4e0 0x7f31381108d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3130009be0 tx=0x7f313002a6c0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.282+0000 7f313ce00640 1 -- 172.21.15.59:0/963550558 shutdown_connections 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.282+0000 7f313ce00640 1 --2- 172.21.15.59:0/963550558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f313810e4e0 0x7f31381108d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.283+0000 7f313ce00640 1 --2- 172.21.15.59:0/963550558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31381012e0 0x7f313810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.283+0000 7f313ce00640 1 --2- 172.21.15.59:0/963550558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3138100930 0x7f3138100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.283+0000 7f313ce00640 1 -- 172.21.15.59:0/963550558 >> 172.21.15.59:0/963550558 conn(0x7f31380fc7a0 msgr2=0x7f31380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.283+0000 7f313ce00640 1 -- 172.21.15.59:0/963550558 shutdown_connections 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.283+0000 7f313ce00640 1 -- 172.21.15.59:0/963550558 wait complete. 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.284+0000 7f313ce00640 1 Processor -- start 2024-09-05T11:54:34.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.284+0000 7f313ce00640 1 -- start start 2024-09-05T11:54:34.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 0x7f313819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:34.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 0x7f313819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:34.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 0x7f313819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3138114110 con 0x7f31381012e0 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3138113f90 con 0x7f3138100930 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f313ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3138114290 con 0x7f313810e4e0 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3135400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 0x7f313819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3135400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 0x7f313819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40400/0 (socket says 172.21.15.59:40400) 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3135400640 1 -- 172.21.15.59:0/1494671611 learned_addr learned my addr 172.21.15.59:0/1494671611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 0x7f313819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:34.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3135e00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 0x7f313819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:34.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 -- 172.21.15.59:0/1494671611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 msgr2=0x7f313819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:34.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 0x7f313819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 -- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 msgr2=0x7f313819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 0x7f313819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3136800640 1 -- 172.21.15.59:0/1494671611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313819f600 con 0x7f313810e4e0 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.285+0000 7f3135e00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 0x7f313819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f3136800640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 0x7f313819ef20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3130034040 tx=0x7f31300305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130041020 con 0x7f313810e4e0 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313819f890 con 0x7f313810e4e0 2024-09-05T11:54:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3130037040 con 0x7f313810e4e0 2024-09-05T11:54:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130036070 con 0x7f313810e4e0 2024-09-05T11:54:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.286+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381a71d0 con 0x7f313810e4e0 2024-09-05T11:54:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.288+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313806b010 con 0x7f313810e4e0 2024-09-05T11:54:34.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.289+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3130044020 con 0x7f313810e4e0 2024-09-05T11:54:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.291+0000 7f312ea00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 0x7f311c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.291+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f31300ef9a0 con 0x7f313810e4e0 2024-09-05T11:54:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.291+0000 7f3135e00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 0x7f311c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.292+0000 7f3135e00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 0x7f311c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3128005fd0 tx=0x7f3128009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.295+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f313004b020 con 0x7f313810e4e0 2024-09-05T11:54:34.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.488+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3138109670 con 0x7f313810e4e0 2024-09-05T11:54:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.569+0000 7f312ea00640 1 -- 172.21.15.59:0/1494671611 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f31300b6e20 con 0x7f313810e4e0 2024-09-05T11:54:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.571+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 msgr2=0x7f311c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.571+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 0x7f311c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3128005fd0 tx=0x7f3128009260 comp rx=0 tx=0).stop 2024-09-05T11:54:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 msgr2=0x7f313819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 0x7f313819ef20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3130034040 tx=0x7f31300305d0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 shutdown_connections 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f311c0a2b80 0x7f311c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f313810e4e0 0x7f313819ef20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31381012e0 0x7f313819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 --2- 172.21.15.59:0/1494671611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3138100930 0x7f313819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 >> 172.21.15.59:0/1494671611 conn(0x7f31380fc7a0 msgr2=0x7f31380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 shutdown_connections 2024-09-05T11:54:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:34.572+0000 7f313ce00640 1 -- 172.21.15.59:0/1494671611 wait complete. 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: pgmap v290: 97 pgs: 45 active+clean, 16 creating+peering, 36 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: Cluster is now healthy 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:active 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1494671611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:34 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: pgmap v290: 97 pgs: 45 active+clean, 16 creating+peering, 36 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: Cluster is now healthy 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:active 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1494671611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:34 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: pgmap v290: 97 pgs: 45 active+clean, 16 creating+peering, 36 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T11:54:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: Cluster is now healthy 2024-09-05T11:54:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] up:active 2024-09-05T11:54:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:34.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1494671611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:34.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:34 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T11:54:35.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-05T11:54:35.434 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 11:54:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: 2024-09-05T11:54:35.105+0000 7f76759b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-05T11:54:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:35.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:35 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T11:54:35.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:35 smithi189 ceph-mon[34772]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T11:54:35.864 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:35 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T11:54:35.864 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:35 smithi059 ceph-mon[30025]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T11:54:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:35 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T11:54:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:35 smithi089 ceph-mon[34499]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T11:54:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 -- 172.21.15.59:0/3811504311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8076720 msgr2=0x7fc5e8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 --2- 172.21.15.59:0/3811504311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8076720 0x7fc5e8076b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 -- 172.21.15.59:0/3811504311 shutdown_connections 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 --2- 172.21.15.59:0/3811504311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 0x7fc5e8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 --2- 172.21.15.59:0/3811504311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8077040 0x7fc5e81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 --2- 172.21.15.59:0/3811504311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8076720 0x7fc5e8076b00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 -- 172.21.15.59:0/3811504311 >> 172.21.15.59:0/3811504311 conn(0x7fc5e80ffa90 msgr2=0x7fc5e8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.244+0000 7fc5f0800640 1 -- 172.21.15.59:0/3811504311 shutdown_connections 2024-09-05T11:54:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.245+0000 7fc5f0800640 1 -- 172.21.15.59:0/3811504311 wait complete. 2024-09-05T11:54:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 Processor -- start 2024-09-05T11:54:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 -- start start 2024-09-05T11:54:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8076720 0x7fc5e81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 0x7fc5e819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81185f0 con 0x7fc5e8077040 2024-09-05T11:54:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8118470 con 0x7fc5e8076720 2024-09-05T11:54:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5f0800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8118770 con 0x7fc5e8112a30 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5ed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40418/0 (socket says 172.21.15.59:40418) 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.246+0000 7fc5ed400640 1 -- 172.21.15.59:0/1346114256 learned_addr learned my addr 172.21.15.59:0/1346114256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ede00640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8076720 0x7fc5e81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ee800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 0x7fc5e819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 -- 172.21.15.59:0/1346114256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 msgr2=0x7fc5e819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 0x7fc5e819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 -- 172.21.15.59:0/1346114256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8076720 msgr2=0x7fc5e81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8076720 0x7fc5e81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 -- 172.21.15.59:0/1346114256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e819ef90 con 0x7fc5e8077040 2024-09-05T11:54:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.247+0000 7fc5ed400640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9f0 tx=0x7fc5d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.248+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800cd80 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.248+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8010910 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.248+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e819f280 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.248+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8018990 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.248+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e81ab570 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.249+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e8076b00 con 0x7fc5e8077040 2024-09-05T11:54:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.251+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc5d8010ac0 con 0x7fc5e8077040 2024-09-05T11:54:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.252+0000 7fc5e2a00640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 0x7fc5d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.252+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7fc5d80cf820 con 0x7fc5e8077040 2024-09-05T11:54:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.252+0000 7fc5ede00640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 0x7fc5d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:36.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.253+0000 7fc5ede00640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 0x7fc5d00a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006010 tx=0x7fc5dc000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:36.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.255+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5d809c0a0 con 0x7fc5e8077040 2024-09-05T11:54:36.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:36.440+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc5e81ab860 con 0x7fc5e8077040 2024-09-05T11:54:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:37 smithi089 ceph-mon[34499]: pgmap v292: 97 pgs: 72 active+clean, 13 creating+peering, 12 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 1 op/s 2024-09-05T11:54:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:37 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T11:54:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:37 smithi059 ceph-mon[30025]: pgmap v292: 97 pgs: 72 active+clean, 13 creating+peering, 12 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 1 op/s 2024-09-05T11:54:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:37 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T11:54:37.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.582+0000 7fc5e2a00640 1 -- 172.21.15.59:0/1346114256 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7fc5d80a03a0 con 0x7fc5e8077040 2024-09-05T11:54:37.584 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' created 2024-09-05T11:54:37.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 msgr2=0x7fc5d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 0x7fc5d00a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006010 tx=0x7fc5dc000bc0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 msgr2=0x7fc5e81a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9f0 tx=0x7fc5d800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 shutdown_connections 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc5d00a2b80 0x7fc5d00a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5e8112a30 0x7fc5e819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5e8077040 0x7fc5e81a4be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 --2- 172.21.15.59:0/1346114256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5e8076720 0x7fc5e81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 >> 172.21.15.59:0/1346114256 conn(0x7fc5e80ffa90 msgr2=0x7fc5e8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:37.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.586+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 shutdown_connections 2024-09-05T11:54:37.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:37 smithi189 ceph-mon[34772]: pgmap v292: 97 pgs: 72 active+clean, 13 creating+peering, 12 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 235 B/s wr, 1 op/s 2024-09-05T11:54:37.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:37 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T11:54:37.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:37.587+0000 7fc5f0800640 1 -- 172.21.15.59:0/1346114256 wait complete. 2024-09-05T11:54:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:38 smithi059 ceph-mon[30025]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-05T11:54:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:38 smithi059 ceph-mon[30025]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T11:54:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:38 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T11:54:38.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:38 smithi059 ceph-mon[30025]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T11:54:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:38 smithi089 ceph-mon[34499]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-05T11:54:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:38 smithi089 ceph-mon[34499]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T11:54:38.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:38 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T11:54:38.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:38 smithi089 ceph-mon[34499]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T11:54:38.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-05T11:54:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:38 smithi189 ceph-mon[34772]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-05T11:54:38.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:38 smithi189 ceph-mon[34772]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T11:54:38.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:38 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1346114256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T11:54:38.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:38 smithi189 ceph-mon[34772]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T11:54:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 -- 172.21.15.59:0/2699873882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c104010 msgr2=0x7f196c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 --2- 172.21.15.59:0/2699873882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c104010 0x7f196c1043f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1960009900 tx=0x7f196002a410 comp rx=0 tx=0).stop 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 -- 172.21.15.59:0/2699873882 shutdown_connections 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 --2- 172.21.15.59:0/2699873882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c109140 0x7f196c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 --2- 172.21.15.59:0/2699873882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 --2- 172.21.15.59:0/2699873882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c104010 0x7f196c1043f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 -- 172.21.15.59:0/2699873882 >> 172.21.15.59:0/2699873882 conn(0x7f196c0ffa50 msgr2=0x7f196c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 -- 172.21.15.59:0/2699873882 shutdown_connections 2024-09-05T11:54:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.577+0000 7f1974800640 1 -- 172.21.15.59:0/2699873882 wait complete. 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 Processor -- start 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 -- start start 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c109140 0x7f196c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f196c114140 con 0x7f196c109140 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f196c113fc0 con 0x7f196c104010 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.578+0000 7f1974800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1142c0 con 0x7f196c1049c0 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:41244/0 (socket says 172.21.15.59:41244) 2024-09-05T11:54:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 -- 172.21.15.59:0/2111737671 learned_addr learned my addr 172.21.15.59:0/2111737671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1972800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c109140 0x7f196c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 -- 172.21.15.59:0/2111737671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 msgr2=0x7f196c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 -- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c109140 msgr2=0x7f196c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c109140 0x7f196c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 -- 172.21.15.59:0/2111737671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c19f5b0 con 0x7f196c104010 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971400640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.579+0000 7f1971e00640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1960006010 tx=0x7f1960004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.580+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196003f030 con 0x7f196c104010 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.580+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1960004440 con 0x7f196c104010 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.580+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960035070 con 0x7f196c104010 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.580+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c19f840 con 0x7f196c104010 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.580+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1a7180 con 0x7f196c104010 2024-09-05T11:54:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.581+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c1043f0 con 0x7f196c104010 2024-09-05T11:54:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.583+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1960042030 con 0x7f196c104010 2024-09-05T11:54:39.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.584+0000 7f1966a00640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 0x7f19540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.584+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f19600ef8b0 con 0x7f196c104010 2024-09-05T11:54:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.585+0000 7f1971400640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 0x7f19540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.585+0000 7f1971400640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 0x7f19540a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f196c0ffe70 tx=0x7f195c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.586+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19600bc000 con 0x7f196c104010 2024-09-05T11:54:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:39.771+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f196c10c320 con 0x7f196c104010 2024-09-05T11:54:39.773 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:39 smithi059 ceph-mon[30025]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T11:54:39.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:39 smithi089 ceph-mon[34499]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T11:54:40.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:39 smithi189 ceph-mon[34772]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T11:54:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.605+0000 7f1966a00640 1 -- 172.21.15.59:0/2111737671 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f19600c0300 con 0x7f196c104010 2024-09-05T11:54:40.607 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 4 allow_ec_overwrites to true 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 msgr2=0x7f19540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 0x7f19540a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f196c0ffe70 tx=0x7f195c005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 msgr2=0x7f196c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1960006010 tx=0x7f1960004040 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 shutdown_connections 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19540a2c00 0x7f19540a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c109140 0x7f196c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f196c1049c0 0x7f196c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 --2- 172.21.15.59:0/2111737671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f196c104010 0x7f196c19a600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 >> 172.21.15.59:0/2111737671 conn(0x7f196c0ffa50 msgr2=0x7f196c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 shutdown_connections 2024-09-05T11:54:40.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:40.609+0000 7f1974800640 1 -- 172.21.15.59:0/2111737671 wait complete. 2024-09-05T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:40 smithi089 ceph-mon[34499]: pgmap v297: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-05T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:40 smithi089 ceph-mon[34499]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:40 smithi089 ceph-mon[34499]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:40 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2111737671' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:40 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:40.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:40 smithi059 ceph-mon[30025]: pgmap v297: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-05T11:54:40.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:40 smithi059 ceph-mon[30025]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:40.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:40 smithi059 ceph-mon[30025]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T11:54:40.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:40 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2111737671' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:40.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:40 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:40 smithi189 ceph-mon[34772]: pgmap v297: 129 pgs: 3 creating+peering, 25 unknown, 101 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-09-05T11:54:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:40 smithi189 ceph-mon[34772]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T11:54:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:40 smithi189 ceph-mon[34772]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T11:54:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:40 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2111737671' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:40 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T11:54:41.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-05T11:54:41.653 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:41.864 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:41 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T11:54:41.864 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:41 smithi059 ceph-mon[30025]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T11:54:41.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:41 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T11:54:41.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:41 smithi089 ceph-mon[34499]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T11:54:42.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:41 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T11:54:42.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:41 smithi189 ceph-mon[34772]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.272+0000 7ffae5400640 1 -- 172.21.15.59:0/4294259123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae01049c0 msgr2=0x7ffae0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.272+0000 7ffae5400640 1 --2- 172.21.15.59:0/4294259123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae01049c0 0x7ffae0108a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009900 tx=0x7ffac802a410 comp rx=0 tx=0).stop 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.272+0000 7ffae5400640 1 -- 172.21.15.59:0/4294259123 shutdown_connections 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.272+0000 7ffae5400640 1 --2- 172.21.15.59:0/4294259123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0109140 0x7ffae0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.272+0000 7ffae5400640 1 --2- 172.21.15.59:0/4294259123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae01049c0 0x7ffae0108a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.273+0000 7ffae5400640 1 --2- 172.21.15.59:0/4294259123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae0104010 0x7ffae01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.273+0000 7ffae5400640 1 -- 172.21.15.59:0/4294259123 >> 172.21.15.59:0/4294259123 conn(0x7ffae00ffa50 msgr2=0x7ffae0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.273+0000 7ffae5400640 1 -- 172.21.15.59:0/4294259123 shutdown_connections 2024-09-05T11:54:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.273+0000 7ffae5400640 1 -- 172.21.15.59:0/4294259123 wait complete. 2024-09-05T11:54:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.273+0000 7ffae5400640 1 Processor -- start 2024-09-05T11:54:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 -- start start 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0104010 0x7ffae019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffae01141b0 con 0x7ffae0109140 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114030 con 0x7ffae0104010 2024-09-05T11:54:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffae5400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114330 con 0x7ffae01049c0 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffadf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:49586/0 (socket says 172.21.15.59:49586) 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffadf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44242/0 (socket says 172.21.15.59:44242) 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffad5e00640 1 -- 172.21.15.59:0/4105381593 learned_addr learned my addr 172.21.15.59:0/4105381593 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.274+0000 7ffade800640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0104010 0x7ffae019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 -- 172.21.15.59:0/4105381593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 msgr2=0x7ffae019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 -- 172.21.15.59:0/4105381593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0104010 msgr2=0x7ffae019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0104010 0x7ffae019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 -- 172.21.15.59:0/4105381593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae019f6c0 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffad5e00640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffadf200640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009fd0 tx=0x7ffad800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.275+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad800ce20 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.276+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae019f9b0 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.276+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad8005020 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.276+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8010700 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.276+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae01a7290 con 0x7ffae0109140 2024-09-05T11:54:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.277+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae01043f0 con 0x7ffae0109140 2024-09-05T11:54:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.279+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffad8005540 con 0x7ffae0109140 2024-09-05T11:54:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.280+0000 7ffad7e00640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.280+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7ffad80cf8c0 con 0x7ffae0109140 2024-09-05T11:54:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.280+0000 7ffade800640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.281+0000 7ffade800640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 0x7ffac40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009bd0 tx=0x7ffad0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.285+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffad809c5a0 con 0x7ffae0109140 2024-09-05T11:54:42.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.508+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ffae010c3e0 con 0x7ffae0109140 2024-09-05T11:54:42.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.613+0000 7ffad7e00640 1 -- 172.21.15.59:0/4105381593 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7ffad809bea0 con 0x7ffae0109140 2024-09-05T11:54:42.615 INFO:teuthology.orchestra.run.smithi059.stderr:added data pool 4 to fsmap 2024-09-05T11:54:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.615+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 msgr2=0x7ffac40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.615+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 0x7ffac40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009bd0 tx=0x7ffad0005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.615+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 msgr2=0x7ffae019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.615+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009fd0 tx=0x7ffad800ef10 comp rx=0 tx=0).stop 2024-09-05T11:54:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 shutdown_connections 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae0109140 0x7ffae019efe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffae01049c0 0x7ffae019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 --2- 172.21.15.59:0/4105381593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffae0104010 0x7ffae019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 >> 172.21.15.59:0/4105381593 conn(0x7ffae00ffa50 msgr2=0x7ffae0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 shutdown_connections 2024-09-05T11:54:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:42.616+0000 7ffae5400640 1 -- 172.21.15.59:0/4105381593 wait complete. 2024-09-05T11:54:42.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:42 smithi089 ceph-mon[34499]: pgmap v300: 129 pgs: 3 creating+peering, 16 unknown, 110 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-05T11:54:42.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:42 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T11:54:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:42 smithi059 ceph-mon[30025]: pgmap v300: 129 pgs: 3 creating+peering, 16 unknown, 110 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-05T11:54:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T11:54:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:42 smithi189 ceph-mon[34772]: pgmap v300: 129 pgs: 3 creating+peering, 16 unknown, 110 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-05T11:54:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:42 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T11:54:43.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:43.901 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:43 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T11:54:43.901 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:43 smithi059 ceph-mon[30025]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T11:54:43.901 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:43 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:43.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:43 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T11:54:43.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:43 smithi089 ceph-mon[34499]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T11:54:43.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:43 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:43 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4105381593' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T11:54:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:43 smithi189 ceph-mon[34772]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T11:54:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:43 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:54:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 -- 172.21.15.59:0/1492829255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb81049a0 msgr2=0x7fbdb81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 --2- 172.21.15.59:0/1492829255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb81049a0 0x7fbdb81089f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402a410 comp rx=0 tx=0).stop 2024-09-05T11:54:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 -- 172.21.15.59:0/1492829255 shutdown_connections 2024-09-05T11:54:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 --2- 172.21.15.59:0/1492829255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8109120 0x7fbdb8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 --2- 172.21.15.59:0/1492829255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb81049a0 0x7fbdb81089f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 --2- 172.21.15.59:0/1492829255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb8103ff0 0x7fbdb81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.304+0000 7fbdbf600640 1 -- 172.21.15.59:0/1492829255 >> 172.21.15.59:0/1492829255 conn(0x7fbdb80ffa90 msgr2=0x7fbdb8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.305+0000 7fbdbf600640 1 -- 172.21.15.59:0/1492829255 shutdown_connections 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.305+0000 7fbdbf600640 1 -- 172.21.15.59:0/1492829255 wait complete. 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.305+0000 7fbdbf600640 1 Processor -- start 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 -- start start 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8103ff0 0x7fbdb819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb81049a0 0x7fbdb819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8114190 con 0x7fbdb8109120 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8114010 con 0x7fbdb81049a0 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbf600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8114310 con 0x7fbdb8103ff0 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44262/0 (socket says 172.21.15.59:44262) 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdbd600640 1 -- 172.21.15.59:0/823725831 learned_addr learned my addr 172.21.15.59:0/823725831 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.306+0000 7fbdb7e00640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb81049a0 0x7fbdb819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbcc00640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8103ff0 0x7fbdb819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 -- 172.21.15.59:0/823725831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8103ff0 msgr2=0x7fbdb819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8103ff0 0x7fbdb819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 -- 172.21.15.59:0/823725831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb81049a0 msgr2=0x7fbdb819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb81049a0 0x7fbdb819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 -- 172.21.15.59:0/823725831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb819f5a0 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdbd600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000eb10 tx=0x7fbdb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.307+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0004b20 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.308+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb0004cc0 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.308+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb819f890 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.308+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0010760 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.308+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb81a7170 con 0x7fbdb8109120 2024-09-05T11:54:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.309+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb81043d0 con 0x7fbdb8109120 2024-09-05T11:54:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.311+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbdb0005020 con 0x7fbdb8109120 2024-09-05T11:54:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.312+0000 7fbdb5600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 0x7fbda00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.312+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbdb00cf750 con 0x7fbdb8109120 2024-09-05T11:54:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.312+0000 7fbdbcc00640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 0x7fbda00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.313+0000 7fbdbcc00640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 0x7fbda00a5190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009b60 tx=0x7fbdac005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:44.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.316+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdb009c410 con 0x7fbdb8109120 2024-09-05T11:54:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.539+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdb81a7410 con 0x7fbdb8109120 2024-09-05T11:54:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.539+0000 7fbdb5600640 1 -- 172.21.15.59:0/823725831 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7fbdb009bd10 con 0x7fbdb8109120 2024-09-05T11:54:44.541 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:44.541 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"btime":"2024-09-05T11:54:42:610654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:54:42.509631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:54:44.542 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 msgr2=0x7fbda00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 0x7fbda00a5190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009b60 tx=0x7fbdac005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 msgr2=0x7fbdb819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000eb10 tx=0x7fbdb000c2d0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 shutdown_connections 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbda00a2cd0 0x7fbda00a5190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdb8109120 0x7fbdb819eec0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbdb81049a0 0x7fbdb819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 --2- 172.21.15.59:0/823725831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb8103ff0 0x7fbdb819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:44.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 >> 172.21.15.59:0/823725831 conn(0x7fbdb80ffa90 msgr2=0x7fbdb8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:44.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 shutdown_connections 2024-09-05T11:54:44.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:44.542+0000 7fbdbf600640 1 -- 172.21.15.59:0/823725831 wait complete. 2024-09-05T11:54:44.922 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:44 smithi059 ceph-mon[30025]: pgmap v302: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-05T11:54:44.922 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:44 smithi059 ceph-mon[30025]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T11:54:44.922 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:44 smithi059 ceph-mon[30025]: Cluster is now healthy 2024-09-05T11:54:44.922 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:44 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/823725831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:44 smithi089 ceph-mon[34499]: pgmap v302: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-05T11:54:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:44 smithi089 ceph-mon[34499]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T11:54:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:44 smithi089 ceph-mon[34499]: Cluster is now healthy 2024-09-05T11:54:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:44 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/823725831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:44 smithi189 ceph-mon[34772]: pgmap v302: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-05T11:54:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:44 smithi189 ceph-mon[34772]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T11:54:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:44 smithi189 ceph-mon[34772]: Cluster is now healthy 2024-09-05T11:54:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:44 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/823725831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:45.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:54:45.774 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:46.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 -- 172.21.15.59:0/3247424836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81079c0 msgr2=0x7f45c8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:46.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 --2- 172.21.15.59:0/3247424836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81079c0 0x7f45c8107da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009930 tx=0x7f45b802a420 comp rx=0 tx=0).stop 2024-09-05T11:54:46.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 -- 172.21.15.59:0/3247424836 shutdown_connections 2024-09-05T11:54:46.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 --2- 172.21.15.59:0/3247424836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c8102360 0x7f45c810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 --2- 172.21.15.59:0/3247424836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c81019c0 0x7f45c8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 --2- 172.21.15.59:0/3247424836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81079c0 0x7f45c8107da0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 -- 172.21.15.59:0/3247424836 >> 172.21.15.59:0/3247424836 conn(0x7f45c80fd740 msgr2=0x7f45c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 -- 172.21.15.59:0/3247424836 shutdown_connections 2024-09-05T11:54:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.320+0000 7f45cd400640 1 -- 172.21.15.59:0/3247424836 wait complete. 2024-09-05T11:54:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.321+0000 7f45cd400640 1 Processor -- start 2024-09-05T11:54:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.321+0000 7f45cd400640 1 -- start start 2024-09-05T11:54:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c8102360 0x7f45c819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c81079c0 0x7f45c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111e10 con 0x7f45c81019c0 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111c90 con 0x7f45c8102360 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cd400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111f90 con 0x7f45c81079c0 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44266/0 (socket says 172.21.15.59:44266) 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45c7e00640 1 -- 172.21.15.59:0/3049958890 learned_addr learned my addr 172.21.15.59:0/3049958890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45cca00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c81079c0 0x7f45c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45bfe00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c8102360 0x7f45c819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.322+0000 7f45c7e00640 1 -- 172.21.15.59:0/3049958890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c81079c0 msgr2=0x7f45c819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c7e00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c81079c0 0x7f45c819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c7e00640 1 -- 172.21.15.59:0/3049958890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c8102360 msgr2=0x7f45c819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c7e00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c8102360 0x7f45c819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c7e00640 1 -- 172.21.15.59:0/3049958890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c819f440 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c7e00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a940 tx=0x7f45b8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b803f070 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b8031c90 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c819f6d0 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8035030 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.323+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81a7010 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.324+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8103100 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.326+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45b8042070 con 0x7f45c81019c0 2024-09-05T11:54:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.329+0000 7f45c5600640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 0x7f45ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:46.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.329+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f45b80f0080 con 0x7f45c81019c0 2024-09-05T11:54:46.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.329+0000 7f45bfe00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 0x7f45ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:46.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.329+0000 7f45bfe00640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 0x7f45ac0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f45c810ec90 tx=0x7f45b0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:46.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.331+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45b80bc6f0 con 0x7f45c81019c0 2024-09-05T11:54:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.520+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45c81a71e0 con 0x7f45c81019c0 2024-09-05T11:54:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.522+0000 7f45c5600640 1 -- 172.21.15.59:0/3049958890 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24202 (secure 0 0 0) 0x7f45b80c09f0 con 0x7f45c81019c0 2024-09-05T11:54:46.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:46.524 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:54:42.509492+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:54:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.525+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 msgr2=0x7f45ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.525+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 0x7f45ac0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f45c810ec90 tx=0x7f45b0005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.525+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 msgr2=0x7f45c819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.525+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a940 tx=0x7f45b8003b20 comp rx=0 tx=0).stop 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 shutdown_connections 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45ac0a2b80 0x7f45ac0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45c81079c0 0x7f45c819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45c8102360 0x7f45c819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 --2- 172.21.15.59:0/3049958890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45c81019c0 0x7f45c819a490 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 >> 172.21.15.59:0/3049958890 conn(0x7f45c80fd740 msgr2=0x7f45c80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 shutdown_connections 2024-09-05T11:54:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:46.526+0000 7f45cd400640 1 -- 172.21.15.59:0/3049958890 wait complete. 2024-09-05T11:54:46.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:46 smithi059 ceph-mon[30025]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:46.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3049958890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:46 smithi089 ceph-mon[34499]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3049958890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:46 smithi189 ceph-mon[34772]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3049958890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:47.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:54:47.637 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:48.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.228+0000 7fbd7fe00640 1 -- 172.21.15.59:0/3093330265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 msgr2=0x7fbd7810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.228+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/3093330265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7810eaa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ca0 tx=0x7fbd7002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 -- 172.21.15.59:0/3093330265 shutdown_connections 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/3093330265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7810eaa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/3093330265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd781027a0 0x7fbd781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/3093330265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd78101df0 0x7fbd781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 -- 172.21.15.59:0/3093330265 >> 172.21.15.59:0/3093330265 conn(0x7fbd780fd8b0 msgr2=0x7fbd780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 -- 172.21.15.59:0/3093330265 shutdown_connections 2024-09-05T11:54:48.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 -- 172.21.15.59:0/3093330265 wait complete. 2024-09-05T11:54:48.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.229+0000 7fbd7fe00640 1 Processor -- start 2024-09-05T11:54:48.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 -- start start 2024-09-05T11:54:48.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 0x7fbd7819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:48.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd781027a0 0x7fbd7819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:48.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78111fd0 con 0x7fbd78106f20 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78111e50 con 0x7fbd781027a0 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78112150 con 0x7fbd78101df0 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 0x7fbd7819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44282/0 (socket says 172.21.15.59:44282) 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.230+0000 7fbd7de00640 1 -- 172.21.15.59:0/588746705 learned_addr learned my addr 172.21.15.59:0/588746705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7ca00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd781027a0 0x7fbd7819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 -- 172.21.15.59:0/588746705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd781027a0 msgr2=0x7fbd7819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd781027a0 0x7fbd7819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 -- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 msgr2=0x7fbd7819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 -- 172.21.15.59:0/588746705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7819f620 con 0x7fbd78101df0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7de00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.231+0000 7fbd7d400640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 0x7fbd7819a670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9f0 tx=0x7fbd6c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.232+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00ccc0 con 0x7fbd78101df0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.232+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7819f910 con 0x7fbd78101df0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.232+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6c010910 con 0x7fbd78101df0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.232+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c018960 con 0x7fbd78101df0 2024-09-05T11:54:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.232+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781a71f0 con 0x7fbd78101df0 2024-09-05T11:54:48.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.233+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd781021d0 con 0x7fbd78101df0 2024-09-05T11:54:48.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.235+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbd6c010ac0 con 0x7fbd78101df0 2024-09-05T11:54:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.236+0000 7fbd76000640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.236+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbd6c0cf9f0 con 0x7fbd78101df0 2024-09-05T11:54:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.236+0000 7fbd7ca00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.237+0000 7fbd7ca00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 0x7fbd600a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd78076480 tx=0x7fbd64005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:48.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.238+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd6c09c060 con 0x7fbd78101df0 2024-09-05T11:54:48.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.423+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd781a73f0 con 0x7fbd78101df0 2024-09-05T11:54:48.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.425+0000 7fbd76000640 1 -- 172.21.15.59:0/588746705 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24202 (secure 0 0 0) 0x7fbd6c0a0360 con 0x7fbd78101df0 2024-09-05T11:54:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:48.427 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:54:42.509492+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:54:48.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 msgr2=0x7fbd600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:48.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 0x7fbd600a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd78076480 tx=0x7fbd64005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:48.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 msgr2=0x7fbd7819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:48.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 0x7fbd7819a670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00e9f0 tx=0x7fbd6c00eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 shutdown_connections 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd78106f20 0x7fbd7819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd781027a0 0x7fbd7819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 --2- 172.21.15.59:0/588746705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd78101df0 0x7fbd7819a670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 >> 172.21.15.59:0/588746705 conn(0x7fbd780fd8b0 msgr2=0x7fbd780fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 shutdown_connections 2024-09-05T11:54:48.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:48.428+0000 7fbd7fe00640 1 -- 172.21.15.59:0/588746705 wait complete. 2024-09-05T11:54:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:48 smithi059 ceph-mon[30025]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:48 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/588746705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:48 smithi089 ceph-mon[34499]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:48 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/588746705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:48 smithi189 ceph-mon[34772]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:48 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/588746705' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:49.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:54:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 -- 172.21.15.59:0/2876075744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228077040 msgr2=0x7fc228112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 --2- 172.21.15.59:0/2876075744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228077040 0x7fc228112510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402a410 comp rx=0 tx=0).stop 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 -- 172.21.15.59:0/2876075744 shutdown_connections 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 --2- 172.21.15.59:0/2876075744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228112a50 0x7fc228114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 --2- 172.21.15.59:0/2876075744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228077040 0x7fc228112510 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 --2- 172.21.15.59:0/2876075744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228076720 0x7fc228076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.295+0000 7fc230600640 1 -- 172.21.15.59:0/2876075744 >> 172.21.15.59:0/2876075744 conn(0x7fc2280ffa50 msgr2=0x7fc228101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.296+0000 7fc230600640 1 -- 172.21.15.59:0/2876075744 shutdown_connections 2024-09-05T11:54:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.296+0000 7fc230600640 1 -- 172.21.15.59:0/2876075744 wait complete. 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 Processor -- start 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 -- start start 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228076720 0x7fc2281a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 0x7fc2281a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 0x7fc22819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc228118610 con 0x7fc228112a50 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc228118490 con 0x7fc228077040 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc230600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc228118790 con 0x7fc228076720 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc22e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 0x7fc22819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc22d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 0x7fc2281a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.297+0000 7fc22e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 0x7fc22819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44300/0 (socket says 172.21.15.59:44300) 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22e600640 1 -- 172.21.15.59:0/4012272698 learned_addr learned my addr 172.21.15.59:0/4012272698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22dc00640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228076720 0x7fc2281a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 -- 172.21.15.59:0/4012272698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228076720 msgr2=0x7fc2281a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228076720 0x7fc2281a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 -- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 msgr2=0x7fc22819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 0x7fc22819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 -- 172.21.15.59:0/4012272698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22819f070 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc22d200640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 0x7fc2281a4cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc214005940 tx=0x7fc2140040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.298+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21403f030 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.299+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2140044c0 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.299+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214035070 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.299+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22819f300 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.299+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281ab650 con 0x7fc228077040 2024-09-05T11:54:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.300+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228077e60 con 0x7fc228077040 2024-09-05T11:54:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.302+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc214042030 con 0x7fc228077040 2024-09-05T11:54:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.303+0000 7fc226000640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 0x7fc2100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:50.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.303+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc2140ef7d0 con 0x7fc228077040 2024-09-05T11:54:50.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.303+0000 7fc22dc00640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 0x7fc2100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:50.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.304+0000 7fc22dc00640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 0x7fc2100a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc22819fa80 tx=0x7fc21c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:50.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.306+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2140bbe40 con 0x7fc228077040 2024-09-05T11:54:50.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.527+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2281ab880 con 0x7fc228077040 2024-09-05T11:54:50.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.528+0000 7fc226000640 1 -- 172.21.15.59:0/4012272698 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7fc2140c0140 con 0x7fc228077040 2024-09-05T11:54:50.530 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:50.530 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"btime":"2024-09-05T11:54:42:610654+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:54:42.509631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:54:50.530 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 msgr2=0x7fc2100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 0x7fc2100a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc22819fa80 tx=0x7fc21c005f90 comp rx=0 tx=0).stop 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 msgr2=0x7fc2281a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 0x7fc2281a4cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc214005940 tx=0x7fc2140040c0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 shutdown_connections 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc2100a2b80 0x7fc2100a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228112a50 0x7fc22819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc228077040 0x7fc2281a4cc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 --2- 172.21.15.59:0/4012272698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc228076720 0x7fc2281a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.530+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 >> 172.21.15.59:0/4012272698 conn(0x7fc2280ffa50 msgr2=0x7fc228101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.531+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 shutdown_connections 2024-09-05T11:54:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:50.531+0000 7fc230600640 1 -- 172.21.15.59:0/4012272698 wait complete. 2024-09-05T11:54:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:50 smithi089 ceph-mon[34499]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:50 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4012272698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:50.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:50 smithi059 ceph-mon[30025]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:50.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:50 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4012272698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:50 smithi189 ceph-mon[34772]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:50 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4012272698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:54:51.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:54:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 -- 172.21.15.59:0/3970008831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 msgr2=0x7fe8781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 --2- 172.21.15.59:0/3970008831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe8781043d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe868009900 tx=0x7fe86802a400 comp rx=0 tx=0).stop 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 -- 172.21.15.59:0/3970008831 shutdown_connections 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 --2- 172.21.15.59:0/3970008831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 0x7fe878110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 --2- 172.21.15.59:0/3970008831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe8781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 --2- 172.21.15.59:0/3970008831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe8781043d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 -- 172.21.15.59:0/3970008831 >> 172.21.15.59:0/3970008831 conn(0x7fe8780ffa90 msgr2=0x7fe878101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 -- 172.21.15.59:0/3970008831 shutdown_connections 2024-09-05T11:54:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.313+0000 7fe87f200640 1 -- 172.21.15.59:0/3970008831 wait complete. 2024-09-05T11:54:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 Processor -- start 2024-09-05T11:54:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 -- start start 2024-09-05T11:54:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe87819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 0x7fe87819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe878114120 con 0x7fe878103ff0 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe878113fa0 con 0x7fe8781049a0 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe87f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781142a0 con 0x7fe878109120 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.315+0000 7fe877e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46784/0 (socket says 172.21.15.59:46784) 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 -- 172.21.15.59:0/3956654715 learned_addr learned my addr 172.21.15.59:0/3956654715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe86fe00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe87819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe87d200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 0x7fe87819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 -- 172.21.15.59:0/3956654715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 msgr2=0x7fe87819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 0x7fe87819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 -- 172.21.15.59:0/3956654715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 msgr2=0x7fe87819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe87819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 -- 172.21.15.59:0/3956654715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87819f5b0 con 0x7fe878103ff0 2024-09-05T11:54:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe86fe00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe87819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.316+0000 7fe877e00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe868006010 tx=0x7fe8680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.317+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86803f030 con 0x7fe878103ff0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.317+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868003ce0 con 0x7fe878103ff0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.317+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87819f840 con 0x7fe878103ff0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.317+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868035070 con 0x7fe878103ff0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.317+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a7180 con 0x7fe878103ff0 2024-09-05T11:54:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.318+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8781043d0 con 0x7fe878103ff0 2024-09-05T11:54:52.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.320+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe868042030 con 0x7fe878103ff0 2024-09-05T11:54:52.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.321+0000 7fe875600640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 0x7fe85c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:52.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.321+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe8680ef7c0 con 0x7fe878103ff0 2024-09-05T11:54:52.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.322+0000 7fe86fe00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 0x7fe85c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:52.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.322+0000 7fe86fe00640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 0x7fe85c0a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe87810aaf0 tx=0x7fe86000b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:52.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.325+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8680bbef0 con 0x7fe878103ff0 2024-09-05T11:54:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.518+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe87810c300 con 0x7fe878103ff0 2024-09-05T11:54:52.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.520+0000 7fe875600640 1 -- 172.21.15.59:0/3956654715 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24202 (secure 0 0 0) 0x7fe8680c01f0 con 0x7fe878103ff0 2024-09-05T11:54:52.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:54:52.522 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:54:42.509492+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:54:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 msgr2=0x7fe85c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 0x7fe85c0a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe87810aaf0 tx=0x7fe86000b390 comp rx=0 tx=0).stop 2024-09-05T11:54:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 msgr2=0x7fe87819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe868006010 tx=0x7fe8680042f0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 shutdown_connections 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe85c0a2c00 0x7fe85c0a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe878109120 0x7fe87819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8781049a0 0x7fe87819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 --2- 172.21.15.59:0/3956654715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe878103ff0 0x7fe87819a600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.522+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 >> 172.21.15.59:0/3956654715 conn(0x7fe8780ffa90 msgr2=0x7fe878100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.523+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 shutdown_connections 2024-09-05T11:54:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:52.523+0000 7fe87f200640 1 -- 172.21.15.59:0/3956654715 wait complete. 2024-09-05T11:54:52.870 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:52 smithi059 ceph-mon[30025]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:52.871 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:52 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3956654715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:52 smithi089 ceph-mon[34499]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:52 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3956654715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:52 smithi189 ceph-mon[34772]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:52 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3956654715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:54:53.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-05T11:54:32.492433+0000', 'modified': '2024-09-05T11:54:42.509631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4050620246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4050620246}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4050620246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24545, 'qdb_cluster': [24545]} 2024-09-05T11:54:53.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T11:54:53.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T11:54:53.597 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.197+0000 7fd7faa00640 1 -- 172.21.15.59:0/1467048156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4077040 msgr2=0x7fd7f41124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.197+0000 7fd7faa00640 1 --2- 172.21.15.59:0/1467048156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4077040 0x7fd7f41124f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009900 tx=0x7fd7dc02a410 comp rx=0 tx=0).stop 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 -- 172.21.15.59:0/1467048156 shutdown_connections 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 --2- 172.21.15.59:0/1467048156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4112a30 0x7fd7f4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 --2- 172.21.15.59:0/1467048156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4077040 0x7fd7f41124f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 --2- 172.21.15.59:0/1467048156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4076720 0x7fd7f4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 -- 172.21.15.59:0/1467048156 >> 172.21.15.59:0/1467048156 conn(0x7fd7f40ffa90 msgr2=0x7fd7f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 -- 172.21.15.59:0/1467048156 shutdown_connections 2024-09-05T11:54:54.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.198+0000 7fd7faa00640 1 -- 172.21.15.59:0/1467048156 wait complete. 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 Processor -- start 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 -- start start 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 0x7fd7f41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4077040 0x7fd7f41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 0x7fd7f419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f41185a0 con 0x7fd7f4112a30 2024-09-05T11:54:54.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4118420 con 0x7fd7f4077040 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.199+0000 7fd7faa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4118720 con 0x7fd7f4076720 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 0x7fd7f419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7ebe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4077040 0x7fd7f41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 0x7fd7f419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46798/0 (socket says 172.21.15.59:46798) 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 0x7fd7f41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f8a00640 1 -- 172.21.15.59:0/2745237500 learned_addr learned my addr 172.21.15.59:0/2745237500 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 -- 172.21.15.59:0/2745237500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4077040 msgr2=0x7fd7f41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4077040 0x7fd7f41a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 -- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 msgr2=0x7fd7f419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 0x7fd7f419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 -- 172.21.15.59:0/2745237500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f419efc0 con 0x7fd7f4076720 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.200+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 0x7fd7f41a46d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9f0 tx=0x7fd7e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.201+0000 7fd7f1600640 1 -- 172.21.15.59:0/2745237500 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400ccc0 con 0x7fd7f4076720 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.201+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f419f2b0 con 0x7fd7f4076720 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.201+0000 7fd7f1600640 1 -- 172.21.15.59:0/2745237500 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7e4010910 con 0x7fd7f4076720 2024-09-05T11:54:54.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.201+0000 7fd7f1600640 1 -- 172.21.15.59:0/2745237500 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4018960 con 0x7fd7f4076720 2024-09-05T11:54:54.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.201+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41ab5a0 con 0x7fd7f4076720 2024-09-05T11:54:54.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7f1600640 1 -- 172.21.15.59:0/2745237500 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd7e4010ac0 con 0x7fd7f4076720 2024-09-05T11:54:54.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 Processor -- start 2024-09-05T11:54:54.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 -- start start 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 0x556f72199710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f721a4510 0x556f721a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f720ff0b0 con 0x556f721a4510 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f720a6ab0 con 0x556f72199330 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.205+0000 7fd7fd601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f720fb070 con 0x556f72199c50 2024-09-05T11:54:54.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 0x556f72199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 0x556f72199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:52062/0 (socket says 172.21.15.59:52062) 2024-09-05T11:54:54.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f8a00640 1 -- 172.21.15.59:0/4052112832 learned_addr learned my addr 172.21.15.59:0/4052112832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:54.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f721a4510 0x556f721a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 -- 172.21.15.59:0/4052112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 msgr2=0x556f72199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 0x556f72199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 -- 172.21.15.59:0/4052112832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f721a4510 msgr2=0x556f721a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f721a4510 0x556f721a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f3e00640 1 -- 172.21.15.59:0/4052112832 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f720b34a0 con 0x556f72199c50 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.206+0000 7fd7f1600640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 0x7fd7d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a3fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40cce70 tx=0x7fd7e40d5bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc28d691c99d8a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7f1600640 1 -- 172.21.15.59:0/2745237500 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd7e40cf670 con 0x7fd7f4076720 2024-09-05T11:54:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 0x7fd7d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7e9200640 1 -- 172.21.15.59:0/4052112832 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4026030 con 0x556f72199c50 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7e9200640 1 -- 172.21.15.59:0/4052112832 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7e40d5e20 con 0x556f72199c50 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.207+0000 7fd7e9200640 1 -- 172.21.15.59:0/4052112832 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e409b820 con 0x556f72199c50 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 0x7fd7d80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec011690 tx=0x7fd7ec009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 -- 172.21.15.59:0/4052112832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 msgr2=0x556f721a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a3fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40cce70 tx=0x7fd7e40d5bd0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 -- 172.21.15.59:0/4052112832 shutdown_connections 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f721a4510 0x556f721a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a3fd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 --2- 172.21.15.59:0/4052112832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199330 0x556f72199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.208+0000 7fd7fd601740 1 -- 172.21.15.59:0/4052112832 >> 172.21.15.59:0/4052112832 conn(0x556f721958c0 msgr2=0x556f72195ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.209+0000 7fd7fd601740 1 -- 172.21.15.59:0/4052112832 shutdown_connections 2024-09-05T11:54:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.209+0000 7fd7fd601740 1 -- 172.21.15.59:0/4052112832 wait complete. 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.209+0000 7fd7fd601740 1 -- start start 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.209+0000 7fd7fd601740 1 Processor -- start 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f721a7060 con 0x556f72199330 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f72087920 con 0x556f721a4510 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f721a71e0 con 0x556f72199c50 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7fd601740 10 client.0 _ll_register_callbacks cb 0x556f720ab8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:50682/0 (socket says 172.21.15.59:50682) 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7ebe00640 1 -- 172.21.15.59:0/716325938 learned_addr learned my addr 172.21.15.59:0/716325938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.210+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 -- 172.21.15.59:0/716325938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 msgr2=0x556f721a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 -- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 msgr2=0x556f721a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 -- 172.21.15.59:0/716325938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f71e702b0 con 0x556f72199330 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721a1580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40980f0 tx=0x7fd7e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fa4eba44a801e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7e1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:54:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e409b640 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd7e40d5220 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.211+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e40d5e20 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd7e409b820 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 10 client.14634 fetch_fsmap learned FSMap version 17 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556f721a7060 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7e40d5e20 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fd7e40a4410 con 0x556f72199330 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 10 client.14634 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 10 client.14634 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 20 client.14634 populate_metadata read hostname 'smithi059' 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f721b4f90 0x556f721b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 4 client.14634 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:54:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.212+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7210c620 con 0x556f721b4f90 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.213+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f721b4f90 0x556f721b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.214+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f721b4f90 0x556f721b6400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1740b8b0da2c8d server_cookie=c215c6bf3f0287c8 in_seq=0 out_seq=0 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.214+0000 7fd7e1600640 10 client.14634 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7e1600640 1 -- 172.21.15.59:0/716325938 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f7210c620 con 0x556f721b4f90 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7e1600640 10 client.14634 handle_command_reply: tid=0 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 client.14634 shutdown 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 msgr2=0x556f721a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721a1580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40980f0 tx=0x7fd7e400ce60 comp rx=0 tx=0).stop 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 shutdown_connections 2024-09-05T11:54:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199c50 0x556f721a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f721b4f90 0x556f721b6400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 --2- 172.21.15.59:0/716325938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721a1580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 >> 172.21.15.59:0/716325938 conn(0x556f721958c0 msgr2=0x556f7219b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7e1600640 0 client.14634 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 shutdown_connections 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.215+0000 7fd7fd601740 1 -- 172.21.15.59:0/716325938 wait complete. 2024-09-05T11:54:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.216+0000 7fd7fd601740 20 client.14634 trim_cache size 0 max 16384 2024-09-05T11:54:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 Processor -- start 2024-09-05T11:54:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 -- start start 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199c50 0x556f7219bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f721b4740 con 0x556f72199330 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f721b4e10 con 0x556f72199c50 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7fd601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f721b54e0 con 0x556f721a4510 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:50688/0 (socket says 172.21.15.59:50688) 2024-09-05T11:54:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.235+0000 7fd7f8a00640 1 -- 172.21.15.59:0/3437423142 learned_addr learned my addr 172.21.15.59:0/3437423142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 -- 172.21.15.59:0/3437423142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199c50 msgr2=0x556f7219bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199c50 0x556f7219bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 -- 172.21.15.59:0/3437423142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 msgr2=0x556f721b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 -- 172.21.15.59:0/3437423142 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f721b52c0 con 0x556f721a4510 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.236+0000 7fd7f8a00640 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00f870 tx=0x7fd7ec01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97ed8004736213b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.237+0000 7fd7e3e00640 1 -- 172.21.15.59:0/3437423142 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00fdf0 con 0x556f721a4510 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.237+0000 7fd7e3e00640 1 -- 172.21.15.59:0/3437423142 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec003e50 con 0x556f721a4510 2024-09-05T11:54:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.237+0000 7fd7e3e00640 1 -- 172.21.15.59:0/3437423142 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0045f0 con 0x556f721a4510 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 -- 172.21.15.59:0/3437423142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 msgr2=0x556f721a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00f870 tx=0x7fd7ec01a000 comp rx=0 tx=0).stop 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 -- 172.21.15.59:0/3437423142 shutdown_connections 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f721a4510 0x556f721a1e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f72199c50 0x556f7219bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 --2- 172.21.15.59:0/3437423142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199330 0x556f721b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 -- 172.21.15.59:0/3437423142 >> 172.21.15.59:0/3437423142 conn(0x556f721958c0 msgr2=0x556f721a3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 -- 172.21.15.59:0/3437423142 shutdown_connections 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.238+0000 7fd7fd601740 1 -- 172.21.15.59:0/3437423142 wait complete. 2024-09-05T11:54:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 -- start start 2024-09-05T11:54:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 Processor -- start 2024-09-05T11:54:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199c50 0x556f721bede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f721b6280 con 0x556f72199c50 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f721b5bb0 con 0x556f721a4510 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f721b6950 con 0x556f72199330 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.239+0000 7fd7fd601740 10 client.0 _ll_register_callbacks cb 0x7fd7e40d55c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199c50 0x556f721bede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:50696/0 (socket says 172.21.15.59:50696) 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:52088/0 (socket says 172.21.15.59:52088) 2024-09-05T11:54:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 -- 172.21.15.59:0/998059107 learned_addr learned my addr 172.21.15.59:0/998059107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 -- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 msgr2=0x556f721b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 -- 172.21.15.59:0/998059107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199c50 msgr2=0x556f721bede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:54:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199c50 0x556f721bede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.240+0000 7fd7f3e00640 1 -- 172.21.15.59:0/998059107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f721b8480 con 0x556f72199330 2024-09-05T11:54:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7f3e00640 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400c6a0 tx=0x7fd7e400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ab29e525eda464 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7e1600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:54:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e409bab0 con 0x556f72199330 2024-09-05T11:54:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd7e40d5200 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ec003e50 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd7e409b640 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7fd601740 10 client.24622 fetch_fsmap learned FSMap version 17 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556f721b8480 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.241+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec003e50 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fd7e40a4410 con 0x556f72199330 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 10 client.24622 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 10 client.24622 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 20 client.24622 populate_metadata read hostname 'smithi059' 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f72223860 0x556f72222570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 4 client.24622 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "status"}) -- 0x556f72106b30 con 0x556f72223860 2024-09-05T11:54:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.242+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f72223860 0x556f72222570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7ebe00640 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f72223860 0x556f72222570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99be7c21ea97b113 server_cookie=cec4026a11e85538 in_seq=0 out_seq=0 2024-09-05T11:54:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7e1600640 10 client.24622 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7e1600640 1 -- 172.21.15.59:0/998059107 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x556f72106b30 con 0x556f72223860 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7e1600640 10 client.24622 handle_command_reply: tid=0 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7fd601740 1 client.24622 shutdown 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 msgr2=0x556f721be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.243+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400c6a0 tx=0x7fd7e400cf60 comp rx=0 tx=0).stop 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 shutdown_connections 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f721a4510 0x556f721b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556f72223860 0x556f72222570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f72199c50 0x556f721bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 --2- 172.21.15.59:0/998059107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f72199330 0x556f721be8a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 >> 172.21.15.59:0/998059107 conn(0x556f721958c0 msgr2=0x556f72194200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7e1600640 0 client.24622 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 shutdown_connections 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 1 -- 172.21.15.59:0/998059107 wait complete. 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.244+0000 7fd7fd601740 20 client.24622 trim_cache size 0 max 16384 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "53d560d4-6b7c-11ef-bcd6-c7b262605968", 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24545, 2024-09-05T11:54:54.248 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 21.697489937, 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 17, 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 89, 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 34.392064763 2024-09-05T11:54:54.249 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 msgr2=0x7fd7d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 0x7fd7d80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec011690 tx=0x7fd7ec009500 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 msgr2=0x7fd7f41a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 0x7fd7f41a46d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9f0 tx=0x7fd7e400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 shutdown_connections 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd7d80a2b80 0x7fd7d80a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7f4112a30 0x7fd7f419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f4077040 0x7fd7f41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 --2- 172.21.15.59:0/2745237500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7f4076720 0x7fd7f41a46d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 >> 172.21.15.59:0/2745237500 conn(0x7fd7f40ffa90 msgr2=0x7fd7f4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 shutdown_connections 2024-09-05T11:54:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:54.245+0000 7fd7faa00640 1 -- 172.21.15.59:0/2745237500 wait complete. 2024-09-05T11:54:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:54 smithi089 ceph-mon[34499]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:54.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:54 smithi059 ceph-mon[30025]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:54 smithi189 ceph-mon[34772]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:55.122 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T11:54:55.122 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:54:55.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T11:54:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:54:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 -- 172.21.15.59:0/521409753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 msgr2=0x7f50f4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 --2- 172.21.15.59:0/521409753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f4074a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009860 tx=0x7f50e402a3d0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 -- 172.21.15.59:0/521409753 shutdown_connections 2024-09-05T11:54:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 --2- 172.21.15.59:0/521409753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 0x7f50f4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 --2- 172.21.15.59:0/521409753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 --2- 172.21.15.59:0/521409753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f4074a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 -- 172.21.15.59:0/521409753 >> 172.21.15.59:0/521409753 conn(0x7f50f40fd8c0 msgr2=0x7f50f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 -- 172.21.15.59:0/521409753 shutdown_connections 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.023+0000 7f50fa600640 1 -- 172.21.15.59:0/521409753 wait complete. 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 Processor -- start 2024-09-05T11:54:56.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 -- start start 2024-09-05T11:54:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4114210 con 0x7f50f4074f40 2024-09-05T11:54:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4114090 con 0x7f50f4074620 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50fa600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4114390 con 0x7f50f410e3c0 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.024+0000 7f50f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46864/0 (socket says 172.21.15.59:46864) 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 -- 172.21.15.59:0/1884558121 learned_addr learned my addr 172.21.15.59:0/1884558121 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f3400640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 -- 172.21.15.59:0/1884558121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 msgr2=0x7f50f419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 -- 172.21.15.59:0/1884558121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 msgr2=0x7f50f419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 -- 172.21.15.59:0/1884558121 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f419f660 con 0x7f50f4074f40 2024-09-05T11:54:56.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.025+0000 7f50f2a00640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50e000e9c0 tx=0x7f50e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.026+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e000cde0 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.026+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e00035f0 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.026+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0010690 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.026+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f419f950 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.026+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a7230 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.027+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4074a00 con 0x7f50f4074f40 2024-09-05T11:54:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.029+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f50e00037a0 con 0x7f50f4074f40 2024-09-05T11:54:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.030+0000 7f50ebe00640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 0x7f50d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:54:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.030+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f50e00cf560 con 0x7f50f4074f40 2024-09-05T11:54:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.030+0000 7f50f3400640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 0x7f50d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:54:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.031+0000 7f50f3400640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 0x7f50d80a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50e402a9c0 tx=0x7f50e4005c90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:54:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.035+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50e009c220 con 0x7f50f4074f40 2024-09-05T11:54:56.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.258+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50f4108fb0 con 0x7f50f4074f40 2024-09-05T11:54:56.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.259+0000 7f50ebe00640 1 -- 172.21.15.59:0/1884558121 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f50e009bb20 con 0x7f50f4074f40 2024-09-05T11:54:56.261 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 msgr2=0x7f50d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 0x7f50d80a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50e402a9c0 tx=0x7f50e4005c90 comp rx=0 tx=0).stop 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 msgr2=0x7f50f419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50e000e9c0 tx=0x7f50e000ee80 comp rx=0 tx=0).stop 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 shutdown_connections 2024-09-05T11:54:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f50d80a2c00 0x7f50d80a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f4074f40 0x7f50f419abf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 --2- 172.21.15.59:0/1884558121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f4074620 0x7f50f419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.261+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 >> 172.21.15.59:0/1884558121 conn(0x7f50f40fd8c0 msgr2=0x7f50f40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.262+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 shutdown_connections 2024-09-05T11:54:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:56.262+0000 7f50fa600640 1 -- 172.21.15.59:0/1884558121 wait complete. 2024-09-05T11:54:56.909 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:56 smithi189 ceph-mon[34772]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:56.909 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:56 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1884558121' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:54:56.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:56 smithi059 ceph-mon[30025]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:56.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:56 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1884558121' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:54:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:56 smithi089 ceph-mon[34499]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:56 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1884558121' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:54:56.971 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T11:54:56.972 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T11:54:56.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:56.972 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-09-05T11:54:57.009 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42202sec preferred_lft 42202sec 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-09-05T11:54:57.010 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-09-05T11:54:57.011 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T11:54:57.011 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:54:57.011 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:54:57.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-09-05T11:54:57.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-09-05T11:54:57.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-09-05T11:54:57.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T11:54:57.012 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:54:57.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:54:57.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:54:57.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:57.441 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T11:54:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-09-05T11:54:57.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:57.502 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-09-05T11:54:57.551 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-09-05T11:54:57.551 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T11:54:57.551 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-05T11:54:57.551 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-05T11:54:57.552 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-05T11:54:57.552 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:54:57.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:54:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:54:58.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:58.126 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-09-05T11:54:58.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:58.162 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-09-05T11:54:58.237 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:54:58.238 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:54:58.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:54:58.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-05T11:54:58.238 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:54:58.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:54:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:54:58.416 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-05T11:54:58.416 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-05T11:54:58.417 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:54:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:54:58.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:54:58.659 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:54:58.660 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:54:58.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-09-05T11:54:58.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T11:54:58.660 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:54:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:54:58.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:54:58.827 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-05T11:54:58.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T11:54:58.827 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:54:58.862 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T11:54:58.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T11:54:58.863 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:54:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:54:58 smithi089 ceph-mon[34499]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:58.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-09-05T11:54:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:54:58 smithi059 ceph-mon[30025]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:59.014 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-09-05T11:54:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-09-05T11:54:59.051 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/9026d2c9bce793347adbac917cbde76d51ac83d1667640b2945bdab14f0c53b1/merged 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/548f94dd56d9b11acaf9a8d58089d1d40768e1918a8eeb0c0cdbc08986322546/merged 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/b8d1318a347b42bb5939446ad7de3b13e7315ccb658b444898375f2e3d8eb9d8/merged 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/c0837e7f196d1f7f90dbc35df002a47ab5bf2bf4c38025b1f23b72ede102d0f8/merged 2024-09-05T11:54:59.052 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/df666602ab789deff96f5873f464f039e72b62ec118d33ae8ee18d17871b0347/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/3405ae623dfb54bb207886e7e729e012708c58194133ac4e3ca92248e76ee218/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/59bd1b168e8525ff25091bb30099edb9d6f4d02c083985e987390541e798140e/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/6a4de2d9495f9fb344fcbd543a2428c61beff77be274cddbc3080c4bbc353e6a/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/1aeb084a9b891dfe6ede2e27b0ab938faa6d951081140ce0960c5789b055a00b/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/7bc6ef4f3332e7bb9157ed1632886a85d7afdef2017102b8a875368e2fd4e5bf/merged 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:54:59.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:54:59.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:59.054 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-09-05T11:54:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:54:58 smithi189 ceph-mon[34772]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:54:59.129 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T11:54:59.129 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-05T11:54:59.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-09-05T11:54:59.230 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-09-05T11:54:59.267 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T11:54:59.268 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/9026d2c9bce793347adbac917cbde76d51ac83d1667640b2945bdab14f0c53b1/merged 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/548f94dd56d9b11acaf9a8d58089d1d40768e1918a8eeb0c0cdbc08986322546/merged 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/b8d1318a347b42bb5939446ad7de3b13e7315ccb658b444898375f2e3d8eb9d8/merged 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/c0837e7f196d1f7f90dbc35df002a47ab5bf2bf4c38025b1f23b72ede102d0f8/merged 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/df666602ab789deff96f5873f464f039e72b62ec118d33ae8ee18d17871b0347/merged 2024-09-05T11:54:59.269 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/3405ae623dfb54bb207886e7e729e012708c58194133ac4e3ca92248e76ee218/merged 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/59bd1b168e8525ff25091bb30099edb9d6f4d02c083985e987390541e798140e/merged 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/6a4de2d9495f9fb344fcbd543a2428c61beff77be274cddbc3080c4bbc353e6a/merged 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/1aeb084a9b891dfe6ede2e27b0ab938faa6d951081140ce0960c5789b055a00b/merged 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/7bc6ef4f3332e7bb9157ed1632886a85d7afdef2017102b8a875368e2fd4e5bf/merged 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:54:59.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:54:59.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:54:59.271 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-09-05T11:54:59.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:2024-09-05T11:54:59.348+0000 7fdc80b42480 -1 init, newargv = 0x55bcf59e33d0 newargc=15 2024-09-05T11:54:59.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[74164]: starting ceph client 2024-09-05T11:54:59.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[74164]: starting fuse 2024-09-05T11:55:00.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-09-05T11:55:00.514 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T11:55:00.552 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-09-05T11:55:00.552 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-09-05T11:55:00.552 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-09-05T11:55:00.552 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-09-05T11:55:00.553 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/9026d2c9bce793347adbac917cbde76d51ac83d1667640b2945bdab14f0c53b1/merged 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/548f94dd56d9b11acaf9a8d58089d1d40768e1918a8eeb0c0cdbc08986322546/merged 2024-09-05T11:55:00.554 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/b8d1318a347b42bb5939446ad7de3b13e7315ccb658b444898375f2e3d8eb9d8/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/c0837e7f196d1f7f90dbc35df002a47ab5bf2bf4c38025b1f23b72ede102d0f8/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/df666602ab789deff96f5873f464f039e72b62ec118d33ae8ee18d17871b0347/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/3405ae623dfb54bb207886e7e729e012708c58194133ac4e3ca92248e76ee218/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/59bd1b168e8525ff25091bb30099edb9d6f4d02c083985e987390541e798140e/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/6a4de2d9495f9fb344fcbd543a2428c61beff77be274cddbc3080c4bbc353e6a/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/1aeb084a9b891dfe6ede2e27b0ab938faa6d951081140ce0960c5789b055a00b/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/7bc6ef4f3332e7bb9157ed1632886a85d7afdef2017102b8a875368e2fd4e5bf/merged 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:55:00.555 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-05T11:55:00.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:00.556 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-09-05T11:55:00.633 INFO:teuthology.orchestra.run.smithi059.stdout:120 2024-09-05T11:55:00.633 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> 2024-09-05T11:55:00.633 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-09-05T11:55:00.634 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-09-05T11:55:00.635 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T11:55:00.635 DEBUG:teuthology.orchestra.run.smithi059:> 2024-09-05T11:55:00.635 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.admin")) 2024-09-05T11:55:00.635 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-09-05T11:55:00.799 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.admin.74164.asok 2024-09-05T11:55:00.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:55:00.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:00 smithi059 ceph-mon[30025]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:55:00.814 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74164.asok 2024-09-05T11:55:00.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:00.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74164.asok status 2024-09-05T11:55:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:00 smithi089 ceph-mon[34499]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "e856d6b2462509a377ef39149a05475ed006ebac", 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "admin", 2024-09-05T11:55:01.002 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "74164", 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 14652, 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-09-05T11:55:01.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 14652 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-09-05T11:55:01.004 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1481118589 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1481118589 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.14652 192.168.0.1:0/1481118589", 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1481118589", 2024-09-05T11:55:01.005 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 17, 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 89, 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-09-05T11:55:01.006 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:55:01.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:01.014 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-09-05T11:55:01.014 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:55:01.050 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-09-05T11:55:01.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:55:01.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:01.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:55:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:00 smithi189 ceph-mon[34772]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T11:55:01.150 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-05T11:55:01.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T11:55:01.253 INFO:teuthology.orchestra.run.smithi059.stdout:# file: . 2024-09-05T11:55:01.254 INFO:teuthology.orchestra.run.smithi059.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-05T11:55:01.254 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:01.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:55:01.262 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:55:01.262 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T11:55:01.299 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T11:55:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:02.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 -- 172.21.15.59:0/4230785121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20112a30 msgr2=0x7f7c20114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 --2- 172.21.15.59:0/4230785121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20112a30 0x7f7c20114e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009ca0 tx=0x7f7c1802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 -- 172.21.15.59:0/4230785121 shutdown_connections 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 --2- 172.21.15.59:0/4230785121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20112a30 0x7f7c20114e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 --2- 172.21.15.59:0/4230785121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 0x7f7c201124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 --2- 172.21.15.59:0/4230785121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20076720 0x7f7c20076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.413+0000 7f7c26800640 1 -- 172.21.15.59:0/4230785121 >> 172.21.15.59:0/4230785121 conn(0x7f7c200ffa90 msgr2=0x7f7c20101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.414+0000 7f7c26800640 1 -- 172.21.15.59:0/4230785121 shutdown_connections 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.414+0000 7f7c26800640 1 -- 172.21.15.59:0/4230785121 wait complete. 2024-09-05T11:55:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.414+0000 7f7c26800640 1 Processor -- start 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 -- start start 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 0x7f7c201a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20112a30 0x7f7c2019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201185c0 con 0x7f7c20076720 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20118440 con 0x7f7c20077040 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c26800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20118740 con 0x7f7c20112a30 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49326/0 (socket says 172.21.15.59:49326) 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c1f400640 1 -- 172.21.15.59:0/2532955021 learned_addr learned my addr 172.21.15.59:0/2532955021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c1fe00640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20112a30 0x7f7c2019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.415+0000 7f7c1ea00640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 0x7f7c201a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 -- 172.21.15.59:0/2532955021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20112a30 msgr2=0x7f7c2019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20112a30 0x7f7c2019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 -- 172.21.15.59:0/2532955021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 msgr2=0x7f7c201a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 0x7f7c201a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 -- 172.21.15.59:0/2532955021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2019ef90 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c1f400640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000d950 tx=0x7f7c1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10017070 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2019f280 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c10009c30 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000f6b0 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.416+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201ab570 con 0x7f7c20076720 2024-09-05T11:55:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.418+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20076b00 con 0x7f7c20076720 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.419+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c100109d0 con 0x7f7c20076720 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.422+0000 7f7c17e00640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 0x7f7c040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.422+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7c100ceea0 con 0x7f7c20076720 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.422+0000 7f7c1ea00640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 0x7f7c040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.423+0000 7f7c1ea00640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 0x7f7c040a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c200792b0 tx=0x7f7c0c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.426+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c10009de0 con 0x7f7c20076720 2024-09-05T11:55:02.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.628+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c2019fd90 con 0x7f7c20076720 2024-09-05T11:55:02.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.629+0000 7f7c17e00640 1 -- 172.21.15.59:0/2532955021 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7f7c1009b510 con 0x7f7c20076720 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3566679520 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1131370065 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1895165557 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3602565252 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1380942177 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:02.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1592551985 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 msgr2=0x7f7c040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 0x7f7c040a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c200792b0 tx=0x7f7c0c005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 msgr2=0x7f7c201a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000d950 tx=0x7f7c1000de10 comp rx=0 tx=0).stop 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 shutdown_connections 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c040a2b80 0x7f7c040a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c20112a30 0x7f7c2019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c20077040 0x7f7c201a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 --2- 172.21.15.59:0/2532955021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c20076720 0x7f7c201a46a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.631+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 >> 172.21.15.59:0/2532955021 conn(0x7f7c200ffa90 msgr2=0x7f7c20101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.632+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 shutdown_connections 2024-09-05T11:55:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:02.632+0000 7f7c26800640 1 -- 172.21.15.59:0/2532955021 wait complete. 2024-09-05T11:55:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:02 smithi089 ceph-mon[34499]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2532955021' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:02 smithi059 ceph-mon[30025]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2532955021' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:02 smithi189 ceph-mon[34772]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2532955021' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:03.484 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:55:03.484 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T11:55:03.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T11:55:03.941 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 -- 172.21.15.59:0/2624038435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40241019c0 msgr2=0x7f4024101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 --2- 172.21.15.59:0/2624038435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40241019c0 0x7f4024101e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f400c009930 tx=0x7f400c02a420 comp rx=0 tx=0).stop 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 -- 172.21.15.59:0/2624038435 shutdown_connections 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 --2- 172.21.15.59:0/2624038435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4024102360 0x7f402410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 --2- 172.21.15.59:0/2624038435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40241019c0 0x7f4024101e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.503+0000 7f4023200640 1 --2- 172.21.15.59:0/2624038435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 0x7f4024107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.504+0000 7f4023200640 1 -- 172.21.15.59:0/2624038435 >> 172.21.15.59:0/2624038435 conn(0x7f40240fd740 msgr2=0x7f40240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.504+0000 7f4023200640 1 -- 172.21.15.59:0/2624038435 shutdown_connections 2024-09-05T11:55:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.504+0000 7f4023200640 1 -- 172.21.15.59:0/2624038435 wait complete. 2024-09-05T11:55:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 Processor -- start 2024-09-05T11:55:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 -- start start 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 0x7f402419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 0x7f402419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4024111dc0 con 0x7f4024102360 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4024111c40 con 0x7f40241019c0 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.505+0000 7f4023200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4024111f40 con 0x7f40241079c0 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f401be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 0x7f402419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f401be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 0x7f402419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49340/0 (socket says 172.21.15.59:49340) 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:41410/0 (socket says 172.21.15.59:41410) 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f401be00640 1 -- 172.21.15.59:0/3428873885 learned_addr learned my addr 172.21.15.59:0/3428873885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4022800640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 0x7f402419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 -- 172.21.15.59:0/3428873885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 msgr2=0x7f402419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 0x7f402419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 -- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 msgr2=0x7f402419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 0x7f402419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 -- 172.21.15.59:0/3428873885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402419f4b0 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.506+0000 7f4021e00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f401400e9f0 tx=0x7f401400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.507+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401400ccc0 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.507+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402419f7a0 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.507+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4014005020 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.507+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40140049e0 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.507+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40241a7080 con 0x7f40241019c0 2024-09-05T11:55:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.508+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4024101e20 con 0x7f40241019c0 2024-09-05T11:55:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.510+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4014005560 con 0x7f40241019c0 2024-09-05T11:55:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.511+0000 7f401aa00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 0x7f40080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.511+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f40140cf130 con 0x7f40241019c0 2024-09-05T11:55:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.511+0000 7f401be00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 0x7f40080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.512+0000 7f401be00640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 0x7f40080a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f402410eb10 tx=0x7f400c009560 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.513+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40140051d0 con 0x7f40241019c0 2024-09-05T11:55:04.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.714+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f402419b860 con 0x7f40241019c0 2024-09-05T11:55:04.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.715+0000 7f401aa00640 1 -- 172.21.15.59:0/3428873885 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7f402419b860 con 0x7f40241019c0 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3566679520 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1131370065 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1895165557 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3602565252 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1380942177 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T11:55:04.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1592551985 2024-09-06T11:46:49.566628+0000 2024-09-05T11:55:04.718 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-09-05T11:55:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 msgr2=0x7f40080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 0x7f40080a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f402410eb10 tx=0x7f400c009560 comp rx=0 tx=0).stop 2024-09-05T11:55:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 msgr2=0x7f402419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f401400e9f0 tx=0x7f401400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 shutdown_connections 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40080a2c00 0x7f40080a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40241079c0 0x7f402419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4024102360 0x7f402419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 --2- 172.21.15.59:0/3428873885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40241019c0 0x7f402419a500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.717+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 >> 172.21.15.59:0/3428873885 conn(0x7f40240fd740 msgr2=0x7f402410d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.718+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 shutdown_connections 2024-09-05T11:55:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:04.718+0000 7f4023200640 1 -- 172.21.15.59:0/3428873885 wait complete. 2024-09-05T11:55:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:04 smithi089 ceph-mon[34499]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:04 smithi059 ceph-mon[30025]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:04 smithi189 ceph-mon[34772]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:05.581 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-09-05T11:55:05.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:05.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-05T11:55:05.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[74164]: fuse finished with error 0 and tester_r 0 2024-09-05T11:55:05.642 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T11:55:05.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T11:55:05.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:55:05.728 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-05T11:55:05.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:55:05.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:05 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3428873885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:05 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3428873885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:06.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:05 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3428873885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T11:55:06.159 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 -- 172.21.15.59:0/2903882772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d81049a0 msgr2=0x7f84d81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 --2- 172.21.15.59:0/2903882772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d81049a0 0x7f84d81089f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009900 tx=0x7f84c402a410 comp rx=0 tx=0).stop 2024-09-05T11:55:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 -- 172.21.15.59:0/2903882772 shutdown_connections 2024-09-05T11:55:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 --2- 172.21.15.59:0/2903882772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8109120 0x7f84d8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 --2- 172.21.15.59:0/2903882772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d81049a0 0x7f84d81089f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 --2- 172.21.15.59:0/2903882772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d8103ff0 0x7f84d81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 -- 172.21.15.59:0/2903882772 >> 172.21.15.59:0/2903882772 conn(0x7f84d80ffa90 msgr2=0x7f84d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 -- 172.21.15.59:0/2903882772 shutdown_connections 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.708+0000 7f84dd800640 1 -- 172.21.15.59:0/2903882772 wait complete. 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.709+0000 7f84dd800640 1 Processor -- start 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.709+0000 7f84dd800640 1 -- start start 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 0x7f84d819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d81049a0 0x7f84d819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:06.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8114140 con 0x7f84d8109120 2024-09-05T11:55:06.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8113fc0 con 0x7f84d8103ff0 2024-09-05T11:55:06.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84dd800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81142c0 con 0x7f84d81049a0 2024-09-05T11:55:06.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:06.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 0x7f84d819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:06.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49346/0 (socket says 172.21.15.59:49346) 2024-09-05T11:55:06.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d7400640 1 -- 172.21.15.59:0/3143875384 learned_addr learned my addr 172.21.15.59:0/3143875384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:06.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d6000640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d81049a0 0x7f84d819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d6a00640 1 -- 172.21.15.59:0/3143875384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d81049a0 msgr2=0x7f84d819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.710+0000 7f84d6a00640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d81049a0 0x7f84d819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84d6a00640 1 -- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 msgr2=0x7f84d819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84d6a00640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84d6a00640 1 -- 172.21.15.59:0/3143875384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819f5d0 con 0x7f84d8103ff0 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84d7400640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:06.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84d6a00640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 0x7f84d819a620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e9f0 tx=0x7f84c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c800ccc0 con 0x7f84d8103ff0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c8005020 con 0x7f84d8103ff0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c80049e0 con 0x7f84d8103ff0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819f8c0 con 0x7f84d8103ff0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.711+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a71a0 con 0x7f84d8103ff0 2024-09-05T11:55:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.712+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d81043d0 con 0x7f84d8103ff0 2024-09-05T11:55:06.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.714+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f84c8005560 con 0x7f84d8103ff0 2024-09-05T11:55:06.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.716+0000 7f84cf400640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 0x7f84bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:06.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.716+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f84c80cf240 con 0x7f84d8103ff0 2024-09-05T11:55:06.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.716+0000 7f84d6000640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 0x7f84bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.716+0000 7f84d6000640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 0x7f84bc0a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009230 tx=0x7f84c40058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:06.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.717+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84c80051d0 con 0x7f84d8103ff0 2024-09-05T11:55:06.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.903+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84d810c3c0 con 0x7f84d8103ff0 2024-09-05T11:55:06.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:06 smithi059 ceph-mon[30025]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:06.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.905+0000 7f84cf400640 1 -- 172.21.15.59:0/3143875384 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24202 (secure 0 0 0) 0x7f84c809bf90 con 0x7f84d8103ff0 2024-09-05T11:55:06.907 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:06.907 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:54:42.509492+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:55:06.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 msgr2=0x7f84bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:06.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 0x7f84bc0a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009230 tx=0x7f84c40058d0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 msgr2=0x7f84d819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 0x7f84d819a620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e9f0 tx=0x7f84c800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 shutdown_connections 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84bc0a2c00 0x7f84bc0a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d8109120 0x7f84d819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84d81049a0 0x7f84d819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 --2- 172.21.15.59:0/3143875384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84d8103ff0 0x7f84d819a620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.907+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 >> 172.21.15.59:0/3143875384 conn(0x7f84d80ffa90 msgr2=0x7f84d8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.908+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 shutdown_connections 2024-09-05T11:55:06.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:06.908+0000 7f84dd800640 1 -- 172.21.15.59:0/3143875384 wait complete. 2024-09-05T11:55:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:06 smithi089 ceph-mon[34499]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:06 smithi189 ceph-mon[34772]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-05T11:55:07.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:55:07.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:07 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3143875384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:08.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:07 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3143875384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:07 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3143875384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:08.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.643+0000 7f1cf7400640 1 -- 172.21.15.59:0/804417308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0104010 msgr2=0x7f1cf01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:08.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 --2- 172.21.15.59:0/804417308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0104010 0x7f1cf01043f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009ca0 tx=0x7f1ce402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 -- 172.21.15.59:0/804417308 shutdown_connections 2024-09-05T11:55:08.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 --2- 172.21.15.59:0/804417308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0109140 0x7f1cf0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 --2- 172.21.15.59:0/804417308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 0x7f1cf0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 --2- 172.21.15.59:0/804417308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0104010 0x7f1cf01043f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 -- 172.21.15.59:0/804417308 >> 172.21.15.59:0/804417308 conn(0x7f1cf00ffa50 msgr2=0x7f1cf0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 -- 172.21.15.59:0/804417308 shutdown_connections 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.644+0000 7f1cf7400640 1 -- 172.21.15.59:0/804417308 wait complete. 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 Processor -- start 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 -- start start 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0104010 0x7f1cf019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 0x7f1cf019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0114140 con 0x7f1cf0109140 2024-09-05T11:55:08.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0113fc0 con 0x7f1cf01049c0 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.645+0000 7f1cf7400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01142c0 con 0x7f1cf0104010 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49366/0 (socket says 172.21.15.59:49366) 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 -- 172.21.15.59:0/387241543 learned_addr learned my addr 172.21.15.59:0/387241543 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cefe00640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 0x7f1cf019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf4a00640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0104010 0x7f1cf019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 -- 172.21.15.59:0/387241543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0104010 msgr2=0x7f1cf019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0104010 0x7f1cf019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 -- 172.21.15.59:0/387241543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 msgr2=0x7f1cf019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 0x7f1cf019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 -- 172.21.15.59:0/387241543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf019f5f0 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.646+0000 7f1cf5400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800d950 tx=0x7f1ce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.647+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017030 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.647+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf019f8e0 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.647+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8004020 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.647+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8003c00 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.647+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf01a71c0 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.648+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf01043f0 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.650+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1ce801a070 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.651+0000 7f1ced600640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 0x7f1cd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.651+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1ce80cea80 con 0x7f1cf0109140 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.651+0000 7f1cf4a00640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 0x7f1cd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:08.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.652+0000 7f1cf4a00640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 0x7f1cd80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4006010 tx=0x7f1ce4009360 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:08.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.655+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ce80d0e20 con 0x7f1cf0109140 2024-09-05T11:55:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.847+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cf010bd90 con 0x7f1cf0109140 2024-09-05T11:55:08.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:08 smithi059 ceph-mon[30025]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.849+0000 7f1ced600640 1 -- 172.21.15.59:0/387241543 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24202 (secure 0 0 0) 0x7f1ce8021070 con 0x7f1cf0109140 2024-09-05T11:55:08.851 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:08.851 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:54:42.509492+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 msgr2=0x7f1cd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 0x7f1cd80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4006010 tx=0x7f1ce4009360 comp rx=0 tx=0).stop 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 msgr2=0x7f1cf019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800d950 tx=0x7f1ce800de10 comp rx=0 tx=0).stop 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 shutdown_connections 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cd80a2b80 0x7f1cd80a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf0109140 0x7f1cf019ef10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cf01049c0 0x7f1cf019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 --2- 172.21.15.59:0/387241543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cf0104010 0x7f1cf019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.851+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 >> 172.21.15.59:0/387241543 conn(0x7f1cf00ffa50 msgr2=0x7f1cf0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.852+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 shutdown_connections 2024-09-05T11:55:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:08.852+0000 7f1cf7400640 1 -- 172.21.15.59:0/387241543 wait complete. 2024-09-05T11:55:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:08 smithi189 ceph-mon[34772]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:08 smithi089 ceph-mon[34499]: pgmap v314: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:09.548 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-05T11:55:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:09.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:09 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/387241543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:10.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:09 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/387241543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:10.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:09 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/387241543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:10.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 -- 172.21.15.59:0/1173591496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0550109120 msgr2=0x7f0550110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:10.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 --2- 172.21.15.59:0/1173591496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0550109120 0x7f0550110ca0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0548009ca0 tx=0x7f054802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 -- 172.21.15.59:0/1173591496 shutdown_connections 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 --2- 172.21.15.59:0/1173591496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0550109120 0x7f0550110ca0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 --2- 172.21.15.59:0/1173591496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05501049a0 0x7f05501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 --2- 172.21.15.59:0/1173591496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f05501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 -- 172.21.15.59:0/1173591496 >> 172.21.15.59:0/1173591496 conn(0x7f05500ffa90 msgr2=0x7f0550101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 -- 172.21.15.59:0/1173591496 shutdown_connections 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.428+0000 7f0557a00640 1 -- 172.21.15.59:0/1173591496 wait complete. 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.429+0000 7f0557a00640 1 Processor -- start 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.429+0000 7f0557a00640 1 -- start start 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f055019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0550109120 0x7f055019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0550114130 con 0x7f05501049a0 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0550113fb0 con 0x7f0550103ff0 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0557a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05501142b0 con 0x7f0550109120 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49384/0 (socket says 172.21.15.59:49384) 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 -- 172.21.15.59:0/4001363346 learned_addr learned my addr 172.21.15.59:0/4001363346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0555000640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f055019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0555a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0550109120 0x7f055019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 -- 172.21.15.59:0/4001363346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0550109120 msgr2=0x7f055019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0550109120 0x7f055019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 -- 172.21.15.59:0/4001363346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 msgr2=0x7f055019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f055019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0547e00640 1 -- 172.21.15.59:0/4001363346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055019f540 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.430+0000 7f0555000640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f055019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f0547e00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f053800d950 tx=0x7f053800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538017030 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0538004020 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055019f830 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380047a0 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.431+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a7110 con 0x7f05501049a0 2024-09-05T11:55:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.432+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05501043d0 con 0x7f05501049a0 2024-09-05T11:55:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.434+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05380041d0 con 0x7f05501049a0 2024-09-05T11:55:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.435+0000 7f054e000640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 0x7f05340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.436+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f05380cebc0 con 0x7f05501049a0 2024-09-05T11:55:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.436+0000 7f0555000640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 0x7f05340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.436+0000 7f0555000640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 0x7f05340a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fc0 tx=0x7f0540005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:10.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.440+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f053809b880 con 0x7f05501049a0 2024-09-05T11:55:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.666+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f05501a73b0 con 0x7f05501049a0 2024-09-05T11:55:10.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.700+0000 7f054e000640 1 -- 172.21.15.59:0/4001363346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f053809b180 con 0x7f05501049a0 2024-09-05T11:55:10.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 msgr2=0x7f05340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 0x7f05340a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fc0 tx=0x7f0540005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 msgr2=0x7f055019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f053800d950 tx=0x7f053800de10 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 shutdown_connections 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f05340a2b80 0x7f05340a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0550109120 0x7f055019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05501049a0 0x7f055019aad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 --2- 172.21.15.59:0/4001363346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0550103ff0 0x7f055019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 >> 172.21.15.59:0/4001363346 conn(0x7f05500ffa90 msgr2=0x7f0550101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:10.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 shutdown_connections 2024-09-05T11:55:10.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:10.703+0000 7f0557a00640 1 -- 172.21.15.59:0/4001363346 wait complete. 2024-09-05T11:55:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:10 smithi059 ceph-mon[30025]: pgmap v315: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:10 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T11:55:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:10 smithi189 ceph-mon[34772]: pgmap v315: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:10 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T11:55:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:10 smithi089 ceph-mon[34499]: pgmap v315: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T11:55:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:10 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T11:55:11.457 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T11:55:11.458 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-05T11:55:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:11.980 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T11:55:11.981 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:11 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:12.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:11 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T11:55:12.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:11 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4001363346' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T11:55:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:11 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 -- 172.21.15.59:0/4230716026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8380745c0 msgr2=0x7fc8380749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 --2- 172.21.15.59:0/4230716026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8380745c0 0x7fc8380749a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc828009900 tx=0x7fc82802a480 comp rx=0 tx=0).stop 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 -- 172.21.15.59:0/4230716026 shutdown_connections 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 --2- 172.21.15.59:0/4230716026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 0x7fc838110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 --2- 172.21.15.59:0/4230716026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc838074ee0 0x7fc83810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 --2- 172.21.15.59:0/4230716026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8380745c0 0x7fc8380749a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 -- 172.21.15.59:0/4230716026 >> 172.21.15.59:0/4230716026 conn(0x7fc8380fd8c0 msgr2=0x7fc8380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 -- 172.21.15.59:0/4230716026 shutdown_connections 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.404+0000 7fc83f800640 1 -- 172.21.15.59:0/4230716026 wait complete. 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.405+0000 7fc83f800640 1 Processor -- start 2024-09-05T11:55:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.405+0000 7fc83f800640 1 -- start start 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.405+0000 7fc83f800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 0x7fc83819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.405+0000 7fc83f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83f800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 0x7fc83819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc838114190 con 0x7fc838074ee0 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83f800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc838114010 con 0x7fc83810e420 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83f800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc838114310 con 0x7fc8380745c0 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38554/0 (socket says 172.21.15.59:38554) 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 -- 172.21.15.59:0/1304254315 learned_addr learned my addr 172.21.15.59:0/1304254315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83ce00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 0x7fc83819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83d800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 0x7fc83819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 -- 172.21.15.59:0/1304254315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 msgr2=0x7fc83819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 0x7fc83819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 -- 172.21.15.59:0/1304254315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 msgr2=0x7fc83819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 0x7fc83819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc82fe00640 1 -- 172.21.15.59:0/1304254315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83819f660 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.406+0000 7fc83ce00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 0x7fc83819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc82fe00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e9f0 tx=0x7fc82000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82000cd80 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc820005500 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820010690 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83819f950 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.407+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a7230 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.408+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8380749a0 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.410+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8200046c0 con 0x7fc838074ee0 2024-09-05T11:55:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.411+0000 7fc836000640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 0x7fc81c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:12.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.412+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc8200cf660 con 0x7fc838074ee0 2024-09-05T11:55:12.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.412+0000 7fc83ce00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 0x7fc81c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:12.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.413+0000 7fc83ce00640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 0x7fc81c0a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc828009230 tx=0x7fc8280058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:12.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.416+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc82009c320 con 0x7fc838074ee0 2024-09-05T11:55:12.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.640+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc83802d7c0 con 0x7fc838074ee0 2024-09-05T11:55:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.709+0000 7fc836000640 1 -- 172.21.15.59:0/1304254315 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fc820086020 con 0x7fc838074ee0 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 msgr2=0x7fc81c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 0x7fc81c0a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc828009230 tx=0x7fc8280058d0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 msgr2=0x7fc83819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e9f0 tx=0x7fc82000edf0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 shutdown_connections 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc81c0a2c00 0x7fc81c0a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83810e420 0x7fc83819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc838074ee0 0x7fc83819abf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 --2- 172.21.15.59:0/1304254315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8380745c0 0x7fc83819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 >> 172.21.15.59:0/1304254315 conn(0x7fc8380fd8c0 msgr2=0x7fc8380fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.711+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 shutdown_connections 2024-09-05T11:55:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:12.712+0000 7fc83f800640 1 -- 172.21.15.59:0/1304254315 wait complete. 2024-09-05T11:55:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:12 smithi059 ceph-mon[30025]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T11:55:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:12 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T11:55:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:12 smithi189 ceph-mon[34772]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T11:55:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:12 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T11:55:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:12 smithi089 ceph-mon[34499]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T11:55:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:12 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T11:55:13.534 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-05T11:55:13.911 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:13.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:13 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T11:55:13.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:13 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:13.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:13 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:14.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:13 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T11:55:14.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:13 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:14.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:13 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:13 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1304254315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T11:55:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:13 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T11:55:14.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:13 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.489+0000 7f5d37600640 1 -- 172.21.15.59:0/2193412273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30109140 msgr2=0x7f5d30110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.489+0000 7f5d37600640 1 --2- 172.21.15.59:0/2193412273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30109140 0x7f5d30110cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ca0 tx=0x7f5d2802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 -- 172.21.15.59:0/2193412273 shutdown_connections 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 --2- 172.21.15.59:0/2193412273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30109140 0x7f5d30110cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 --2- 172.21.15.59:0/2193412273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d301049c0 0x7f5d30108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 --2- 172.21.15.59:0/2193412273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d30104010 0x7f5d301043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 -- 172.21.15.59:0/2193412273 >> 172.21.15.59:0/2193412273 conn(0x7f5d300ffa50 msgr2=0x7f5d30101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 -- 172.21.15.59:0/2193412273 shutdown_connections 2024-09-05T11:55:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.490+0000 7f5d37600640 1 -- 172.21.15.59:0/2193412273 wait complete. 2024-09-05T11:55:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 Processor -- start 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 -- start start 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d301049c0 0x7f5d3019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d30109140 0x7f5d3019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301141a0 con 0x7f5d30104010 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30114020 con 0x7f5d30109140 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d37600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30114320 con 0x7f5d301049c0 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d34c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d34c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38564/0 (socket says 172.21.15.59:38564) 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d34c00640 1 -- 172.21.15.59:0/4155481023 learned_addr learned my addr 172.21.15.59:0/4155481023 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.491+0000 7f5d2fe00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d301049c0 0x7f5d3019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d35600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d30109140 0x7f5d3019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 -- 172.21.15.59:0/4155481023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d301049c0 msgr2=0x7f5d3019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d301049c0 0x7f5d3019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 -- 172.21.15.59:0/4155481023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d30109140 msgr2=0x7f5d3019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d30109140 0x7f5d3019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 -- 172.21.15.59:0/4155481023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3019f5b0 con 0x7f5d30104010 2024-09-05T11:55:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.492+0000 7f5d34c00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400e9f0 tx=0x7f5d2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.493+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400cd80 con 0x7f5d30104010 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.493+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d24010910 con 0x7f5d30104010 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.493+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3019f8a0 con 0x7f5d30104010 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.493+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24018990 con 0x7f5d30104010 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.493+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301a7180 con 0x7f5d30104010 2024-09-05T11:55:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.494+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d301043f0 con 0x7f5d30104010 2024-09-05T11:55:14.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.496+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d24010ac0 con 0x7f5d30104010 2024-09-05T11:55:14.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.497+0000 7f5d2d600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 0x7f5d180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.497+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5d240cf5f0 con 0x7f5d30104010 2024-09-05T11:55:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.497+0000 7f5d2fe00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 0x7f5d180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.498+0000 7f5d2fe00640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 0x7f5d180a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d301002c0 tx=0x7f5d1c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.501+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d2409bc60 con 0x7f5d30104010 2024-09-05T11:55:14.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:14.723+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5d3019b960 con 0x7f5d30104010 2024-09-05T11:55:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:14 smithi189 ceph-mon[34772]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-05T11:55:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:14 smithi089 ceph-mon[34499]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-05T11:55:15.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:14 smithi059 ceph-mon[30025]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-05T11:55:15.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.727+0000 7f5d2d600640 1 -- 172.21.15.59:0/4155481023 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f5d2409ff60 con 0x7f5d30104010 2024-09-05T11:55:15.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.730+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 msgr2=0x7f5d180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:15.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.730+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 0x7f5d180a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d301002c0 tx=0x7f5d1c005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:15.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.730+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 msgr2=0x7f5d3019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:15.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.730+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400e9f0 tx=0x7f5d2400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 shutdown_connections 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5d180a2b80 0x7f5d180a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d30109140 0x7f5d3019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d301049c0 0x7f5d3019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 --2- 172.21.15.59:0/4155481023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d30104010 0x7f5d3019a600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 >> 172.21.15.59:0/4155481023 conn(0x7f5d300ffa50 msgr2=0x7f5d30100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 shutdown_connections 2024-09-05T11:55:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:15.731+0000 7f5d37600640 1 -- 172.21.15.59:0/4155481023 wait complete. 2024-09-05T11:55:16.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:15 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T11:55:16.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:15 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T11:55:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:15 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T11:55:16.489 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-05T11:55:16.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-05T11:55:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:17.009 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:16 smithi059 ceph-mon[30025]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 0 op/s 2024-09-05T11:55:17.009 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:16 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T11:55:17.009 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:16 smithi059 ceph-mon[30025]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:16 smithi189 ceph-mon[34772]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 0 op/s 2024-09-05T11:55:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:16 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T11:55:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:16 smithi189 ceph-mon[34772]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:16 smithi089 ceph-mon[34499]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 255 B/s wr, 0 op/s 2024-09-05T11:55:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:16 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4155481023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T11:55:17.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:16 smithi089 ceph-mon[34499]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-05T11:55:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 -- 172.21.15.59:0/2678375966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f701049c0 msgr2=0x7f7f70108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 --2- 172.21.15.59:0/2678375966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f701049c0 0x7f7f70108a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009900 tx=0x7f7f5802a410 comp rx=0 tx=0).stop 2024-09-05T11:55:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 -- 172.21.15.59:0/2678375966 shutdown_connections 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 --2- 172.21.15.59:0/2678375966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f70109140 0x7f7f70110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 --2- 172.21.15.59:0/2678375966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f701049c0 0x7f7f70108a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 --2- 172.21.15.59:0/2678375966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f70104010 0x7f7f701043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 -- 172.21.15.59:0/2678375966 >> 172.21.15.59:0/2678375966 conn(0x7f7f700ffa50 msgr2=0x7f7f70101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 -- 172.21.15.59:0/2678375966 shutdown_connections 2024-09-05T11:55:17.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.455+0000 7f7f76600640 1 -- 172.21.15.59:0/2678375966 wait complete. 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.456+0000 7f7f76600640 1 Processor -- start 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.456+0000 7f7f76600640 1 -- start start 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f70104010 0x7f7f7019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f701049c0 0x7f7f7019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70114160 con 0x7f7f70109140 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70113fe0 con 0x7f7f701049c0 2024-09-05T11:55:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f76600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701142e0 con 0x7f7f70104010 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38576/0 (socket says 172.21.15.59:38576) 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f6fe00640 1 -- 172.21.15.59:0/890740856 learned_addr learned my addr 172.21.15.59:0/890740856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.457+0000 7f7f66a00640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f701049c0 0x7f7f7019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 -- 172.21.15.59:0/890740856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f70104010 msgr2=0x7f7f7019a660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f70104010 0x7f7f7019a660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 -- 172.21.15.59:0/890740856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f701049c0 msgr2=0x7f7f7019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f701049c0 0x7f7f7019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 -- 172.21.15.59:0/890740856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7019f610 con 0x7f7f70109140 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6fe00640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800eb10 tx=0x7f7f6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68004b20 con 0x7f7f70109140 2024-09-05T11:55:17.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f68004cc0 con 0x7f7f70109140 2024-09-05T11:55:17.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7019f900 con 0x7f7f70109140 2024-09-05T11:55:17.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68010760 con 0x7f7f70109140 2024-09-05T11:55:17.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.458+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a71e0 con 0x7f7f70109140 2024-09-05T11:55:17.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.460+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f701043f0 con 0x7f7f70109140 2024-09-05T11:55:17.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.461+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7f68005020 con 0x7f7f70109140 2024-09-05T11:55:17.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.463+0000 7f7f6cc00640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 0x7f7f540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:17.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.463+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7f680cf940 con 0x7f7f70109140 2024-09-05T11:55:17.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.463+0000 7f7f6f400640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 0x7f7f540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.464+0000 7f7f6f400640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 0x7f7f540a50c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009a60 tx=0x7f7f60005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:17.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.467+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f6809c600 con 0x7f7f70109140 2024-09-05T11:55:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.677+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7f70062630 con 0x7f7f540a2c00 2024-09-05T11:55:17.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.700+0000 7f7f6cc00640 1 -- 172.21.15.59:0/890740856 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f70062630 con 0x7f7f540a2c00 2024-09-05T11:55:17.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.702+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 msgr2=0x7f7f540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:17.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.702+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 0x7f7f540a50c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009a60 tx=0x7f7f60005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:17.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.702+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 msgr2=0x7f7f7019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:17.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.702+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800eb10 tx=0x7f7f6800c2d0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 shutdown_connections 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7f540a2c00 0x7f7f540a50c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f70109140 0x7f7f7019ef30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f701049c0 0x7f7f7019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 --2- 172.21.15.59:0/890740856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f70104010 0x7f7f7019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 >> 172.21.15.59:0/890740856 conn(0x7f7f700ffa50 msgr2=0x7f7f70100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 shutdown_connections 2024-09-05T11:55:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:17.703+0000 7f7f76600640 1 -- 172.21.15.59:0/890740856 wait complete. 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-05T11:55:17.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:55:17.685+0000 7fa9f1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:55:17.685+0000 7fa9f1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:55:17.685+0000 7fa9f1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:55:17.685+0000 7fa9f1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T11:55:17.959 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 11:55:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T11:55:17.685+0000 7fa9f1600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T11:55:18.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T11:55:18.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:17 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-05T11:55:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-05T11:55:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-05T11:55:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T11:55:18.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:17 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-05T11:55:18.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-05T11:55:18.825 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:19.009 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T11:55:19.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T11:55:19.011 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-05T11:55:19.011 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:18 smithi059 ceph-mon[30025]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T11:55:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-05T11:55:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:18 smithi189 ceph-mon[34772]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T11:55:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-05T11:55:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-05T11:55:19.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:18 smithi089 ceph-mon[34499]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T11:55:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 -- 172.21.15.59:0/2569150939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 msgr2=0x7fcc101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:19.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 --2- 172.21.15.59:0/2569150939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc101067e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009860 tx=0x7fcbf802a340 comp rx=0 tx=0).stop 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 -- 172.21.15.59:0/2569150939 shutdown_connections 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 --2- 172.21.15.59:0/2569150939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10106f10 0x7fcc1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 --2- 172.21.15.59:0/2569150939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc101067e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 --2- 172.21.15.59:0/2569150939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10101de0 0x7fcc101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 -- 172.21.15.59:0/2569150939 >> 172.21.15.59:0/2569150939 conn(0x7fcc100fd8c0 msgr2=0x7fcc100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 -- 172.21.15.59:0/2569150939 shutdown_connections 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.414+0000 7fcc16200640 1 -- 172.21.15.59:0/2569150939 wait complete. 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 Processor -- start 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 -- start start 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10101de0 0x7fcc100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 0x7fcc10074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10111f70 con 0x7fcc10102790 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10111df0 con 0x7fcc10101de0 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc16200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc101120f0 con 0x7fcc10106f10 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.415+0000 7fcc06a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38614/0 (socket says 172.21.15.59:38614) 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 -- 172.21.15.59:0/1496124638 learned_addr learned my addr 172.21.15.59:0/1496124638 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc0f400640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10101de0 0x7fcc100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc0fe00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 0x7fcc10074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 -- 172.21.15.59:0/1496124638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 msgr2=0x7fcc10074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 0x7fcc10074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 -- 172.21.15.59:0/1496124638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10101de0 msgr2=0x7fcc100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10101de0 0x7fcc100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 -- 172.21.15.59:0/1496124638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc101aae20 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc0fe00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 0x7fcc10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc06a00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8035040 tx=0x7fcbf8033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.416+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8037040 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.417+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc101aaff0 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.417+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf8040070 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.417+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80039b0 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.417+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc101ab450 con 0x7fcc10102790 2024-09-05T11:55:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.418+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc101032c0 con 0x7fcc10102790 2024-09-05T11:55:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.420+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcbf80305e0 con 0x7fcc10102790 2024-09-05T11:55:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.421+0000 7fcc0cc00640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 0x7fcbf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.421+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcbf80ef9b0 con 0x7fcc10102790 2024-09-05T11:55:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.422+0000 7fcc0f400640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 0x7fcbf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.422+0000 7fcc0f400640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 0x7fcbf40a5040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005fd0 tx=0x7fcc00009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:19.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.424+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbf80bc550 con 0x7fcc10102790 2024-09-05T11:55:19.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.640+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcc10062630 con 0x7fcbf40a2b80 2024-09-05T11:55:19.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.665+0000 7fcc0cc00640 1 -- 172.21.15.59:0/1496124638 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc10062630 con 0x7fcbf40a2b80 2024-09-05T11:55:19.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 msgr2=0x7fcbf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 0x7fcbf40a5040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005fd0 tx=0x7fcc00009260 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 msgr2=0x7fcc10074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8035040 tx=0x7fcbf8033b60 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 shutdown_connections 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbf40a2b80 0x7fcbf40a5040 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc10106f10 0x7fcc10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc10102790 0x7fcc10074a30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 --2- 172.21.15.59:0/1496124638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc10101de0 0x7fcc100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.667+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 >> 172.21.15.59:0/1496124638 conn(0x7fcc100fd8c0 msgr2=0x7fcc100feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.668+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 shutdown_connections 2024-09-05T11:55:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:19.668+0000 7fcc16200640 1 -- 172.21.15.59:0/1496124638 wait complete. 2024-09-05T11:55:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:19 smithi059 ceph-mon[30025]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T11:55:20.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:19 smithi189 ceph-mon[34772]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T11:55:20.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:19 smithi089 ceph-mon[34499]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T11:55:20.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-05T11:55:20.816 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:21.077 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:20 smithi059 ceph-mon[30025]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-05T11:55:21.077 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:20 smithi059 ceph-mon[30025]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:21.077 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:20 smithi059 ceph-mon[30025]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T11:55:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:20 smithi189 ceph-mon[34772]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-05T11:55:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:20 smithi189 ceph-mon[34772]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:20 smithi189 ceph-mon[34772]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T11:55:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:20 smithi089 ceph-mon[34499]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-05T11:55:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:20 smithi089 ceph-mon[34499]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:20 smithi089 ceph-mon[34499]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T11:55:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 -- 172.21.15.59:0/3757681123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe124101df0 msgr2=0x7fe1241021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 --2- 172.21.15.59:0/3757681123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe124101df0 0x7fe1241021d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11402a410 comp rx=0 tx=0).stop 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 -- 172.21.15.59:0/3757681123 shutdown_connections 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 --2- 172.21.15.59:0/3757681123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 0x7fe12410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 --2- 172.21.15.59:0/3757681123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1241027a0 0x7fe1241067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 --2- 172.21.15.59:0/3757681123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe124101df0 0x7fe1241021d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.344+0000 7fe12a800640 1 -- 172.21.15.59:0/3757681123 >> 172.21.15.59:0/3757681123 conn(0x7fe1240fd8b0 msgr2=0x7fe1240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.345+0000 7fe12a800640 1 -- 172.21.15.59:0/3757681123 shutdown_connections 2024-09-05T11:55:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.345+0000 7fe12a800640 1 -- 172.21.15.59:0/3757681123 wait complete. 2024-09-05T11:55:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.345+0000 7fe12a800640 1 Processor -- start 2024-09-05T11:55:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 -- start start 2024-09-05T11:55:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe124101df0 0x7fe12419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 0x7fe12419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe124111f90 con 0x7fe1241027a0 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe124111e10 con 0x7fe124101df0 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe12a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112110 con 0x7fe124106f20 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe122a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe122a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57372/0 (socket says 172.21.15.59:57372) 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.346+0000 7fe122a00640 1 -- 172.21.15.59:0/2959803580 learned_addr learned my addr 172.21.15.59:0/2959803580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe123400640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe124101df0 0x7fe12419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe123e00640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 0x7fe12419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 -- 172.21.15.59:0/2959803580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 msgr2=0x7fe12419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 0x7fe12419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 -- 172.21.15.59:0/2959803580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe124101df0 msgr2=0x7fe12419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe124101df0 0x7fe12419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 -- 172.21.15.59:0/2959803580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12419f5a0 con 0x7fe1241027a0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe122a00640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe11000d950 tx=0x7fe11000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.347+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110017030 con 0x7fe1241027a0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.348+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe110004020 con 0x7fe1241027a0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.348+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12419f890 con 0x7fe1241027a0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.348+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100047a0 con 0x7fe1241027a0 2024-09-05T11:55:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.348+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241a7170 con 0x7fe1241027a0 2024-09-05T11:55:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.349+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1241021d0 con 0x7fe1241027a0 2024-09-05T11:55:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.351+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe1100041d0 con 0x7fe1241027a0 2024-09-05T11:55:21.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.352+0000 7fe11be00640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 0x7fe1080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:21.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.352+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe1100cee80 con 0x7fe1241027a0 2024-09-05T11:55:21.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.352+0000 7fe123400640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 0x7fe1080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:21.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.353+0000 7fe123400640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 0x7fe1080a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c00 tx=0x7fe114005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:21.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.356+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe11009b920 con 0x7fe1241027a0 2024-09-05T11:55:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.569+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe124075430 con 0x7fe1080a2c10 2024-09-05T11:55:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.574+0000 7fe11be00640 1 -- 172.21.15.59:0/2959803580 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe124075430 con 0x7fe1080a2c10 2024-09-05T11:55:21.576 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_0/c3aecac2-2692-4c09-85a9-69b522b3b5cd 2024-09-05T11:55:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 msgr2=0x7fe1080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 0x7fe1080a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c00 tx=0x7fe114005b50 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 msgr2=0x7fe12419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe11000d950 tx=0x7fe11000de10 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 shutdown_connections 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1080a2c10 0x7fe1080a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124106f20 0x7fe12419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1241027a0 0x7fe12419ab30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 --2- 172.21.15.59:0/2959803580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe124101df0 0x7fe12419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 >> 172.21.15.59:0/2959803580 conn(0x7fe1240fd8b0 msgr2=0x7fe1240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 shutdown_connections 2024-09-05T11:55:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:21.577+0000 7fe12a800640 1 -- 172.21.15.59:0/2959803580 wait complete. 2024-09-05T11:55:22.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-05T11:55:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:23.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:22 smithi059 ceph-mon[30025]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:23.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:22 smithi059 ceph-mon[30025]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:22 smithi189 ceph-mon[34772]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:22 smithi189 ceph-mon[34772]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:22 smithi089 ceph-mon[34499]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:22 smithi089 ceph-mon[34499]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 -- 172.21.15.59:0/1019347333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96810e3c0 msgr2=0x7ff968110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 --2- 172.21.15.59:0/1019347333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96810e3c0 0x7ff968110810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff960009ca0 tx=0x7ff96002a950 comp rx=0 tx=0).stop 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 -- 172.21.15.59:0/1019347333 shutdown_connections 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 --2- 172.21.15.59:0/1019347333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96810e3c0 0x7ff968110810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 --2- 172.21.15.59:0/1019347333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074f40 0x7ff96810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 --2- 172.21.15.59:0/1019347333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff968074620 0x7ff968074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 -- 172.21.15.59:0/1019347333 >> 172.21.15.59:0/1019347333 conn(0x7ff9680fd8c0 msgr2=0x7ff9680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 -- 172.21.15.59:0/1019347333 shutdown_connections 2024-09-05T11:55:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.314+0000 7ff96de00640 1 -- 172.21.15.59:0/1019347333 wait complete. 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 Processor -- start 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 -- start start 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074620 0x7ff96819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff96810e3c0 0x7ff96819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff968114150 con 0x7ff968074f40 2024-09-05T11:55:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff968113fd0 con 0x7ff96810e3c0 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.315+0000 7ff96de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681142d0 con 0x7ff968074620 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57386/0 (socket says 172.21.15.59:57386) 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 -- 172.21.15.59:0/3582978298 learned_addr learned my addr 172.21.15.59:0/3582978298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff966a00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074620 0x7ff96819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff967400640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff96810e3c0 0x7ff96819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 -- 172.21.15.59:0/3582978298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074620 msgr2=0x7ff96819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074620 0x7ff96819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 -- 172.21.15.59:0/3582978298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff96810e3c0 msgr2=0x7ff96819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff96810e3c0 0x7ff96819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 -- 172.21.15.59:0/3582978298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96819f6c0 con 0x7ff968074f40 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.316+0000 7ff95fe00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9f0 tx=0x7ff95000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.317+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000cd80 con 0x7ff968074f40 2024-09-05T11:55:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.317+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950010910 con 0x7ff968074f40 2024-09-05T11:55:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.317+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950018990 con 0x7ff968074f40 2024-09-05T11:55:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.317+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96819f9b0 con 0x7ff968074f40 2024-09-05T11:55:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.317+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681a7290 con 0x7ff968074f40 2024-09-05T11:55:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.318+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff968074a00 con 0x7ff968074f40 2024-09-05T11:55:23.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.320+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff950010ac0 con 0x7ff968074f40 2024-09-05T11:55:23.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.322+0000 7ff96ca00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 0x7ff94c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.322+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff9500cf760 con 0x7ff968074f40 2024-09-05T11:55:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.322+0000 7ff966a00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 0x7ff94c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.323+0000 7ff966a00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 0x7ff94c0a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff958009bd0 tx=0x7ff958005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:23.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.326+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff95009bb50 con 0x7ff968074f40 2024-09-05T11:55:23.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.539+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff968109570 con 0x7ff94c0a2c90 2024-09-05T11:55:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.568+0000 7ff96ca00640 1 -- 172.21.15.59:0/3582978298 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff968109570 con 0x7ff94c0a2c90 2024-09-05T11:55:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.570+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 msgr2=0x7ff94c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.570+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 0x7ff94c0a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff958009bd0 tx=0x7ff958005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.570+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 msgr2=0x7ff96819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.570+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9f0 tx=0x7ff95000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 shutdown_connections 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff94c0a2c90 0x7ff94c0a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff96810e3c0 0x7ff96819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff968074f40 0x7ff96819ac50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 --2- 172.21.15.59:0/3582978298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968074620 0x7ff96819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 >> 172.21.15.59:0/3582978298 conn(0x7ff9680fd8c0 msgr2=0x7ff9680fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 shutdown_connections 2024-09-05T11:55:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:23.571+0000 7ff96de00640 1 -- 172.21.15.59:0/3582978298 wait complete. 2024-09-05T11:55:24.307 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-05T11:55:24.681 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:25.026 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:24 smithi059 ceph-mon[30025]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:25.026 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:24 smithi059 ceph-mon[30025]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:24 smithi189 ceph-mon[34772]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:25.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:24 smithi189 ceph-mon[34772]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:24 smithi089 ceph-mon[34499]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-09-05T11:55:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:24 smithi089 ceph-mon[34499]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2009639104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 msgr2=0x7f4bb4114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2009639104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb4114c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009ca0 tx=0x7f4bac02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2009639104 shutdown_connections 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2009639104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb4114c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2009639104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 0x7f4bb4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2009639104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.232+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2009639104 >> 172.21.15.59:0/2009639104 conn(0x7f4bb40ff8d0 msgr2=0x7f4bb4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.233+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2009639104 shutdown_connections 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.233+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2009639104 wait complete. 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.233+0000 7f4bb9a00640 1 Processor -- start 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 -- start start 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb41a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 0x7f4bb41a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4118420 con 0x7f4bb4112870 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41182a0 con 0x7f4bb4077040 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41185a0 con 0x7f4bb4076720 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57412/0 (socket says 172.21.15.59:57412) 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb9000640 1 -- 172.21.15.59:0/2351888404 learned_addr learned my addr 172.21.15.59:0/2351888404 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.234+0000 7f4bb3400640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 0x7f4bb41a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb3e00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb41a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 -- 172.21.15.59:0/2351888404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 msgr2=0x7f4bb41a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb41a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 -- 172.21.15.59:0/2351888404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 msgr2=0x7f4bb41a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 0x7f4bb41a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 -- 172.21.15.59:0/2351888404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb419ef80 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb3e00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb41a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9000640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4bac02ade0 tx=0x7f4bac0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac003c30 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac003dd0 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac02fd60 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb419f210 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.235+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41ab3f0 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.236+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb4077e60 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.238+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4bac004050 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.240+0000 7f4bb0c00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 0x7f4b980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.240+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4bac0f0860 con 0x7f4bb4112870 2024-09-05T11:55:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.240+0000 7f4bb3e00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 0x7f4b980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:25.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.241+0000 7f4bb3e00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 0x7f4b980a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006fd0 tx=0x7f4ba4008040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:25.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.244+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bac0bd560 con 0x7f4bb4112870 2024-09-05T11:55:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.455+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4bb4062630 con 0x7f4b980a2c10 2024-09-05T11:55:25.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.461+0000 7f4bb0c00640 1 -- 172.21.15.59:0/2351888404 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4bb4062630 con 0x7f4b980a2c10 2024-09-05T11:55:25.462 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_1/7feb4e0e-1bee-4d90-aa3c-22db3410b75a 2024-09-05T11:55:25.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 msgr2=0x7f4b980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 0x7f4b980a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006fd0 tx=0x7f4ba4008040 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 msgr2=0x7f4bb419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4bac02ade0 tx=0x7f4bac0319f0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 shutdown_connections 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b980a2c10 0x7f4b980a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bb4112870 0x7f4bb419e710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bb4077040 0x7f4bb41a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 --2- 172.21.15.59:0/2351888404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb4076720 0x7f4bb41a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 >> 172.21.15.59:0/2351888404 conn(0x7f4bb40ff8d0 msgr2=0x7f4bb4101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 shutdown_connections 2024-09-05T11:55:25.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:25.463+0000 7f4bb9a00640 1 -- 172.21.15.59:0/2351888404 wait complete. 2024-09-05T11:55:26.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:55:26.630 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:27.058 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:26 smithi189 ceph-mon[34772]: pgmap v325: 129 pgs: 129 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T11:55:27.058 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:26 smithi189 ceph-mon[34772]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:27.058 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:26 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:55:27.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:26 smithi059 ceph-mon[30025]: pgmap v325: 129 pgs: 129 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T11:55:27.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:26 smithi059 ceph-mon[30025]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:27.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:26 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:55:27.079 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:26 smithi089 ceph-mon[34499]: pgmap v325: 129 pgs: 129 active+clean; 606 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T11:55:27.080 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:26 smithi089 ceph-mon[34499]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:55:27.080 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:26 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:55:27.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 -- 172.21.15.59:0/1921252844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41079d0 msgr2=0x7fbfa4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:27.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/1921252844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41079d0 0x7fbfa4107db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009890 tx=0x7fbf9402a360 comp rx=0 tx=0).stop 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 -- 172.21.15.59:0/1921252844 shutdown_connections 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/1921252844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 0x7fbfa410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/1921252844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41019d0 0x7fbfa4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/1921252844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41079d0 0x7fbfa4107db0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.178+0000 7fbfa8e00640 1 -- 172.21.15.59:0/1921252844 >> 172.21.15.59:0/1921252844 conn(0x7fbfa40fd710 msgr2=0x7fbfa40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.179+0000 7fbfa8e00640 1 -- 172.21.15.59:0/1921252844 shutdown_connections 2024-09-05T11:55:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.179+0000 7fbfa8e00640 1 -- 172.21.15.59:0/1921252844 wait complete. 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.180+0000 7fbfa8e00640 1 Processor -- start 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.180+0000 7fbfa8e00640 1 -- start start 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.180+0000 7fbfa8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 0x7fbfa4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 0x7fbfa419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa8e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4111dd0 con 0x7fbfa41079d0 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4111c50 con 0x7fbfa4102370 2024-09-05T11:55:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa8e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4111f50 con 0x7fbfa41019d0 2024-09-05T11:55:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57630/0 (socket says 172.21.15.59:57630) 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 -- 172.21.15.59:0/3024292663 learned_addr learned my addr 172.21.15.59:0/3024292663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa2a00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 0x7fbfa4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 0x7fbfa419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 -- 172.21.15.59:0/3024292663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 msgr2=0x7fbfa4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 0x7fbfa4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 -- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 msgr2=0x7fbfa419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 0x7fbfa419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3400640 1 -- 172.21.15.59:0/3024292663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa419d280 con 0x7fbfa41019d0 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.181+0000 7fbfa3e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 0x7fbfa419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbfa3400640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbf94035040 tx=0x7fbf94033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94037040 con 0x7fbfa41019d0 2024-09-05T11:55:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa419d510 con 0x7fbfa41019d0 2024-09-05T11:55:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf94040070 con 0x7fbfa41019d0 2024-09-05T11:55:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94003970 con 0x7fbfa41019d0 2024-09-05T11:55:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.182+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa4074420 con 0x7fbfa41019d0 2024-09-05T11:55:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.183+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa4103110 con 0x7fbfa41019d0 2024-09-05T11:55:27.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.187+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf940306a0 con 0x7fbfa41019d0 2024-09-05T11:55:27.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.188+0000 7fbf9be00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 0x7fbf880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:27.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.188+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbf940ef660 con 0x7fbfa41019d0 2024-09-05T11:55:27.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.188+0000 7fbfa2a00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 0x7fbf880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:27.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.189+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf940bc1b0 con 0x7fbfa41019d0 2024-09-05T11:55:27.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.189+0000 7fbfa2a00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 0x7fbf880a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ffd50 tx=0x7fbf90014040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:27.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.435+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfa4101e30 con 0x7fbfa41019d0 2024-09-05T11:55:27.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.436+0000 7fbf9be00640 1 -- 172.21.15.59:0/3024292663 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fbf940bbab0 con 0x7fbfa41019d0 2024-09-05T11:55:27.437 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:27.438 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:55:27.438 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 msgr2=0x7fbf880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 0x7fbf880a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ffd50 tx=0x7fbf90014040 comp rx=0 tx=0).stop 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 msgr2=0x7fbfa41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbf94035040 tx=0x7fbf94033b60 comp rx=0 tx=0).stop 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 shutdown_connections 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbf880a2c10 0x7fbf880a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa41079d0 0x7fbfa419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102370 0x7fbfa4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 --2- 172.21.15.59:0/3024292663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfa41019d0 0x7fbfa41982d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:27.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.438+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 >> 172.21.15.59:0/3024292663 conn(0x7fbfa40fd710 msgr2=0x7fbfa40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:27.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.439+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 shutdown_connections 2024-09-05T11:55:27.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:27.439+0000 7fbfa8e00640 1 -- 172.21.15.59:0/3024292663 wait complete. 2024-09-05T11:55:27.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:27 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3024292663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:55:27.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:27 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:28.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:27 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3024292663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:55:28.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:27 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:27 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3024292663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:27 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:28.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:55:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:28 smithi189 ceph-mon[34772]: pgmap v326: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2024-09-05T11:55:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:28 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:28 smithi089 ceph-mon[34499]: pgmap v326: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2024-09-05T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:28 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:28 smithi059 ceph-mon[30025]: pgmap v326: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2024-09-05T11:55:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:28 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:29.367 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:29.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 -- 172.21.15.59:0/3015874660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8112a30 msgr2=0x7fb1b8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 --2- 172.21.15.59:0/3015874660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8112a30 0x7fb1b8114e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009ca0 tx=0x7fb1b002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 -- 172.21.15.59:0/3015874660 shutdown_connections 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 --2- 172.21.15.59:0/3015874660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8112a30 0x7fb1b8114e20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 --2- 172.21.15.59:0/3015874660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8077040 0x7fb1b81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 --2- 172.21.15.59:0/3015874660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8076720 0x7fb1b8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 -- 172.21.15.59:0/3015874660 >> 172.21.15.59:0/3015874660 conn(0x7fb1b80ffa90 msgr2=0x7fb1b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 -- 172.21.15.59:0/3015874660 shutdown_connections 2024-09-05T11:55:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.922+0000 7fb1bca00640 1 -- 172.21.15.59:0/3015874660 wait complete. 2024-09-05T11:55:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.923+0000 7fb1bca00640 1 Processor -- start 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 -- start start 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8076720 0x7fb1b81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8112a30 0x7fb1b819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8118570 con 0x7fb1b8077040 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81183f0 con 0x7fb1b8112a30 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1bca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81186f0 con 0x7fb1b8076720 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1b5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1b5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57432/0 (socket says 172.21.15.59:57432) 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1b5400640 1 -- 172.21.15.59:0/1412884570 learned_addr learned my addr 172.21.15.59:0/1412884570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.924+0000 7fb1b5e00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8076720 0x7fb1b81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b6800640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8112a30 0x7fb1b819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 -- 172.21.15.59:0/1412884570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8076720 msgr2=0x7fb1b81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8076720 0x7fb1b81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 -- 172.21.15.59:0/1412884570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8112a30 msgr2=0x7fb1b819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8112a30 0x7fb1b819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 -- 172.21.15.59:0/1412884570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b819efc0 con 0x7fb1b8077040 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1b5400640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400d950 tx=0x7fb1a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4017070 con 0x7fb1b8077040 2024-09-05T11:55:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.925+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b819f2b0 con 0x7fb1b8077040 2024-09-05T11:55:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.926+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4009c30 con 0x7fb1b8077040 2024-09-05T11:55:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.926+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400f6b0 con 0x7fb1b8077040 2024-09-05T11:55:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.926+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81ab500 con 0x7fb1b8077040 2024-09-05T11:55:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.928+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8077e60 con 0x7fb1b8077040 2024-09-05T11:55:29.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.930+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1a40109d0 con 0x7fb1b8077040 2024-09-05T11:55:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.931+0000 7fb1aea00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.931+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1a40ce8c0 con 0x7fb1b8077040 2024-09-05T11:55:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.932+0000 7fb1b5e00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.932+0000 7fb1b5e00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 0x7fb19c0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009bd0 tx=0x7fb1a8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:29.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:29.936+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1a4009de0 con 0x7fb1b8077040 2024-09-05T11:55:30.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.130+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1b810dbc0 con 0x7fb1b8077040 2024-09-05T11:55:30.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.132+0000 7fb1aea00640 1 -- 172.21.15.59:0/1412884570 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24674 (secure 0 0 0) 0x7fb1a409ad10 con 0x7fb1b8077040 2024-09-05T11:55:30.134 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:30.134 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:55:18.740638+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:55:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.135+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 msgr2=0x7fb19c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.135+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 0x7fb19c0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009bd0 tx=0x7fb1a8005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.135+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 msgr2=0x7fb1b81a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.135+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400d950 tx=0x7fb1a400de10 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 shutdown_connections 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb19c0a2c90 0x7fb19c0a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1b8112a30 0x7fb1b819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b8077040 0x7fb1b81a4be0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 --2- 172.21.15.59:0/1412884570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1b8076720 0x7fb1b81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 >> 172.21.15.59:0/1412884570 conn(0x7fb1b80ffa90 msgr2=0x7fb1b8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 shutdown_connections 2024-09-05T11:55:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:30.136+0000 7fb1bca00640 1 -- 172.21.15.59:0/1412884570 wait complete. 2024-09-05T11:55:30.859 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T11:55:30.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: pgmap v327: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1412884570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:55:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:30 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: pgmap v327: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-09-05T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1412884570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:55:31.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:55:31.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:30 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: pgmap v327: 129 pgs: 129 active+clean; 632 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-09-05T11:55:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1412884570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:55:31.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:55:31.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:30 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:55:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 -- 172.21.15.59:0/4176527920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34112870 msgr2=0x7fbd34114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 --2- 172.21.15.59:0/4176527920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34112870 0x7fbd34114c60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009ca0 tx=0x7fbd3002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 -- 172.21.15.59:0/4176527920 shutdown_connections 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 --2- 172.21.15.59:0/4176527920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34112870 0x7fbd34114c60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 --2- 172.21.15.59:0/4176527920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34077040 0x7fbd34112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 --2- 172.21.15.59:0/4176527920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34076720 0x7fbd34076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 -- 172.21.15.59:0/4176527920 >> 172.21.15.59:0/4176527920 conn(0x7fbd340ff8d0 msgr2=0x7fbd34101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 -- 172.21.15.59:0/4176527920 shutdown_connections 2024-09-05T11:55:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.793+0000 7fbd3b200640 1 -- 172.21.15.59:0/4176527920 wait complete. 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.794+0000 7fbd3b200640 1 Processor -- start 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.794+0000 7fbd3b200640 1 -- start start 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.794+0000 7fbd3b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.794+0000 7fbd3b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34077040 0x7fbd341a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd3b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34112870 0x7fbd3419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd3b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341183c0 con 0x7fbd34077040 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd3b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34118240 con 0x7fbd34076720 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd3b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34118540 con 0x7fbd34112870 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:48984/0 (socket says 172.21.15.59:48984) 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34077040 0x7fbd341a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 -- 172.21.15.59:0/3443080689 learned_addr learned my addr 172.21.15.59:0/3443080689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd3a800640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34112870 0x7fbd3419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 -- 172.21.15.59:0/3443080689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34112870 msgr2=0x7fbd3419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34112870 0x7fbd3419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 -- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34077040 msgr2=0x7fbd341a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34077040 0x7fbd341a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.795+0000 7fbd39e00640 1 -- 172.21.15.59:0/3443080689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3419ef40 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd39e00640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800d770 tx=0x7fbd2800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28017070 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd28005500 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004e00 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3419f230 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.796+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd341ab3b0 con 0x7fbd34076720 2024-09-05T11:55:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.797+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34077e60 con 0x7fbd34076720 2024-09-05T11:55:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.799+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd28004fa0 con 0x7fbd34076720 2024-09-05T11:55:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.801+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 0x7fbd1c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.801+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd280cf310 con 0x7fbd34076720 2024-09-05T11:55:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.801+0000 7fbd39400640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 0x7fbd1c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:31.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.802+0000 7fbd39400640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 0x7fbd1c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd3419ff20 tx=0x7fbd2400b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.803+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd2809b780 con 0x7fbd34076720 2024-09-05T11:55:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.987+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd3410da70 con 0x7fbd1c0a2c90 2024-09-05T11:55:31.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:31.998+0000 7fbd2ea00640 1 -- 172.21.15.59:0/3443080689 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340291 (secure 0 0 0) 0x7fbd24010ba0 con 0x7fbd1c0a2c90 2024-09-05T11:55:32.005 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:32.007 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 msgr2=0x7fbd1c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 0x7fbd1c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd3419ff20 tx=0x7fbd2400b410 comp rx=0 tx=0).stop 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 msgr2=0x7fbd341a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800d770 tx=0x7fbd2800dc30 comp rx=0 tx=0).stop 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 shutdown_connections 2024-09-05T11:55:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbd1c0a2c90 0x7fbd1c0a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd34112870 0x7fbd3419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd34077040 0x7fbd341a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 --2- 172.21.15.59:0/3443080689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd34076720 0x7fbd341a4600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 >> 172.21.15.59:0/3443080689 conn(0x7fbd340ff8d0 msgr2=0x7fbd34101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 shutdown_connections 2024-09-05T11:55:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:32.003+0000 7fbd3b200640 1 -- 172.21.15.59:0/3443080689 wait complete. 2024-09-05T11:55:32.706 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-05T11:55:30.864744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651128,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":175,"num_read_kb":190,"num_write":265,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375816,"kb_used_data":14072,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566008,"statfs":{"total":1151940427776,"available":1151555592192,"internally_reserved":0,"allocated":14409728,"data_stored":8081759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56118,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":67,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003232"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748911+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:55:18.748911+0000","last_peered":"2024-09-05T11:55:18.748911+0000","last_clean":"2024-09-05T11:55:18.748911+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:55:18.748911+0000","last_undegraded":"2024-09-05T11:55:18.748911+0000","last_fullsized":"2024-09-05T11:55:18.748911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959651+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:55:18.959651+0000","last_peered":"2024-09-05T11:55:18.959651+0000","last_clean":"2024-09-05T11:55:18.959651+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:55:18.959651+0000","last_undegraded":"2024-09-05T11:55:18.959651+0000","last_fullsized":"2024-09-05T11:55:18.959651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960937+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:55:18.960937+0000","last_peered":"2024-09-05T11:55:18.960937+0000","last_clean":"2024-09-05T11:55:18.960937+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:55:18.960937+0000","last_undegraded":"2024-09-05T11:55:18.960937+0000","last_fullsized":"2024-09-05T11:55:18.960937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959828+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:55:18.959828+0000","last_peered":"2024-09-05T11:55:18.959828+0000","last_clean":"2024-09-05T11:55:18.959828+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:55:18.959828+0000","last_undegraded":"2024-09-05T11:55:18.959828+0000","last_fullsized":"2024-09-05T11:55:18.959828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829255+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:55:18.829255+0000","last_peered":"2024-09-05T11:55:18.829255+0000","last_clean":"2024-09-05T11:55:18.829255+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:55:18.829255+0000","last_undegraded":"2024-09-05T11:55:18.829255+0000","last_fullsized":"2024-09-05T11:55:18.829255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960868+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:55:18.960868+0000","last_peered":"2024-09-05T11:55:18.960868+0000","last_clean":"2024-09-05T11:55:18.960868+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:55:18.960868+0000","last_undegraded":"2024-09-05T11:55:18.960868+0000","last_fullsized":"2024-09-05T11:55:18.960868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750148+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:55:18.750148+0000","last_peered":"2024-09-05T11:55:18.750148+0000","last_clean":"2024-09-05T11:55:18.750148+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:55:18.750148+0000","last_undegraded":"2024-09-05T11:55:18.750148+0000","last_fullsized":"2024-09-05T11:55:18.750148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061069+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:55:19.061069+0000","last_peered":"2024-09-05T11:55:19.061069+0000","last_clean":"2024-09-05T11:55:19.061069+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:55:19.061069+0000","last_undegraded":"2024-09-05T11:55:19.061069+0000","last_fullsized":"2024-09-05T11:55:19.061069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746730+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:55:18.746730+0000","last_peered":"2024-09-05T11:55:18.746730+0000","last_clean":"2024-09-05T11:55:18.746730+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:55:18.746730+0000","last_undegraded":"2024-09-05T11:55:18.746730+0000","last_fullsized":"2024-09-05T11:55:18.746730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750651+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:55:18.750651+0000","last_peered":"2024-09-05T11:55:18.750651+0000","last_clean":"2024-09-05T11:55:18.750651+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:55:18.750651+0000","last_undegraded":"2024-09-05T11:55:18.750651+0000","last_fullsized":"2024-09-05T11:55:18.750651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749383+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:55:18.749383+0000","last_peered":"2024-09-05T11:55:18.749383+0000","last_clean":"2024-09-05T11:55:18.749383+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:55:18.749383+0000","last_undegraded":"2024-09-05T11:55:18.749383+0000","last_fullsized":"2024-09-05T11:55:18.749383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746451+0000","last_change":"2024-09-05T11:54:38.607159+0000","last_active":"2024-09-05T11:55:18.746451+0000","last_peered":"2024-09-05T11:55:18.746451+0000","last_clean":"2024-09-05T11:55:18.746451+0000","last_became_active":"2024-09-05T11:54:38.606569+0000","last_became_peered":"2024-09-05T11:54:38.606569+0000","last_unstale":"2024-09-05T11:55:18.746451+0000","last_undegraded":"2024-09-05T11:55:18.746451+0000","last_fullsized":"2024-09-05T11:55:18.746451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:28:07.583731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749879+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:55:18.749879+0000","last_peered":"2024-09-05T11:55:18.749879+0000","last_clean":"2024-09-05T11:55:18.749879+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:55:18.749879+0000","last_undegraded":"2024-09-05T11:55:18.749879+0000","last_fullsized":"2024-09-05T11:55:18.749879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749076+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:55:18.749076+0000","last_peered":"2024-09-05T11:55:18.749076+0000","last_clean":"2024-09-05T11:55:18.749076+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:55:18.749076+0000","last_undegraded":"2024-09-05T11:55:18.749076+0000","last_fullsized":"2024-09-05T11:55:18.749076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746802+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:55:18.746802+0000","last_peered":"2024-09-05T11:55:18.746802+0000","last_clean":"2024-09-05T11:55:18.746802+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:55:18.746802+0000","last_undegraded":"2024-09-05T11:55:18.746802+0000","last_fullsized":"2024-09-05T11:55:18.746802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959081+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:55:18.959081+0000","last_peered":"2024-09-05T11:55:18.959081+0000","last_clean":"2024-09-05T11:55:18.959081+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:55:18.959081+0000","last_undegraded":"2024-09-05T11:55:18.959081+0000","last_fullsized":"2024-09-05T11:55:18.959081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744681+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:55:18.744681+0000","last_peered":"2024-09-05T11:55:18.744681+0000","last_clean":"2024-09-05T11:55:18.744681+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:55:18.744681+0000","last_undegraded":"2024-09-05T11:55:18.744681+0000","last_fullsized":"2024-09-05T11:55:18.744681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753048+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:55:18.753048+0000","last_peered":"2024-09-05T11:55:18.753048+0000","last_clean":"2024-09-05T11:55:18.753048+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:55:18.753048+0000","last_undegraded":"2024-09-05T11:55:18.753048+0000","last_fullsized":"2024-09-05T11:55:18.753048+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748420+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:55:18.748420+0000","last_peered":"2024-09-05T11:55:18.748420+0000","last_clean":"2024-09-05T11:55:18.748420+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:55:18.748420+0000","last_undegraded":"2024-09-05T11:55:18.748420+0000","last_fullsized":"2024-09-05T11:55:18.748420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959409+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:55:18.959409+0000","last_peered":"2024-09-05T11:55:18.959409+0000","last_clean":"2024-09-05T11:55:18.959409+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:55:18.959409+0000","last_undegraded":"2024-09-05T11:55:18.959409+0000","last_fullsized":"2024-09-05T11:55:18.959409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748594+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:55:18.748594+0000","last_peered":"2024-09-05T11:55:18.748594+0000","last_clean":"2024-09-05T11:55:18.748594+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:55:18.748594+0000","last_undegraded":"2024-09-05T11:55:18.748594+0000","last_fullsized":"2024-09-05T11:55:18.748594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751886+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:55:18.751886+0000","last_peered":"2024-09-05T11:55:18.751886+0000","last_clean":"2024-09-05T11:55:18.751886+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:55:18.751886+0000","last_undegraded":"2024-09-05T11:55:18.751886+0000","last_fullsized":"2024-09-05T11:55:18.751886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060880+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:55:19.060880+0000","last_peered":"2024-09-05T11:55:19.060880+0000","last_clean":"2024-09-05T11:55:19.060880+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:55:19.060880+0000","last_undegraded":"2024-09-05T11:55:19.060880+0000","last_fullsized":"2024-09-05T11:55:19.060880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746843+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:55:18.746843+0000","last_peered":"2024-09-05T11:55:18.746843+0000","last_clean":"2024-09-05T11:55:18.746843+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:55:18.746843+0000","last_undegraded":"2024-09-05T11:55:18.746843+0000","last_fullsized":"2024-09-05T11:55:18.746843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829155+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:55:18.829155+0000","last_peered":"2024-09-05T11:55:18.829155+0000","last_clean":"2024-09-05T11:55:18.829155+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:55:18.829155+0000","last_undegraded":"2024-09-05T11:55:18.829155+0000","last_fullsized":"2024-09-05T11:55:18.829155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747880+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:55:18.747880+0000","last_peered":"2024-09-05T11:55:18.747880+0000","last_clean":"2024-09-05T11:55:18.747880+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:55:18.747880+0000","last_undegraded":"2024-09-05T11:55:18.747880+0000","last_fullsized":"2024-09-05T11:55:18.747880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744630+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:55:18.744630+0000","last_peered":"2024-09-05T11:55:18.744630+0000","last_clean":"2024-09-05T11:55:18.744630+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:55:18.744630+0000","last_undegraded":"2024-09-05T11:55:18.744630+0000","last_fullsized":"2024-09-05T11:55:18.744630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959726+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:55:18.959726+0000","last_peered":"2024-09-05T11:55:18.959726+0000","last_clean":"2024-09-05T11:55:18.959726+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:55:18.959726+0000","last_undegraded":"2024-09-05T11:55:18.959726+0000","last_fullsized":"2024-09-05T11:55:18.959726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:25.561771+0000","last_change":"2024-09-05T11:54:29.103741+0000","last_active":"2024-09-05T11:55:25.561771+0000","last_peered":"2024-09-05T11:55:25.561771+0000","last_clean":"2024-09-05T11:55:25.561771+0000","last_became_active":"2024-09-05T11:54:29.103285+0000","last_became_peered":"2024-09-05T11:54:29.103285+0000","last_unstale":"2024-09-05T11:55:25.561771+0000","last_undegraded":"2024-09-05T11:55:25.561771+0000","last_fullsized":"2024-09-05T11:55:25.561771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:01:16.713162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.958714+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:55:18.958714+0000","last_peered":"2024-09-05T11:55:18.958714+0000","last_clean":"2024-09-05T11:55:18.958714+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:55:18.958714+0000","last_undegraded":"2024-09-05T11:55:18.958714+0000","last_fullsized":"2024-09-05T11:55:18.958714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744903+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:55:18.744903+0000","last_peered":"2024-09-05T11:55:18.744903+0000","last_clean":"2024-09-05T11:55:18.744903+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:55:18.744903+0000","last_undegraded":"2024-09-05T11:55:18.744903+0000","last_fullsized":"2024-09-05T11:55:18.744903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750269+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:55:18.750269+0000","last_peered":"2024-09-05T11:55:18.750269+0000","last_clean":"2024-09-05T11:55:18.750269+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:55:18.750269+0000","last_undegraded":"2024-09-05T11:55:18.750269+0000","last_fullsized":"2024-09-05T11:55:18.750269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748452+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:55:18.748452+0000","last_peered":"2024-09-05T11:55:18.748452+0000","last_clean":"2024-09-05T11:55:18.748452+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:55:18.748452+0000","last_undegraded":"2024-09-05T11:55:18.748452+0000","last_fullsized":"2024-09-05T11:55:18.748452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745443+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:55:18.745443+0000","last_peered":"2024-09-05T11:55:18.745443+0000","last_clean":"2024-09-05T11:55:18.745443+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:55:18.745443+0000","last_undegraded":"2024-09-05T11:55:18.745443+0000","last_fullsized":"2024-09-05T11:55:18.745443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748502+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:55:18.748502+0000","last_peered":"2024-09-05T11:55:18.748502+0000","last_clean":"2024-09-05T11:55:18.748502+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:55:18.748502+0000","last_undegraded":"2024-09-05T11:55:18.748502+0000","last_fullsized":"2024-09-05T11:55:18.748502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750324+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:55:18.750324+0000","last_peered":"2024-09-05T11:55:18.750324+0000","last_clean":"2024-09-05T11:55:18.750324+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:55:18.750324+0000","last_undegraded":"2024-09-05T11:55:18.750324+0000","last_fullsized":"2024-09-05T11:55:18.750324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745255+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:55:18.745255+0000","last_peered":"2024-09-05T11:55:18.745255+0000","last_clean":"2024-09-05T11:55:18.745255+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:55:18.745255+0000","last_undegraded":"2024-09-05T11:55:18.745255+0000","last_fullsized":"2024-09-05T11:55:18.745255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749954+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:55:18.749954+0000","last_peered":"2024-09-05T11:55:18.749954+0000","last_clean":"2024-09-05T11:55:18.749954+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:55:18.749954+0000","last_undegraded":"2024-09-05T11:55:18.749954+0000","last_fullsized":"2024-09-05T11:55:18.749954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744674+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:55:18.744674+0000","last_peered":"2024-09-05T11:55:18.744674+0000","last_clean":"2024-09-05T11:55:18.744674+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:55:18.744674+0000","last_undegraded":"2024-09-05T11:55:18.744674+0000","last_fullsized":"2024-09-05T11:55:18.744674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751392+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:55:18.751392+0000","last_peered":"2024-09-05T11:55:18.751392+0000","last_clean":"2024-09-05T11:55:18.751392+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:55:18.751392+0000","last_undegraded":"2024-09-05T11:55:18.751392+0000","last_fullsized":"2024-09-05T11:55:18.751392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744437+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:55:18.744437+0000","last_peered":"2024-09-05T11:55:18.744437+0000","last_clean":"2024-09-05T11:55:18.744437+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:55:18.744437+0000","last_undegraded":"2024-09-05T11:55:18.744437+0000","last_fullsized":"2024-09-05T11:55:18.744437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744739+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:55:18.744739+0000","last_peered":"2024-09-05T11:55:18.744739+0000","last_clean":"2024-09-05T11:55:18.744739+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:55:18.744739+0000","last_undegraded":"2024-09-05T11:55:18.744739+0000","last_fullsized":"2024-09-05T11:55:18.744739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744301+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:55:18.744301+0000","last_peered":"2024-09-05T11:55:18.744301+0000","last_clean":"2024-09-05T11:55:18.744301+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:55:18.744301+0000","last_undegraded":"2024-09-05T11:55:18.744301+0000","last_fullsized":"2024-09-05T11:55:18.744301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749802+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:55:18.749802+0000","last_peered":"2024-09-05T11:55:18.749802+0000","last_clean":"2024-09-05T11:55:18.749802+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:55:18.749802+0000","last_undegraded":"2024-09-05T11:55:18.749802+0000","last_fullsized":"2024-09-05T11:55:18.749802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749669+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:55:18.749669+0000","last_peered":"2024-09-05T11:55:18.749669+0000","last_clean":"2024-09-05T11:55:18.749669+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:55:18.749669+0000","last_undegraded":"2024-09-05T11:55:18.749669+0000","last_fullsized":"2024-09-05T11:55:18.749669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959105+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:55:18.959105+0000","last_peered":"2024-09-05T11:55:18.959105+0000","last_clean":"2024-09-05T11:55:18.959105+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:55:18.959105+0000","last_undegraded":"2024-09-05T11:55:18.959105+0000","last_fullsized":"2024-09-05T11:55:18.959105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747099+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:55:18.747099+0000","last_peered":"2024-09-05T11:55:18.747099+0000","last_clean":"2024-09-05T11:55:18.747099+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:55:18.747099+0000","last_undegraded":"2024-09-05T11:55:18.747099+0000","last_fullsized":"2024-09-05T11:55:18.747099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749930+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:55:18.749930+0000","last_peered":"2024-09-05T11:55:18.749930+0000","last_clean":"2024-09-05T11:55:18.749930+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:55:18.749930+0000","last_undegraded":"2024-09-05T11:55:18.749930+0000","last_fullsized":"2024-09-05T11:55:18.749930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750328+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:55:18.750328+0000","last_peered":"2024-09-05T11:55:18.750328+0000","last_clean":"2024-09-05T11:55:18.750328+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:55:18.750328+0000","last_undegraded":"2024-09-05T11:55:18.750328+0000","last_fullsized":"2024-09-05T11:55:18.750328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750439+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:55:18.750439+0000","last_peered":"2024-09-05T11:55:18.750439+0000","last_clean":"2024-09-05T11:55:18.750439+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:55:18.750439+0000","last_undegraded":"2024-09-05T11:55:18.750439+0000","last_fullsized":"2024-09-05T11:55:18.750439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745209+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:55:18.745209+0000","last_peered":"2024-09-05T11:55:18.745209+0000","last_clean":"2024-09-05T11:55:18.745209+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:55:18.745209+0000","last_undegraded":"2024-09-05T11:55:18.745209+0000","last_fullsized":"2024-09-05T11:55:18.745209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060892+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:55:19.060892+0000","last_peered":"2024-09-05T11:55:19.060892+0000","last_clean":"2024-09-05T11:55:19.060892+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:55:19.060892+0000","last_undegraded":"2024-09-05T11:55:19.060892+0000","last_fullsized":"2024-09-05T11:55:19.060892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750265+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:55:18.750265+0000","last_peered":"2024-09-05T11:55:18.750265+0000","last_clean":"2024-09-05T11:55:18.750265+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:55:18.750265+0000","last_undegraded":"2024-09-05T11:55:18.750265+0000","last_fullsized":"2024-09-05T11:55:18.750265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750840+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:55:18.750840+0000","last_peered":"2024-09-05T11:55:18.750840+0000","last_clean":"2024-09-05T11:55:18.750840+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:55:18.750840+0000","last_undegraded":"2024-09-05T11:55:18.750840+0000","last_fullsized":"2024-09-05T11:55:18.750840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828776+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:55:18.828776+0000","last_peered":"2024-09-05T11:55:18.828776+0000","last_clean":"2024-09-05T11:55:18.828776+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:55:18.828776+0000","last_undegraded":"2024-09-05T11:55:18.828776+0000","last_fullsized":"2024-09-05T11:55:18.828776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750453+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:55:18.750453+0000","last_peered":"2024-09-05T11:55:18.750453+0000","last_clean":"2024-09-05T11:55:18.750453+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:55:18.750453+0000","last_undegraded":"2024-09-05T11:55:18.750453+0000","last_fullsized":"2024-09-05T11:55:18.750453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745094+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:55:18.745094+0000","last_peered":"2024-09-05T11:55:18.745094+0000","last_clean":"2024-09-05T11:55:18.745094+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:55:18.745094+0000","last_undegraded":"2024-09-05T11:55:18.745094+0000","last_fullsized":"2024-09-05T11:55:18.745094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749801+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:55:18.749801+0000","last_peered":"2024-09-05T11:55:18.749801+0000","last_clean":"2024-09-05T11:55:18.749801+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:55:18.749801+0000","last_undegraded":"2024-09-05T11:55:18.749801+0000","last_fullsized":"2024-09-05T11:55:18.749801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749962+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:55:18.749962+0000","last_peered":"2024-09-05T11:55:18.749962+0000","last_clean":"2024-09-05T11:55:18.749962+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:55:18.749962+0000","last_undegraded":"2024-09-05T11:55:18.749962+0000","last_fullsized":"2024-09-05T11:55:18.749962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749212+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:55:18.749212+0000","last_peered":"2024-09-05T11:55:18.749212+0000","last_clean":"2024-09-05T11:55:18.749212+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:55:18.749212+0000","last_undegraded":"2024-09-05T11:55:18.749212+0000","last_fullsized":"2024-09-05T11:55:18.749212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748596+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:55:18.748596+0000","last_peered":"2024-09-05T11:55:18.748596+0000","last_clean":"2024-09-05T11:55:18.748596+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:55:18.748596+0000","last_undegraded":"2024-09-05T11:55:18.748596+0000","last_fullsized":"2024-09-05T11:55:18.748596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749735+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:55:18.749735+0000","last_peered":"2024-09-05T11:55:18.749735+0000","last_clean":"2024-09-05T11:55:18.749735+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:55:18.749735+0000","last_undegraded":"2024-09-05T11:55:18.749735+0000","last_fullsized":"2024-09-05T11:55:18.749735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751378+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:55:18.751378+0000","last_peered":"2024-09-05T11:55:18.751378+0000","last_clean":"2024-09-05T11:55:18.751378+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:55:18.751378+0000","last_undegraded":"2024-09-05T11:55:18.751378+0000","last_fullsized":"2024-09-05T11:55:18.751378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744997+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:55:18.744997+0000","last_peered":"2024-09-05T11:55:18.744997+0000","last_clean":"2024-09-05T11:55:18.744997+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:55:18.744997+0000","last_undegraded":"2024-09-05T11:55:18.744997+0000","last_fullsized":"2024-09-05T11:55:18.744997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749638+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:55:18.749638+0000","last_peered":"2024-09-05T11:55:18.749638+0000","last_clean":"2024-09-05T11:55:18.749638+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:55:18.749638+0000","last_undegraded":"2024-09-05T11:55:18.749638+0000","last_fullsized":"2024-09-05T11:55:18.749638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747345+0000","last_change":"2024-09-05T11:54:29.102616+0000","last_active":"2024-09-05T11:55:18.747345+0000","last_peered":"2024-09-05T11:55:18.747345+0000","last_clean":"2024-09-05T11:55:18.747345+0000","last_became_active":"2024-09-05T11:54:29.102126+0000","last_became_peered":"2024-09-05T11:54:29.102126+0000","last_unstale":"2024-09-05T11:55:18.747345+0000","last_undegraded":"2024-09-05T11:55:18.747345+0000","last_fullsized":"2024-09-05T11:55:18.747345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:09:28.848395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748687+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:55:18.748687+0000","last_peered":"2024-09-05T11:55:18.748687+0000","last_clean":"2024-09-05T11:55:18.748687+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:55:18.748687+0000","last_undegraded":"2024-09-05T11:55:18.748687+0000","last_fullsized":"2024-09-05T11:55:18.748687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750129+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:55:18.750129+0000","last_peered":"2024-09-05T11:55:18.750129+0000","last_clean":"2024-09-05T11:55:18.750129+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:55:18.750129+0000","last_undegraded":"2024-09-05T11:55:18.750129+0000","last_fullsized":"2024-09-05T11:55:18.750129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751077+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:55:18.751077+0000","last_peered":"2024-09-05T11:55:18.751077+0000","last_clean":"2024-09-05T11:55:18.751077+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:55:18.751077+0000","last_undegraded":"2024-09-05T11:55:18.751077+0000","last_fullsized":"2024-09-05T11:55:18.751077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748683+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:55:18.748683+0000","last_peered":"2024-09-05T11:55:18.748683+0000","last_clean":"2024-09-05T11:55:18.748683+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:55:18.748683+0000","last_undegraded":"2024-09-05T11:55:18.748683+0000","last_fullsized":"2024-09-05T11:55:18.748683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744841+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:55:18.744841+0000","last_peered":"2024-09-05T11:55:18.744841+0000","last_clean":"2024-09-05T11:55:18.744841+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:55:18.744841+0000","last_undegraded":"2024-09-05T11:55:18.744841+0000","last_fullsized":"2024-09-05T11:55:18.744841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744915+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:55:18.744915+0000","last_peered":"2024-09-05T11:55:18.744915+0000","last_clean":"2024-09-05T11:55:18.744915+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:55:18.744915+0000","last_undegraded":"2024-09-05T11:55:18.744915+0000","last_fullsized":"2024-09-05T11:55:18.744915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750719+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:55:18.750719+0000","last_peered":"2024-09-05T11:55:18.750719+0000","last_clean":"2024-09-05T11:55:18.750719+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:55:18.750719+0000","last_undegraded":"2024-09-05T11:55:18.750719+0000","last_fullsized":"2024-09-05T11:55:18.750719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749744+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:55:18.749744+0000","last_peered":"2024-09-05T11:55:18.749744+0000","last_clean":"2024-09-05T11:55:18.749744+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:55:18.749744+0000","last_undegraded":"2024-09-05T11:55:18.749744+0000","last_fullsized":"2024-09-05T11:55:18.749744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:29.753930+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:55:29.753930+0000","last_peered":"2024-09-05T11:55:29.753930+0000","last_clean":"2024-09-05T11:55:29.753930+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:55:29.753930+0000","last_undegraded":"2024-09-05T11:55:29.753930+0000","last_fullsized":"2024-09-05T11:55:29.753930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59728,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":75,"num_write":30,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747427+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:55:18.747427+0000","last_peered":"2024-09-05T11:55:18.747427+0000","last_clean":"2024-09-05T11:55:18.747427+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:55:18.747427+0000","last_undegraded":"2024-09-05T11:55:18.747427+0000","last_fullsized":"2024-09-05T11:55:18.747427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061114+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:55:19.061114+0000","last_peered":"2024-09-05T11:55:19.061114+0000","last_clean":"2024-09-05T11:55:19.061114+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:55:19.061114+0000","last_undegraded":"2024-09-05T11:55:19.061114+0000","last_fullsized":"2024-09-05T11:55:19.061114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750985+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:55:18.750985+0000","last_peered":"2024-09-05T11:55:18.750985+0000","last_clean":"2024-09-05T11:55:18.750985+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:55:18.750985+0000","last_undegraded":"2024-09-05T11:55:18.750985+0000","last_fullsized":"2024-09-05T11:55:18.750985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746602+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:55:18.746602+0000","last_peered":"2024-09-05T11:55:18.746602+0000","last_clean":"2024-09-05T11:55:18.746602+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:55:18.746602+0000","last_undegraded":"2024-09-05T11:55:18.746602+0000","last_fullsized":"2024-09-05T11:55:18.746602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829307+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:55:18.829307+0000","last_peered":"2024-09-05T11:55:18.829307+0000","last_clean":"2024-09-05T11:55:18.829307+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:55:18.829307+0000","last_undegraded":"2024-09-05T11:55:18.829307+0000","last_fullsized":"2024-09-05T11:55:18.829307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750531+0000","last_change":"2024-09-05T11:54:29.106596+0000","last_active":"2024-09-05T11:55:18.750531+0000","last_peered":"2024-09-05T11:55:18.750531+0000","last_clean":"2024-09-05T11:55:18.750531+0000","last_became_active":"2024-09-05T11:54:29.106213+0000","last_became_peered":"2024-09-05T11:54:29.106213+0000","last_unstale":"2024-09-05T11:55:18.750531+0000","last_undegraded":"2024-09-05T11:55:18.750531+0000","last_fullsized":"2024-09-05T11:55:18.750531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:24:44.592047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747050+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:55:18.747050+0000","last_peered":"2024-09-05T11:55:18.747050+0000","last_clean":"2024-09-05T11:55:18.747050+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:55:18.747050+0000","last_undegraded":"2024-09-05T11:55:18.747050+0000","last_fullsized":"2024-09-05T11:55:18.747050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061360+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:55:19.061360+0000","last_peered":"2024-09-05T11:55:19.061360+0000","last_clean":"2024-09-05T11:55:19.061360+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:55:19.061360+0000","last_undegraded":"2024-09-05T11:55:19.061360+0000","last_fullsized":"2024-09-05T11:55:19.061360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745488+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:55:18.745488+0000","last_peered":"2024-09-05T11:55:18.745488+0000","last_clean":"2024-09-05T11:55:18.745488+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:55:18.745488+0000","last_undegraded":"2024-09-05T11:55:18.745488+0000","last_fullsized":"2024-09-05T11:55:18.745488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959078+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:55:18.959078+0000","last_peered":"2024-09-05T11:55:18.959078+0000","last_clean":"2024-09-05T11:55:18.959078+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:55:18.959078+0000","last_undegraded":"2024-09-05T11:55:18.959078+0000","last_fullsized":"2024-09-05T11:55:18.959078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829040+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:55:18.829040+0000","last_peered":"2024-09-05T11:55:18.829040+0000","last_clean":"2024-09-05T11:55:18.829040+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:55:18.829040+0000","last_undegraded":"2024-09-05T11:55:18.829040+0000","last_fullsized":"2024-09-05T11:55:18.829040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747393+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:55:18.747393+0000","last_peered":"2024-09-05T11:55:18.747393+0000","last_clean":"2024-09-05T11:55:18.747393+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:55:18.747393+0000","last_undegraded":"2024-09-05T11:55:18.747393+0000","last_fullsized":"2024-09-05T11:55:18.747393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751542+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:55:18.751542+0000","last_peered":"2024-09-05T11:55:18.751542+0000","last_clean":"2024-09-05T11:55:18.751542+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:55:18.751542+0000","last_undegraded":"2024-09-05T11:55:18.751542+0000","last_fullsized":"2024-09-05T11:55:18.751542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749841+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:55:18.749841+0000","last_peered":"2024-09-05T11:55:18.749841+0000","last_clean":"2024-09-05T11:55:18.749841+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:55:18.749841+0000","last_undegraded":"2024-09-05T11:55:18.749841+0000","last_fullsized":"2024-09-05T11:55:18.749841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959536+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:55:18.959536+0000","last_peered":"2024-09-05T11:55:18.959536+0000","last_clean":"2024-09-05T11:55:18.959536+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:55:18.959536+0000","last_undegraded":"2024-09-05T11:55:18.959536+0000","last_fullsized":"2024-09-05T11:55:18.959536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744321+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:55:18.744321+0000","last_peered":"2024-09-05T11:55:18.744321+0000","last_clean":"2024-09-05T11:55:18.744321+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:55:18.744321+0000","last_undegraded":"2024-09-05T11:55:18.744321+0000","last_fullsized":"2024-09-05T11:55:18.744321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745069+0000","last_change":"2024-09-05T11:54:29.101408+0000","last_active":"2024-09-05T11:55:18.745069+0000","last_peered":"2024-09-05T11:55:18.745069+0000","last_clean":"2024-09-05T11:55:18.745069+0000","last_became_active":"2024-09-05T11:54:29.101052+0000","last_became_peered":"2024-09-05T11:54:29.101052+0000","last_unstale":"2024-09-05T11:55:18.745069+0000","last_undegraded":"2024-09-05T11:55:18.745069+0000","last_fullsized":"2024-09-05T11:55:18.745069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:40.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960761+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:55:18.960761+0000","last_peered":"2024-09-05T11:55:18.960761+0000","last_clean":"2024-09-05T11:55:18.960761+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:55:18.960761+0000","last_undegraded":"2024-09-05T11:55:18.960761+0000","last_fullsized":"2024-09-05T11:55:18.960761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744820+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:55:18.744820+0000","last_peered":"2024-09-05T11:55:18.744820+0000","last_clean":"2024-09-05T11:55:18.744820+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:55:18.744820+0000","last_undegraded":"2024-09-05T11:55:18.744820+0000","last_fullsized":"2024-09-05T11:55:18.744820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750928+0000","last_change":"2024-09-05T11:54:29.102371+0000","last_active":"2024-09-05T11:55:18.750928+0000","last_peered":"2024-09-05T11:55:18.750928+0000","last_clean":"2024-09-05T11:55:18.750928+0000","last_became_active":"2024-09-05T11:54:29.102066+0000","last_became_peered":"2024-09-05T11:54:29.102066+0000","last_unstale":"2024-09-05T11:55:18.750928+0000","last_undegraded":"2024-09-05T11:55:18.750928+0000","last_fullsized":"2024-09-05T11:55:18.750928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:55.008088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754810+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:55:18.754810+0000","last_peered":"2024-09-05T11:55:18.754810+0000","last_clean":"2024-09-05T11:55:18.754810+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:55:18.754810+0000","last_undegraded":"2024-09-05T11:55:18.754810+0000","last_fullsized":"2024-09-05T11:55:18.754810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744917+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:55:18.744917+0000","last_peered":"2024-09-05T11:55:18.744917+0000","last_clean":"2024-09-05T11:55:18.744917+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:55:18.744917+0000","last_undegraded":"2024-09-05T11:55:18.744917+0000","last_fullsized":"2024-09-05T11:55:18.744917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748298+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:55:18.748298+0000","last_peered":"2024-09-05T11:55:18.748298+0000","last_clean":"2024-09-05T11:55:18.748298+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:55:18.748298+0000","last_undegraded":"2024-09-05T11:55:18.748298+0000","last_fullsized":"2024-09-05T11:55:18.748298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745098+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:55:18.745098+0000","last_peered":"2024-09-05T11:55:18.745098+0000","last_clean":"2024-09-05T11:55:18.745098+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:55:18.745098+0000","last_undegraded":"2024-09-05T11:55:18.745098+0000","last_fullsized":"2024-09-05T11:55:18.745098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744606+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:55:18.744606+0000","last_peered":"2024-09-05T11:55:18.744606+0000","last_clean":"2024-09-05T11:55:18.744606+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:55:18.744606+0000","last_undegraded":"2024-09-05T11:55:18.744606+0000","last_fullsized":"2024-09-05T11:55:18.744606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828789+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:55:18.828789+0000","last_peered":"2024-09-05T11:55:18.828789+0000","last_clean":"2024-09-05T11:55:18.828789+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:55:18.828789+0000","last_undegraded":"2024-09-05T11:55:18.828789+0000","last_fullsized":"2024-09-05T11:55:18.828789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744825+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:55:18.744825+0000","last_peered":"2024-09-05T11:55:18.744825+0000","last_clean":"2024-09-05T11:55:18.744825+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:55:18.744825+0000","last_undegraded":"2024-09-05T11:55:18.744825+0000","last_fullsized":"2024-09-05T11:55:18.744825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749366+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:55:18.749366+0000","last_peered":"2024-09-05T11:55:18.749366+0000","last_clean":"2024-09-05T11:55:18.749366+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:55:18.749366+0000","last_undegraded":"2024-09-05T11:55:18.749366+0000","last_fullsized":"2024-09-05T11:55:18.749366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745046+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:55:18.745046+0000","last_peered":"2024-09-05T11:55:18.745046+0000","last_clean":"2024-09-05T11:55:18.745046+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:55:18.745046+0000","last_undegraded":"2024-09-05T11:55:18.745046+0000","last_fullsized":"2024-09-05T11:55:18.745046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959692+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:55:18.959692+0000","last_peered":"2024-09-05T11:55:18.959692+0000","last_clean":"2024-09-05T11:55:18.959692+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:55:18.959692+0000","last_undegraded":"2024-09-05T11:55:18.959692+0000","last_fullsized":"2024-09-05T11:55:18.959692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751303+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:55:18.751303+0000","last_peered":"2024-09-05T11:55:18.751303+0000","last_clean":"2024-09-05T11:55:18.751303+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:55:18.751303+0000","last_undegraded":"2024-09-05T11:55:18.751303+0000","last_fullsized":"2024-09-05T11:55:18.751303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959252+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:55:18.959252+0000","last_peered":"2024-09-05T11:55:18.959252+0000","last_clean":"2024-09-05T11:55:18.959252+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:55:18.959252+0000","last_undegraded":"2024-09-05T11:55:18.959252+0000","last_fullsized":"2024-09-05T11:55:18.959252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061014+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:55:19.061014+0000","last_peered":"2024-09-05T11:55:19.061014+0000","last_clean":"2024-09-05T11:55:19.061014+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:55:19.061014+0000","last_undegraded":"2024-09-05T11:55:19.061014+0000","last_fullsized":"2024-09-05T11:55:19.061014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744715+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:55:18.744715+0000","last_peered":"2024-09-05T11:55:18.744715+0000","last_clean":"2024-09-05T11:55:18.744715+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:55:18.744715+0000","last_undegraded":"2024-09-05T11:55:18.744715+0000","last_fullsized":"2024-09-05T11:55:18.744715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754299+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:55:18.754299+0000","last_peered":"2024-09-05T11:55:18.754299+0000","last_clean":"2024-09-05T11:55:18.754299+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:55:18.754299+0000","last_undegraded":"2024-09-05T11:55:18.754299+0000","last_fullsized":"2024-09-05T11:55:18.754299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745216+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:55:18.745216+0000","last_peered":"2024-09-05T11:55:18.745216+0000","last_clean":"2024-09-05T11:55:18.745216+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:55:18.745216+0000","last_undegraded":"2024-09-05T11:55:18.745216+0000","last_fullsized":"2024-09-05T11:55:18.745216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748542+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:55:18.748542+0000","last_peered":"2024-09-05T11:55:18.748542+0000","last_clean":"2024-09-05T11:55:18.748542+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:55:18.748542+0000","last_undegraded":"2024-09-05T11:55:18.748542+0000","last_fullsized":"2024-09-05T11:55:18.748542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750435+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:55:18.750435+0000","last_peered":"2024-09-05T11:55:18.750435+0000","last_clean":"2024-09-05T11:55:18.750435+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:55:18.750435+0000","last_undegraded":"2024-09-05T11:55:18.750435+0000","last_fullsized":"2024-09-05T11:55:18.750435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060771+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:55:19.060771+0000","last_peered":"2024-09-05T11:55:19.060771+0000","last_clean":"2024-09-05T11:55:19.060771+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:55:19.060771+0000","last_undegraded":"2024-09-05T11:55:19.060771+0000","last_fullsized":"2024-09-05T11:55:19.060771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828621+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:55:18.828621+0000","last_peered":"2024-09-05T11:55:18.828621+0000","last_clean":"2024-09-05T11:55:18.828621+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:55:18.828621+0000","last_undegraded":"2024-09-05T11:55:18.828621+0000","last_fullsized":"2024-09-05T11:55:18.828621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828531+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:55:18.828531+0000","last_peered":"2024-09-05T11:55:18.828531+0000","last_clean":"2024-09-05T11:55:18.828531+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:55:18.828531+0000","last_undegraded":"2024-09-05T11:55:18.828531+0000","last_fullsized":"2024-09-05T11:55:18.828531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744676+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:55:18.744676+0000","last_peered":"2024-09-05T11:55:18.744676+0000","last_clean":"2024-09-05T11:55:18.744676+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:55:18.744676+0000","last_undegraded":"2024-09-05T11:55:18.744676+0000","last_fullsized":"2024-09-05T11:55:18.744676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747214+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:55:18.747214+0000","last_peered":"2024-09-05T11:55:18.747214+0000","last_clean":"2024-09-05T11:55:18.747214+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:55:18.747214+0000","last_undegraded":"2024-09-05T11:55:18.747214+0000","last_fullsized":"2024-09-05T11:55:18.747214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750519+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:55:18.750519+0000","last_peered":"2024-09-05T11:55:18.750519+0000","last_clean":"2024-09-05T11:55:18.750519+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:55:18.750519+0000","last_undegraded":"2024-09-05T11:55:18.750519+0000","last_fullsized":"2024-09-05T11:55:18.750519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745888+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:55:18.745888+0000","last_peered":"2024-09-05T11:55:18.745888+0000","last_clean":"2024-09-05T11:55:18.745888+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:55:18.745888+0000","last_undegraded":"2024-09-05T11:55:18.745888+0000","last_fullsized":"2024-09-05T11:55:18.745888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061418+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:55:19.061418+0000","last_peered":"2024-09-05T11:55:19.061418+0000","last_clean":"2024-09-05T11:55:19.061418+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:55:19.061418+0000","last_undegraded":"2024-09-05T11:55:19.061418+0000","last_fullsized":"2024-09-05T11:55:19.061418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753244+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:55:18.753244+0000","last_peered":"2024-09-05T11:55:18.753244+0000","last_clean":"2024-09-05T11:55:18.753244+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:55:18.753244+0000","last_undegraded":"2024-09-05T11:55:18.753244+0000","last_fullsized":"2024-09-05T11:55:18.753244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754442+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:55:18.754442+0000","last_peered":"2024-09-05T11:55:18.754442+0000","last_clean":"2024-09-05T11:55:18.754442+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:55:18.754442+0000","last_undegraded":"2024-09-05T11:55:18.754442+0000","last_fullsized":"2024-09-05T11:55:18.754442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751541+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:55:18.751541+0000","last_peered":"2024-09-05T11:55:18.751541+0000","last_clean":"2024-09-05T11:55:18.751541+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:55:18.751541+0000","last_undegraded":"2024-09-05T11:55:18.751541+0000","last_fullsized":"2024-09-05T11:55:18.751541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748945+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:55:18.748945+0000","last_peered":"2024-09-05T11:55:18.748945+0000","last_clean":"2024-09-05T11:55:18.748945+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:55:18.748945+0000","last_undegraded":"2024-09-05T11:55:18.748945+0000","last_fullsized":"2024-09-05T11:55:18.748945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.752031+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:55:18.752031+0000","last_peered":"2024-09-05T11:55:18.752031+0000","last_clean":"2024-09-05T11:55:18.752031+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:55:18.752031+0000","last_undegraded":"2024-09-05T11:55:18.752031+0000","last_fullsized":"2024-09-05T11:55:18.752031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828990+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:55:18.828990+0000","last_peered":"2024-09-05T11:55:18.828990+0000","last_clean":"2024-09-05T11:55:18.828990+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:55:18.828990+0000","last_undegraded":"2024-09-05T11:55:18.828990+0000","last_fullsized":"2024-09-05T11:55:18.828990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829035+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:55:18.829035+0000","last_peered":"2024-09-05T11:55:18.829035+0000","last_clean":"2024-09-05T11:55:18.829035+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:55:18.829035+0000","last_undegraded":"2024-09-05T11:55:18.829035+0000","last_fullsized":"2024-09-05T11:55:18.829035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750338+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:55:18.750338+0000","last_peered":"2024-09-05T11:55:18.750338+0000","last_clean":"2024-09-05T11:55:18.750338+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:55:18.750338+0000","last_undegraded":"2024-09-05T11:55:18.750338+0000","last_fullsized":"2024-09-05T11:55:18.750338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60526,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":49,"num_read_kb":81,"num_write":82,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":173048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":513188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1703936,"data_stored":1159213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1108234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":509092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1638400,"data_stored":1103738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":509196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":564589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":564555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":55497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:55:32.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:55:32.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:32 smithi059 ceph-mon[30025]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 5 op/s 2024-09-05T11:55:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:32 smithi189 ceph-mon[34772]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 5 op/s 2024-09-05T11:55:33.090 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:32 smithi089 ceph-mon[34499]: pgmap v328: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 5 op/s 2024-09-05T11:55:33.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 -- 172.21.15.59:0/513936104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 msgr2=0x7fd1cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 --2- 172.21.15.59:0/513936104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc104210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009900 tx=0x7fd1c002a410 comp rx=0 tx=0).stop 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 -- 172.21.15.59:0/513936104 shutdown_connections 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 --2- 172.21.15.59:0/513936104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc108f60 0x7fd1cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 --2- 172.21.15.59:0/513936104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 --2- 172.21.15.59:0/513936104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc104210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.653+0000 7fd1d2600640 1 -- 172.21.15.59:0/513936104 >> 172.21.15.59:0/513936104 conn(0x7fd1cc0ff8d0 msgr2=0x7fd1cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.654+0000 7fd1d2600640 1 -- 172.21.15.59:0/513936104 shutdown_connections 2024-09-05T11:55:33.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.654+0000 7fd1d2600640 1 -- 172.21.15.59:0/513936104 wait complete. 2024-09-05T11:55:33.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 Processor -- start 2024-09-05T11:55:33.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 -- start start 2024-09-05T11:55:33.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:33.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1cc108f60 0x7fd1cc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113f80 con 0x7fd1cc103e30 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113e00 con 0x7fd1cc1047e0 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d2600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc114100 con 0x7fd1cc108f60 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42150/0 (socket says 172.21.15.59:42150) 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.655+0000 7fd1d1200640 1 -- 172.21.15.59:0/1920981864 learned_addr learned my addr 172.21.15.59:0/1920981864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1cbe00640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 -- 172.21.15.59:0/1920981864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1cc108f60 msgr2=0x7fd1cc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1cc108f60 0x7fd1cc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 -- 172.21.15.59:0/1920981864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 msgr2=0x7fd1cc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:33.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 -- 172.21.15.59:0/1920981864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc19f410 con 0x7fd1cc103e30 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1cbe00640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.656+0000 7fd1d1200640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0006010 tx=0x7fd1c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.657+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c003f030 con 0x7fd1cc103e30 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.657+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0004440 con 0x7fd1cc103e30 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.657+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc19f6a0 con 0x7fd1cc103e30 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.657+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0035070 con 0x7fd1cc103e30 2024-09-05T11:55:33.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.657+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a6fe0 con 0x7fd1cc103e30 2024-09-05T11:55:33.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.658+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc104210 con 0x7fd1cc103e30 2024-09-05T11:55:33.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.660+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd1c0042030 con 0x7fd1cc103e30 2024-09-05T11:55:33.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.661+0000 7fd1c9600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 0x7fd1b40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:33.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.661+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd1c00ef910 con 0x7fd1cc103e30 2024-09-05T11:55:33.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.661+0000 7fd1cbe00640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 0x7fd1b40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:33.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.662+0000 7fd1cbe00640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 0x7fd1b40a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0ffcf0 tx=0x7fd1b8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:33.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.665+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1c00bc350 con 0x7fd1cc103e30 2024-09-05T11:55:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.849+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1cc062630 con 0x7fd1b40a2c90 2024-09-05T11:55:33.850 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:33 smithi059 ceph-mon[30025]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.857+0000 7fd1c9600640 1 -- 172.21.15.59:0/1920981864 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340291 (secure 0 0 0) 0x7fd1cc062630 con 0x7fd1b40a2c90 2024-09-05T11:55:33.863 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.862+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 msgr2=0x7fd1b40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.862+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 0x7fd1b40a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0ffcf0 tx=0x7fd1b8005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.862+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 msgr2=0x7fd1cc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.862+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0006010 tx=0x7fd1c0004040 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 shutdown_connections 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1b40a2c90 0x7fd1b40a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1cc108f60 0x7fd1cc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1cc1047e0 0x7fd1cc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 --2- 172.21.15.59:0/1920981864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1cc103e30 0x7fd1cc19a460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 >> 172.21.15.59:0/1920981864 conn(0x7fd1cc0ff8d0 msgr2=0x7fd1cc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 shutdown_connections 2024-09-05T11:55:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:33.863+0000 7fd1d2600640 1 -- 172.21.15.59:0/1920981864 wait complete. 2024-09-05T11:55:34.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:33 smithi189 ceph-mon[34772]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:33 smithi089 ceph-mon[34499]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:34.685 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-05T11:55:32.865242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651128,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":175,"num_read_kb":190,"num_write":265,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375824,"kb_used_data":14080,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566000,"statfs":{"total":1151940427776,"available":1151555584000,"internally_reserved":0,"allocated":14417920,"data_stored":8090289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30634,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":41,"num_write":23,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748911+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:55:18.748911+0000","last_peered":"2024-09-05T11:55:18.748911+0000","last_clean":"2024-09-05T11:55:18.748911+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:55:18.748911+0000","last_undegraded":"2024-09-05T11:55:18.748911+0000","last_fullsized":"2024-09-05T11:55:18.748911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959651+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:55:18.959651+0000","last_peered":"2024-09-05T11:55:18.959651+0000","last_clean":"2024-09-05T11:55:18.959651+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:55:18.959651+0000","last_undegraded":"2024-09-05T11:55:18.959651+0000","last_fullsized":"2024-09-05T11:55:18.959651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960937+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:55:18.960937+0000","last_peered":"2024-09-05T11:55:18.960937+0000","last_clean":"2024-09-05T11:55:18.960937+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:55:18.960937+0000","last_undegraded":"2024-09-05T11:55:18.960937+0000","last_fullsized":"2024-09-05T11:55:18.960937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959828+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:55:18.959828+0000","last_peered":"2024-09-05T11:55:18.959828+0000","last_clean":"2024-09-05T11:55:18.959828+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:55:18.959828+0000","last_undegraded":"2024-09-05T11:55:18.959828+0000","last_fullsized":"2024-09-05T11:55:18.959828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829255+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:55:18.829255+0000","last_peered":"2024-09-05T11:55:18.829255+0000","last_clean":"2024-09-05T11:55:18.829255+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:55:18.829255+0000","last_undegraded":"2024-09-05T11:55:18.829255+0000","last_fullsized":"2024-09-05T11:55:18.829255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960868+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:55:18.960868+0000","last_peered":"2024-09-05T11:55:18.960868+0000","last_clean":"2024-09-05T11:55:18.960868+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:55:18.960868+0000","last_undegraded":"2024-09-05T11:55:18.960868+0000","last_fullsized":"2024-09-05T11:55:18.960868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750148+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:55:18.750148+0000","last_peered":"2024-09-05T11:55:18.750148+0000","last_clean":"2024-09-05T11:55:18.750148+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:55:18.750148+0000","last_undegraded":"2024-09-05T11:55:18.750148+0000","last_fullsized":"2024-09-05T11:55:18.750148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061069+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:55:19.061069+0000","last_peered":"2024-09-05T11:55:19.061069+0000","last_clean":"2024-09-05T11:55:19.061069+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:55:19.061069+0000","last_undegraded":"2024-09-05T11:55:19.061069+0000","last_fullsized":"2024-09-05T11:55:19.061069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746730+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:55:18.746730+0000","last_peered":"2024-09-05T11:55:18.746730+0000","last_clean":"2024-09-05T11:55:18.746730+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:55:18.746730+0000","last_undegraded":"2024-09-05T11:55:18.746730+0000","last_fullsized":"2024-09-05T11:55:18.746730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750651+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:55:18.750651+0000","last_peered":"2024-09-05T11:55:18.750651+0000","last_clean":"2024-09-05T11:55:18.750651+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:55:18.750651+0000","last_undegraded":"2024-09-05T11:55:18.750651+0000","last_fullsized":"2024-09-05T11:55:18.750651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749383+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:55:18.749383+0000","last_peered":"2024-09-05T11:55:18.749383+0000","last_clean":"2024-09-05T11:55:18.749383+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:55:18.749383+0000","last_undegraded":"2024-09-05T11:55:18.749383+0000","last_fullsized":"2024-09-05T11:55:18.749383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746451+0000","last_change":"2024-09-05T11:54:38.607159+0000","last_active":"2024-09-05T11:55:18.746451+0000","last_peered":"2024-09-05T11:55:18.746451+0000","last_clean":"2024-09-05T11:55:18.746451+0000","last_became_active":"2024-09-05T11:54:38.606569+0000","last_became_peered":"2024-09-05T11:54:38.606569+0000","last_unstale":"2024-09-05T11:55:18.746451+0000","last_undegraded":"2024-09-05T11:55:18.746451+0000","last_fullsized":"2024-09-05T11:55:18.746451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:28:07.583731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749879+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:55:18.749879+0000","last_peered":"2024-09-05T11:55:18.749879+0000","last_clean":"2024-09-05T11:55:18.749879+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:55:18.749879+0000","last_undegraded":"2024-09-05T11:55:18.749879+0000","last_fullsized":"2024-09-05T11:55:18.749879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749076+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:55:18.749076+0000","last_peered":"2024-09-05T11:55:18.749076+0000","last_clean":"2024-09-05T11:55:18.749076+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:55:18.749076+0000","last_undegraded":"2024-09-05T11:55:18.749076+0000","last_fullsized":"2024-09-05T11:55:18.749076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746802+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:55:18.746802+0000","last_peered":"2024-09-05T11:55:18.746802+0000","last_clean":"2024-09-05T11:55:18.746802+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:55:18.746802+0000","last_undegraded":"2024-09-05T11:55:18.746802+0000","last_fullsized":"2024-09-05T11:55:18.746802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959081+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:55:18.959081+0000","last_peered":"2024-09-05T11:55:18.959081+0000","last_clean":"2024-09-05T11:55:18.959081+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:55:18.959081+0000","last_undegraded":"2024-09-05T11:55:18.959081+0000","last_fullsized":"2024-09-05T11:55:18.959081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744681+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:55:18.744681+0000","last_peered":"2024-09-05T11:55:18.744681+0000","last_clean":"2024-09-05T11:55:18.744681+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:55:18.744681+0000","last_undegraded":"2024-09-05T11:55:18.744681+0000","last_fullsized":"2024-09-05T11:55:18.744681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753048+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:55:18.753048+0000","last_peered":"2024-09-05T11:55:18.753048+0000","last_clean":"2024-09-05T11:55:18.753048+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:55:18.753048+0000","last_undegraded":"2024-09-05T11:55:18.753048+0000","last_fullsized":"2024-09-05T11:55:18.753048+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748420+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:55:18.748420+0000","last_peered":"2024-09-05T11:55:18.748420+0000","last_clean":"2024-09-05T11:55:18.748420+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:55:18.748420+0000","last_undegraded":"2024-09-05T11:55:18.748420+0000","last_fullsized":"2024-09-05T11:55:18.748420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959409+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:55:18.959409+0000","last_peered":"2024-09-05T11:55:18.959409+0000","last_clean":"2024-09-05T11:55:18.959409+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:55:18.959409+0000","last_undegraded":"2024-09-05T11:55:18.959409+0000","last_fullsized":"2024-09-05T11:55:18.959409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748594+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:55:18.748594+0000","last_peered":"2024-09-05T11:55:18.748594+0000","last_clean":"2024-09-05T11:55:18.748594+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:55:18.748594+0000","last_undegraded":"2024-09-05T11:55:18.748594+0000","last_fullsized":"2024-09-05T11:55:18.748594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751886+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:55:18.751886+0000","last_peered":"2024-09-05T11:55:18.751886+0000","last_clean":"2024-09-05T11:55:18.751886+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:55:18.751886+0000","last_undegraded":"2024-09-05T11:55:18.751886+0000","last_fullsized":"2024-09-05T11:55:18.751886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060880+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:55:19.060880+0000","last_peered":"2024-09-05T11:55:19.060880+0000","last_clean":"2024-09-05T11:55:19.060880+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:55:19.060880+0000","last_undegraded":"2024-09-05T11:55:19.060880+0000","last_fullsized":"2024-09-05T11:55:19.060880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746843+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:55:18.746843+0000","last_peered":"2024-09-05T11:55:18.746843+0000","last_clean":"2024-09-05T11:55:18.746843+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:55:18.746843+0000","last_undegraded":"2024-09-05T11:55:18.746843+0000","last_fullsized":"2024-09-05T11:55:18.746843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829155+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:55:18.829155+0000","last_peered":"2024-09-05T11:55:18.829155+0000","last_clean":"2024-09-05T11:55:18.829155+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:55:18.829155+0000","last_undegraded":"2024-09-05T11:55:18.829155+0000","last_fullsized":"2024-09-05T11:55:18.829155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747880+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:55:18.747880+0000","last_peered":"2024-09-05T11:55:18.747880+0000","last_clean":"2024-09-05T11:55:18.747880+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:55:18.747880+0000","last_undegraded":"2024-09-05T11:55:18.747880+0000","last_fullsized":"2024-09-05T11:55:18.747880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744630+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:55:18.744630+0000","last_peered":"2024-09-05T11:55:18.744630+0000","last_clean":"2024-09-05T11:55:18.744630+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:55:18.744630+0000","last_undegraded":"2024-09-05T11:55:18.744630+0000","last_fullsized":"2024-09-05T11:55:18.744630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959726+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:55:18.959726+0000","last_peered":"2024-09-05T11:55:18.959726+0000","last_clean":"2024-09-05T11:55:18.959726+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:55:18.959726+0000","last_undegraded":"2024-09-05T11:55:18.959726+0000","last_fullsized":"2024-09-05T11:55:18.959726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:25.561771+0000","last_change":"2024-09-05T11:54:29.103741+0000","last_active":"2024-09-05T11:55:25.561771+0000","last_peered":"2024-09-05T11:55:25.561771+0000","last_clean":"2024-09-05T11:55:25.561771+0000","last_became_active":"2024-09-05T11:54:29.103285+0000","last_became_peered":"2024-09-05T11:54:29.103285+0000","last_unstale":"2024-09-05T11:55:25.561771+0000","last_undegraded":"2024-09-05T11:55:25.561771+0000","last_fullsized":"2024-09-05T11:55:25.561771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:01:16.713162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.958714+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:55:18.958714+0000","last_peered":"2024-09-05T11:55:18.958714+0000","last_clean":"2024-09-05T11:55:18.958714+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:55:18.958714+0000","last_undegraded":"2024-09-05T11:55:18.958714+0000","last_fullsized":"2024-09-05T11:55:18.958714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744903+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:55:18.744903+0000","last_peered":"2024-09-05T11:55:18.744903+0000","last_clean":"2024-09-05T11:55:18.744903+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:55:18.744903+0000","last_undegraded":"2024-09-05T11:55:18.744903+0000","last_fullsized":"2024-09-05T11:55:18.744903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750269+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:55:18.750269+0000","last_peered":"2024-09-05T11:55:18.750269+0000","last_clean":"2024-09-05T11:55:18.750269+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:55:18.750269+0000","last_undegraded":"2024-09-05T11:55:18.750269+0000","last_fullsized":"2024-09-05T11:55:18.750269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748452+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:55:18.748452+0000","last_peered":"2024-09-05T11:55:18.748452+0000","last_clean":"2024-09-05T11:55:18.748452+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:55:18.748452+0000","last_undegraded":"2024-09-05T11:55:18.748452+0000","last_fullsized":"2024-09-05T11:55:18.748452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745443+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:55:18.745443+0000","last_peered":"2024-09-05T11:55:18.745443+0000","last_clean":"2024-09-05T11:55:18.745443+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:55:18.745443+0000","last_undegraded":"2024-09-05T11:55:18.745443+0000","last_fullsized":"2024-09-05T11:55:18.745443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748502+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:55:18.748502+0000","last_peered":"2024-09-05T11:55:18.748502+0000","last_clean":"2024-09-05T11:55:18.748502+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:55:18.748502+0000","last_undegraded":"2024-09-05T11:55:18.748502+0000","last_fullsized":"2024-09-05T11:55:18.748502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750324+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:55:18.750324+0000","last_peered":"2024-09-05T11:55:18.750324+0000","last_clean":"2024-09-05T11:55:18.750324+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:55:18.750324+0000","last_undegraded":"2024-09-05T11:55:18.750324+0000","last_fullsized":"2024-09-05T11:55:18.750324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745255+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:55:18.745255+0000","last_peered":"2024-09-05T11:55:18.745255+0000","last_clean":"2024-09-05T11:55:18.745255+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:55:18.745255+0000","last_undegraded":"2024-09-05T11:55:18.745255+0000","last_fullsized":"2024-09-05T11:55:18.745255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749954+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:55:18.749954+0000","last_peered":"2024-09-05T11:55:18.749954+0000","last_clean":"2024-09-05T11:55:18.749954+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:55:18.749954+0000","last_undegraded":"2024-09-05T11:55:18.749954+0000","last_fullsized":"2024-09-05T11:55:18.749954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744674+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:55:18.744674+0000","last_peered":"2024-09-05T11:55:18.744674+0000","last_clean":"2024-09-05T11:55:18.744674+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:55:18.744674+0000","last_undegraded":"2024-09-05T11:55:18.744674+0000","last_fullsized":"2024-09-05T11:55:18.744674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751392+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:55:18.751392+0000","last_peered":"2024-09-05T11:55:18.751392+0000","last_clean":"2024-09-05T11:55:18.751392+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:55:18.751392+0000","last_undegraded":"2024-09-05T11:55:18.751392+0000","last_fullsized":"2024-09-05T11:55:18.751392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744437+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:55:18.744437+0000","last_peered":"2024-09-05T11:55:18.744437+0000","last_clean":"2024-09-05T11:55:18.744437+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:55:18.744437+0000","last_undegraded":"2024-09-05T11:55:18.744437+0000","last_fullsized":"2024-09-05T11:55:18.744437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744739+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:55:18.744739+0000","last_peered":"2024-09-05T11:55:18.744739+0000","last_clean":"2024-09-05T11:55:18.744739+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:55:18.744739+0000","last_undegraded":"2024-09-05T11:55:18.744739+0000","last_fullsized":"2024-09-05T11:55:18.744739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744301+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:55:18.744301+0000","last_peered":"2024-09-05T11:55:18.744301+0000","last_clean":"2024-09-05T11:55:18.744301+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:55:18.744301+0000","last_undegraded":"2024-09-05T11:55:18.744301+0000","last_fullsized":"2024-09-05T11:55:18.744301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749802+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:55:18.749802+0000","last_peered":"2024-09-05T11:55:18.749802+0000","last_clean":"2024-09-05T11:55:18.749802+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:55:18.749802+0000","last_undegraded":"2024-09-05T11:55:18.749802+0000","last_fullsized":"2024-09-05T11:55:18.749802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749669+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:55:18.749669+0000","last_peered":"2024-09-05T11:55:18.749669+0000","last_clean":"2024-09-05T11:55:18.749669+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:55:18.749669+0000","last_undegraded":"2024-09-05T11:55:18.749669+0000","last_fullsized":"2024-09-05T11:55:18.749669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959105+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:55:18.959105+0000","last_peered":"2024-09-05T11:55:18.959105+0000","last_clean":"2024-09-05T11:55:18.959105+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:55:18.959105+0000","last_undegraded":"2024-09-05T11:55:18.959105+0000","last_fullsized":"2024-09-05T11:55:18.959105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747099+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:55:18.747099+0000","last_peered":"2024-09-05T11:55:18.747099+0000","last_clean":"2024-09-05T11:55:18.747099+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:55:18.747099+0000","last_undegraded":"2024-09-05T11:55:18.747099+0000","last_fullsized":"2024-09-05T11:55:18.747099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749930+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:55:18.749930+0000","last_peered":"2024-09-05T11:55:18.749930+0000","last_clean":"2024-09-05T11:55:18.749930+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:55:18.749930+0000","last_undegraded":"2024-09-05T11:55:18.749930+0000","last_fullsized":"2024-09-05T11:55:18.749930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750328+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:55:18.750328+0000","last_peered":"2024-09-05T11:55:18.750328+0000","last_clean":"2024-09-05T11:55:18.750328+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:55:18.750328+0000","last_undegraded":"2024-09-05T11:55:18.750328+0000","last_fullsized":"2024-09-05T11:55:18.750328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750439+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:55:18.750439+0000","last_peered":"2024-09-05T11:55:18.750439+0000","last_clean":"2024-09-05T11:55:18.750439+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:55:18.750439+0000","last_undegraded":"2024-09-05T11:55:18.750439+0000","last_fullsized":"2024-09-05T11:55:18.750439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745209+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:55:18.745209+0000","last_peered":"2024-09-05T11:55:18.745209+0000","last_clean":"2024-09-05T11:55:18.745209+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:55:18.745209+0000","last_undegraded":"2024-09-05T11:55:18.745209+0000","last_fullsized":"2024-09-05T11:55:18.745209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060892+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:55:19.060892+0000","last_peered":"2024-09-05T11:55:19.060892+0000","last_clean":"2024-09-05T11:55:19.060892+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:55:19.060892+0000","last_undegraded":"2024-09-05T11:55:19.060892+0000","last_fullsized":"2024-09-05T11:55:19.060892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750265+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:55:18.750265+0000","last_peered":"2024-09-05T11:55:18.750265+0000","last_clean":"2024-09-05T11:55:18.750265+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:55:18.750265+0000","last_undegraded":"2024-09-05T11:55:18.750265+0000","last_fullsized":"2024-09-05T11:55:18.750265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750840+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:55:18.750840+0000","last_peered":"2024-09-05T11:55:18.750840+0000","last_clean":"2024-09-05T11:55:18.750840+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:55:18.750840+0000","last_undegraded":"2024-09-05T11:55:18.750840+0000","last_fullsized":"2024-09-05T11:55:18.750840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828776+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:55:18.828776+0000","last_peered":"2024-09-05T11:55:18.828776+0000","last_clean":"2024-09-05T11:55:18.828776+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:55:18.828776+0000","last_undegraded":"2024-09-05T11:55:18.828776+0000","last_fullsized":"2024-09-05T11:55:18.828776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750453+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:55:18.750453+0000","last_peered":"2024-09-05T11:55:18.750453+0000","last_clean":"2024-09-05T11:55:18.750453+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:55:18.750453+0000","last_undegraded":"2024-09-05T11:55:18.750453+0000","last_fullsized":"2024-09-05T11:55:18.750453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745094+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:55:18.745094+0000","last_peered":"2024-09-05T11:55:18.745094+0000","last_clean":"2024-09-05T11:55:18.745094+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:55:18.745094+0000","last_undegraded":"2024-09-05T11:55:18.745094+0000","last_fullsized":"2024-09-05T11:55:18.745094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749801+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:55:18.749801+0000","last_peered":"2024-09-05T11:55:18.749801+0000","last_clean":"2024-09-05T11:55:18.749801+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:55:18.749801+0000","last_undegraded":"2024-09-05T11:55:18.749801+0000","last_fullsized":"2024-09-05T11:55:18.749801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749962+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:55:18.749962+0000","last_peered":"2024-09-05T11:55:18.749962+0000","last_clean":"2024-09-05T11:55:18.749962+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:55:18.749962+0000","last_undegraded":"2024-09-05T11:55:18.749962+0000","last_fullsized":"2024-09-05T11:55:18.749962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749212+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:55:18.749212+0000","last_peered":"2024-09-05T11:55:18.749212+0000","last_clean":"2024-09-05T11:55:18.749212+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:55:18.749212+0000","last_undegraded":"2024-09-05T11:55:18.749212+0000","last_fullsized":"2024-09-05T11:55:18.749212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748596+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:55:18.748596+0000","last_peered":"2024-09-05T11:55:18.748596+0000","last_clean":"2024-09-05T11:55:18.748596+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:55:18.748596+0000","last_undegraded":"2024-09-05T11:55:18.748596+0000","last_fullsized":"2024-09-05T11:55:18.748596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749735+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:55:18.749735+0000","last_peered":"2024-09-05T11:55:18.749735+0000","last_clean":"2024-09-05T11:55:18.749735+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:55:18.749735+0000","last_undegraded":"2024-09-05T11:55:18.749735+0000","last_fullsized":"2024-09-05T11:55:18.749735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751378+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:55:18.751378+0000","last_peered":"2024-09-05T11:55:18.751378+0000","last_clean":"2024-09-05T11:55:18.751378+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:55:18.751378+0000","last_undegraded":"2024-09-05T11:55:18.751378+0000","last_fullsized":"2024-09-05T11:55:18.751378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744997+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:55:18.744997+0000","last_peered":"2024-09-05T11:55:18.744997+0000","last_clean":"2024-09-05T11:55:18.744997+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:55:18.744997+0000","last_undegraded":"2024-09-05T11:55:18.744997+0000","last_fullsized":"2024-09-05T11:55:18.744997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749638+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:55:18.749638+0000","last_peered":"2024-09-05T11:55:18.749638+0000","last_clean":"2024-09-05T11:55:18.749638+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:55:18.749638+0000","last_undegraded":"2024-09-05T11:55:18.749638+0000","last_fullsized":"2024-09-05T11:55:18.749638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747345+0000","last_change":"2024-09-05T11:54:29.102616+0000","last_active":"2024-09-05T11:55:18.747345+0000","last_peered":"2024-09-05T11:55:18.747345+0000","last_clean":"2024-09-05T11:55:18.747345+0000","last_became_active":"2024-09-05T11:54:29.102126+0000","last_became_peered":"2024-09-05T11:54:29.102126+0000","last_unstale":"2024-09-05T11:55:18.747345+0000","last_undegraded":"2024-09-05T11:55:18.747345+0000","last_fullsized":"2024-09-05T11:55:18.747345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:09:28.848395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748687+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:55:18.748687+0000","last_peered":"2024-09-05T11:55:18.748687+0000","last_clean":"2024-09-05T11:55:18.748687+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:55:18.748687+0000","last_undegraded":"2024-09-05T11:55:18.748687+0000","last_fullsized":"2024-09-05T11:55:18.748687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750129+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:55:18.750129+0000","last_peered":"2024-09-05T11:55:18.750129+0000","last_clean":"2024-09-05T11:55:18.750129+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:55:18.750129+0000","last_undegraded":"2024-09-05T11:55:18.750129+0000","last_fullsized":"2024-09-05T11:55:18.750129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751077+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:55:18.751077+0000","last_peered":"2024-09-05T11:55:18.751077+0000","last_clean":"2024-09-05T11:55:18.751077+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:55:18.751077+0000","last_undegraded":"2024-09-05T11:55:18.751077+0000","last_fullsized":"2024-09-05T11:55:18.751077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748683+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:55:18.748683+0000","last_peered":"2024-09-05T11:55:18.748683+0000","last_clean":"2024-09-05T11:55:18.748683+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:55:18.748683+0000","last_undegraded":"2024-09-05T11:55:18.748683+0000","last_fullsized":"2024-09-05T11:55:18.748683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744841+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:55:18.744841+0000","last_peered":"2024-09-05T11:55:18.744841+0000","last_clean":"2024-09-05T11:55:18.744841+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:55:18.744841+0000","last_undegraded":"2024-09-05T11:55:18.744841+0000","last_fullsized":"2024-09-05T11:55:18.744841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744915+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:55:18.744915+0000","last_peered":"2024-09-05T11:55:18.744915+0000","last_clean":"2024-09-05T11:55:18.744915+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:55:18.744915+0000","last_undegraded":"2024-09-05T11:55:18.744915+0000","last_fullsized":"2024-09-05T11:55:18.744915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750719+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:55:18.750719+0000","last_peered":"2024-09-05T11:55:18.750719+0000","last_clean":"2024-09-05T11:55:18.750719+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:55:18.750719+0000","last_undegraded":"2024-09-05T11:55:18.750719+0000","last_fullsized":"2024-09-05T11:55:18.750719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749744+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:55:18.749744+0000","last_peered":"2024-09-05T11:55:18.749744+0000","last_clean":"2024-09-05T11:55:18.749744+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:55:18.749744+0000","last_undegraded":"2024-09-05T11:55:18.749744+0000","last_fullsized":"2024-09-05T11:55:18.749744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:29.753930+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:55:29.753930+0000","last_peered":"2024-09-05T11:55:29.753930+0000","last_clean":"2024-09-05T11:55:29.753930+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:55:29.753930+0000","last_undegraded":"2024-09-05T11:55:29.753930+0000","last_fullsized":"2024-09-05T11:55:29.753930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59728,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":75,"num_write":30,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747427+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:55:18.747427+0000","last_peered":"2024-09-05T11:55:18.747427+0000","last_clean":"2024-09-05T11:55:18.747427+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:55:18.747427+0000","last_undegraded":"2024-09-05T11:55:18.747427+0000","last_fullsized":"2024-09-05T11:55:18.747427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061114+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:55:19.061114+0000","last_peered":"2024-09-05T11:55:19.061114+0000","last_clean":"2024-09-05T11:55:19.061114+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:55:19.061114+0000","last_undegraded":"2024-09-05T11:55:19.061114+0000","last_fullsized":"2024-09-05T11:55:19.061114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750985+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:55:18.750985+0000","last_peered":"2024-09-05T11:55:18.750985+0000","last_clean":"2024-09-05T11:55:18.750985+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:55:18.750985+0000","last_undegraded":"2024-09-05T11:55:18.750985+0000","last_fullsized":"2024-09-05T11:55:18.750985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746602+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:55:18.746602+0000","last_peered":"2024-09-05T11:55:18.746602+0000","last_clean":"2024-09-05T11:55:18.746602+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:55:18.746602+0000","last_undegraded":"2024-09-05T11:55:18.746602+0000","last_fullsized":"2024-09-05T11:55:18.746602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829307+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:55:18.829307+0000","last_peered":"2024-09-05T11:55:18.829307+0000","last_clean":"2024-09-05T11:55:18.829307+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:55:18.829307+0000","last_undegraded":"2024-09-05T11:55:18.829307+0000","last_fullsized":"2024-09-05T11:55:18.829307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750531+0000","last_change":"2024-09-05T11:54:29.106596+0000","last_active":"2024-09-05T11:55:18.750531+0000","last_peered":"2024-09-05T11:55:18.750531+0000","last_clean":"2024-09-05T11:55:18.750531+0000","last_became_active":"2024-09-05T11:54:29.106213+0000","last_became_peered":"2024-09-05T11:54:29.106213+0000","last_unstale":"2024-09-05T11:55:18.750531+0000","last_undegraded":"2024-09-05T11:55:18.750531+0000","last_fullsized":"2024-09-05T11:55:18.750531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:24:44.592047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747050+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:55:18.747050+0000","last_peered":"2024-09-05T11:55:18.747050+0000","last_clean":"2024-09-05T11:55:18.747050+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:55:18.747050+0000","last_undegraded":"2024-09-05T11:55:18.747050+0000","last_fullsized":"2024-09-05T11:55:18.747050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061360+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:55:19.061360+0000","last_peered":"2024-09-05T11:55:19.061360+0000","last_clean":"2024-09-05T11:55:19.061360+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:55:19.061360+0000","last_undegraded":"2024-09-05T11:55:19.061360+0000","last_fullsized":"2024-09-05T11:55:19.061360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745488+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:55:18.745488+0000","last_peered":"2024-09-05T11:55:18.745488+0000","last_clean":"2024-09-05T11:55:18.745488+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:55:18.745488+0000","last_undegraded":"2024-09-05T11:55:18.745488+0000","last_fullsized":"2024-09-05T11:55:18.745488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959078+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:55:18.959078+0000","last_peered":"2024-09-05T11:55:18.959078+0000","last_clean":"2024-09-05T11:55:18.959078+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:55:18.959078+0000","last_undegraded":"2024-09-05T11:55:18.959078+0000","last_fullsized":"2024-09-05T11:55:18.959078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829040+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:55:18.829040+0000","last_peered":"2024-09-05T11:55:18.829040+0000","last_clean":"2024-09-05T11:55:18.829040+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:55:18.829040+0000","last_undegraded":"2024-09-05T11:55:18.829040+0000","last_fullsized":"2024-09-05T11:55:18.829040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747393+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:55:18.747393+0000","last_peered":"2024-09-05T11:55:18.747393+0000","last_clean":"2024-09-05T11:55:18.747393+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:55:18.747393+0000","last_undegraded":"2024-09-05T11:55:18.747393+0000","last_fullsized":"2024-09-05T11:55:18.747393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751542+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:55:18.751542+0000","last_peered":"2024-09-05T11:55:18.751542+0000","last_clean":"2024-09-05T11:55:18.751542+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:55:18.751542+0000","last_undegraded":"2024-09-05T11:55:18.751542+0000","last_fullsized":"2024-09-05T11:55:18.751542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749841+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:55:18.749841+0000","last_peered":"2024-09-05T11:55:18.749841+0000","last_clean":"2024-09-05T11:55:18.749841+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:55:18.749841+0000","last_undegraded":"2024-09-05T11:55:18.749841+0000","last_fullsized":"2024-09-05T11:55:18.749841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959536+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:55:18.959536+0000","last_peered":"2024-09-05T11:55:18.959536+0000","last_clean":"2024-09-05T11:55:18.959536+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:55:18.959536+0000","last_undegraded":"2024-09-05T11:55:18.959536+0000","last_fullsized":"2024-09-05T11:55:18.959536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744321+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:55:18.744321+0000","last_peered":"2024-09-05T11:55:18.744321+0000","last_clean":"2024-09-05T11:55:18.744321+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:55:18.744321+0000","last_undegraded":"2024-09-05T11:55:18.744321+0000","last_fullsized":"2024-09-05T11:55:18.744321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745069+0000","last_change":"2024-09-05T11:54:29.101408+0000","last_active":"2024-09-05T11:55:18.745069+0000","last_peered":"2024-09-05T11:55:18.745069+0000","last_clean":"2024-09-05T11:55:18.745069+0000","last_became_active":"2024-09-05T11:54:29.101052+0000","last_became_peered":"2024-09-05T11:54:29.101052+0000","last_unstale":"2024-09-05T11:55:18.745069+0000","last_undegraded":"2024-09-05T11:55:18.745069+0000","last_fullsized":"2024-09-05T11:55:18.745069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:40.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960761+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:55:18.960761+0000","last_peered":"2024-09-05T11:55:18.960761+0000","last_clean":"2024-09-05T11:55:18.960761+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:55:18.960761+0000","last_undegraded":"2024-09-05T11:55:18.960761+0000","last_fullsized":"2024-09-05T11:55:18.960761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744820+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:55:18.744820+0000","last_peered":"2024-09-05T11:55:18.744820+0000","last_clean":"2024-09-05T11:55:18.744820+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:55:18.744820+0000","last_undegraded":"2024-09-05T11:55:18.744820+0000","last_fullsized":"2024-09-05T11:55:18.744820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750928+0000","last_change":"2024-09-05T11:54:29.102371+0000","last_active":"2024-09-05T11:55:18.750928+0000","last_peered":"2024-09-05T11:55:18.750928+0000","last_clean":"2024-09-05T11:55:18.750928+0000","last_became_active":"2024-09-05T11:54:29.102066+0000","last_became_peered":"2024-09-05T11:54:29.102066+0000","last_unstale":"2024-09-05T11:55:18.750928+0000","last_undegraded":"2024-09-05T11:55:18.750928+0000","last_fullsized":"2024-09-05T11:55:18.750928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:55.008088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754810+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:55:18.754810+0000","last_peered":"2024-09-05T11:55:18.754810+0000","last_clean":"2024-09-05T11:55:18.754810+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:55:18.754810+0000","last_undegraded":"2024-09-05T11:55:18.754810+0000","last_fullsized":"2024-09-05T11:55:18.754810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744917+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:55:18.744917+0000","last_peered":"2024-09-05T11:55:18.744917+0000","last_clean":"2024-09-05T11:55:18.744917+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:55:18.744917+0000","last_undegraded":"2024-09-05T11:55:18.744917+0000","last_fullsized":"2024-09-05T11:55:18.744917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748298+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:55:18.748298+0000","last_peered":"2024-09-05T11:55:18.748298+0000","last_clean":"2024-09-05T11:55:18.748298+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:55:18.748298+0000","last_undegraded":"2024-09-05T11:55:18.748298+0000","last_fullsized":"2024-09-05T11:55:18.748298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745098+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:55:18.745098+0000","last_peered":"2024-09-05T11:55:18.745098+0000","last_clean":"2024-09-05T11:55:18.745098+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:55:18.745098+0000","last_undegraded":"2024-09-05T11:55:18.745098+0000","last_fullsized":"2024-09-05T11:55:18.745098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744606+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:55:18.744606+0000","last_peered":"2024-09-05T11:55:18.744606+0000","last_clean":"2024-09-05T11:55:18.744606+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:55:18.744606+0000","last_undegraded":"2024-09-05T11:55:18.744606+0000","last_fullsized":"2024-09-05T11:55:18.744606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828789+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:55:18.828789+0000","last_peered":"2024-09-05T11:55:18.828789+0000","last_clean":"2024-09-05T11:55:18.828789+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:55:18.828789+0000","last_undegraded":"2024-09-05T11:55:18.828789+0000","last_fullsized":"2024-09-05T11:55:18.828789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744825+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:55:18.744825+0000","last_peered":"2024-09-05T11:55:18.744825+0000","last_clean":"2024-09-05T11:55:18.744825+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:55:18.744825+0000","last_undegraded":"2024-09-05T11:55:18.744825+0000","last_fullsized":"2024-09-05T11:55:18.744825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749366+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:55:18.749366+0000","last_peered":"2024-09-05T11:55:18.749366+0000","last_clean":"2024-09-05T11:55:18.749366+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:55:18.749366+0000","last_undegraded":"2024-09-05T11:55:18.749366+0000","last_fullsized":"2024-09-05T11:55:18.749366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745046+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:55:18.745046+0000","last_peered":"2024-09-05T11:55:18.745046+0000","last_clean":"2024-09-05T11:55:18.745046+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:55:18.745046+0000","last_undegraded":"2024-09-05T11:55:18.745046+0000","last_fullsized":"2024-09-05T11:55:18.745046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959692+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:55:18.959692+0000","last_peered":"2024-09-05T11:55:18.959692+0000","last_clean":"2024-09-05T11:55:18.959692+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:55:18.959692+0000","last_undegraded":"2024-09-05T11:55:18.959692+0000","last_fullsized":"2024-09-05T11:55:18.959692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751303+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:55:18.751303+0000","last_peered":"2024-09-05T11:55:18.751303+0000","last_clean":"2024-09-05T11:55:18.751303+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:55:18.751303+0000","last_undegraded":"2024-09-05T11:55:18.751303+0000","last_fullsized":"2024-09-05T11:55:18.751303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959252+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:55:18.959252+0000","last_peered":"2024-09-05T11:55:18.959252+0000","last_clean":"2024-09-05T11:55:18.959252+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:55:18.959252+0000","last_undegraded":"2024-09-05T11:55:18.959252+0000","last_fullsized":"2024-09-05T11:55:18.959252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061014+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:55:19.061014+0000","last_peered":"2024-09-05T11:55:19.061014+0000","last_clean":"2024-09-05T11:55:19.061014+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:55:19.061014+0000","last_undegraded":"2024-09-05T11:55:19.061014+0000","last_fullsized":"2024-09-05T11:55:19.061014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744715+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:55:18.744715+0000","last_peered":"2024-09-05T11:55:18.744715+0000","last_clean":"2024-09-05T11:55:18.744715+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:55:18.744715+0000","last_undegraded":"2024-09-05T11:55:18.744715+0000","last_fullsized":"2024-09-05T11:55:18.744715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754299+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:55:18.754299+0000","last_peered":"2024-09-05T11:55:18.754299+0000","last_clean":"2024-09-05T11:55:18.754299+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:55:18.754299+0000","last_undegraded":"2024-09-05T11:55:18.754299+0000","last_fullsized":"2024-09-05T11:55:18.754299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745216+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:55:18.745216+0000","last_peered":"2024-09-05T11:55:18.745216+0000","last_clean":"2024-09-05T11:55:18.745216+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:55:18.745216+0000","last_undegraded":"2024-09-05T11:55:18.745216+0000","last_fullsized":"2024-09-05T11:55:18.745216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748542+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:55:18.748542+0000","last_peered":"2024-09-05T11:55:18.748542+0000","last_clean":"2024-09-05T11:55:18.748542+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:55:18.748542+0000","last_undegraded":"2024-09-05T11:55:18.748542+0000","last_fullsized":"2024-09-05T11:55:18.748542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750435+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:55:18.750435+0000","last_peered":"2024-09-05T11:55:18.750435+0000","last_clean":"2024-09-05T11:55:18.750435+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:55:18.750435+0000","last_undegraded":"2024-09-05T11:55:18.750435+0000","last_fullsized":"2024-09-05T11:55:18.750435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060771+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:55:19.060771+0000","last_peered":"2024-09-05T11:55:19.060771+0000","last_clean":"2024-09-05T11:55:19.060771+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:55:19.060771+0000","last_undegraded":"2024-09-05T11:55:19.060771+0000","last_fullsized":"2024-09-05T11:55:19.060771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828621+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:55:18.828621+0000","last_peered":"2024-09-05T11:55:18.828621+0000","last_clean":"2024-09-05T11:55:18.828621+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:55:18.828621+0000","last_undegraded":"2024-09-05T11:55:18.828621+0000","last_fullsized":"2024-09-05T11:55:18.828621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828531+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:55:18.828531+0000","last_peered":"2024-09-05T11:55:18.828531+0000","last_clean":"2024-09-05T11:55:18.828531+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:55:18.828531+0000","last_undegraded":"2024-09-05T11:55:18.828531+0000","last_fullsized":"2024-09-05T11:55:18.828531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744676+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:55:18.744676+0000","last_peered":"2024-09-05T11:55:18.744676+0000","last_clean":"2024-09-05T11:55:18.744676+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:55:18.744676+0000","last_undegraded":"2024-09-05T11:55:18.744676+0000","last_fullsized":"2024-09-05T11:55:18.744676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747214+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:55:18.747214+0000","last_peered":"2024-09-05T11:55:18.747214+0000","last_clean":"2024-09-05T11:55:18.747214+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:55:18.747214+0000","last_undegraded":"2024-09-05T11:55:18.747214+0000","last_fullsized":"2024-09-05T11:55:18.747214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750519+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:55:18.750519+0000","last_peered":"2024-09-05T11:55:18.750519+0000","last_clean":"2024-09-05T11:55:18.750519+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:55:18.750519+0000","last_undegraded":"2024-09-05T11:55:18.750519+0000","last_fullsized":"2024-09-05T11:55:18.750519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745888+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:55:18.745888+0000","last_peered":"2024-09-05T11:55:18.745888+0000","last_clean":"2024-09-05T11:55:18.745888+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:55:18.745888+0000","last_undegraded":"2024-09-05T11:55:18.745888+0000","last_fullsized":"2024-09-05T11:55:18.745888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061418+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:55:19.061418+0000","last_peered":"2024-09-05T11:55:19.061418+0000","last_clean":"2024-09-05T11:55:19.061418+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:55:19.061418+0000","last_undegraded":"2024-09-05T11:55:19.061418+0000","last_fullsized":"2024-09-05T11:55:19.061418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753244+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:55:18.753244+0000","last_peered":"2024-09-05T11:55:18.753244+0000","last_clean":"2024-09-05T11:55:18.753244+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:55:18.753244+0000","last_undegraded":"2024-09-05T11:55:18.753244+0000","last_fullsized":"2024-09-05T11:55:18.753244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754442+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:55:18.754442+0000","last_peered":"2024-09-05T11:55:18.754442+0000","last_clean":"2024-09-05T11:55:18.754442+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:55:18.754442+0000","last_undegraded":"2024-09-05T11:55:18.754442+0000","last_fullsized":"2024-09-05T11:55:18.754442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751541+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:55:18.751541+0000","last_peered":"2024-09-05T11:55:18.751541+0000","last_clean":"2024-09-05T11:55:18.751541+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:55:18.751541+0000","last_undegraded":"2024-09-05T11:55:18.751541+0000","last_fullsized":"2024-09-05T11:55:18.751541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748945+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:55:18.748945+0000","last_peered":"2024-09-05T11:55:18.748945+0000","last_clean":"2024-09-05T11:55:18.748945+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:55:18.748945+0000","last_undegraded":"2024-09-05T11:55:18.748945+0000","last_fullsized":"2024-09-05T11:55:18.748945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.752031+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:55:18.752031+0000","last_peered":"2024-09-05T11:55:18.752031+0000","last_clean":"2024-09-05T11:55:18.752031+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:55:18.752031+0000","last_undegraded":"2024-09-05T11:55:18.752031+0000","last_fullsized":"2024-09-05T11:55:18.752031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828990+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:55:18.828990+0000","last_peered":"2024-09-05T11:55:18.828990+0000","last_clean":"2024-09-05T11:55:18.828990+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:55:18.828990+0000","last_undegraded":"2024-09-05T11:55:18.828990+0000","last_fullsized":"2024-09-05T11:55:18.828990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829035+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:55:18.829035+0000","last_peered":"2024-09-05T11:55:18.829035+0000","last_clean":"2024-09-05T11:55:18.829035+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:55:18.829035+0000","last_undegraded":"2024-09-05T11:55:18.829035+0000","last_fullsized":"2024-09-05T11:55:18.829035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750338+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:55:18.750338+0000","last_peered":"2024-09-05T11:55:18.750338+0000","last_clean":"2024-09-05T11:55:18.750338+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:55:18.750338+0000","last_undegraded":"2024-09-05T11:55:18.750338+0000","last_fullsized":"2024-09-05T11:55:18.750338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60526,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":49,"num_read_kb":81,"num_write":82,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":181578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":513188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1703936,"data_stored":1159213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1108234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":509092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1638400,"data_stored":1103738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":509196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":568854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":568820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:55:34.691 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T11:55:34.691 INFO:tasks.cephadm:Setting up client nodes... 2024-09-05T11:55:34.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-05T11:55:34.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:34 smithi059 ceph-mon[30025]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T11:55:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:34 smithi189 ceph-mon[34772]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T11:55:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:34 smithi089 ceph-mon[34499]: pgmap v329: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 -- 172.21.15.59:0/1557843598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c200761a0 msgr2=0x7f3c20074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 --2- 172.21.15.59:0/1557843598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c200761a0 0x7f3c20074600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009860 tx=0x7f3c1002a340 comp rx=0 tx=0).stop 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 -- 172.21.15.59:0/1557843598 shutdown_connections 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 --2- 172.21.15.59:0/1557843598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20109a70 0x7f3c2010bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 --2- 172.21.15.59:0/1557843598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c20074b40 0x7f3c20074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 --2- 172.21.15.59:0/1557843598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c200761a0 0x7f3c20074600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 -- 172.21.15.59:0/1557843598 >> 172.21.15.59:0/1557843598 conn(0x7f3c200fd390 msgr2=0x7f3c200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 -- 172.21.15.59:0/1557843598 shutdown_connections 2024-09-05T11:55:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.610+0000 7f3c25000640 1 -- 172.21.15.59:0/1557843598 wait complete. 2024-09-05T11:55:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.611+0000 7f3c25000640 1 Processor -- start 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 -- start start 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c200761a0 0x7f3c2019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 0x7f3c201a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2010fa70 con 0x7f3c200761a0 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2010f8f0 con 0x7f3c20109a70 2024-09-05T11:55:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c25000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2010fbf0 con 0x7f3c20074b40 2024-09-05T11:55:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c1de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c1de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52550/0 (socket says 172.21.15.59:52550) 2024-09-05T11:55:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.612+0000 7f3c1de00640 1 -- 172.21.15.59:0/992523967 learned_addr learned my addr 172.21.15.59:0/992523967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1d400640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c200761a0 0x7f3c2019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1e800640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 0x7f3c201a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 -- 172.21.15.59:0/992523967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 msgr2=0x7f3c201a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 0x7f3c201a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 -- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c200761a0 msgr2=0x7f3c2019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c200761a0 0x7f3c2019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 -- 172.21.15.59:0/992523967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201a1d00 con 0x7f3c20074b40 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1e800640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 0x7f3c201a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.613+0000 7f3c1de00640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c10005940 tx=0x7f3c10033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.614+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1003f020 con 0x7f3c20074b40 2024-09-05T11:55:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.614+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c201a1f90 con 0x7f3c20074b40 2024-09-05T11:55:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.614+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c10030850 con 0x7f3c20074b40 2024-09-05T11:55:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.614+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10037040 con 0x7f3c20074b40 2024-09-05T11:55:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.614+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201a23e0 con 0x7f3c20074b40 2024-09-05T11:55:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.616+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c20108880 con 0x7f3c20074b40 2024-09-05T11:55:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.617+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3c10042020 con 0x7f3c20074b40 2024-09-05T11:55:35.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.618+0000 7f3c16a00640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 0x7f3c040a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:35.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.619+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3c100ef6d0 con 0x7f3c20074b40 2024-09-05T11:55:35.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.619+0000 7f3c1d400640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 0x7f3c040a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:35.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.619+0000 7f3c1d400640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 0x7f3c040a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c2019b5f0 tx=0x7f3c0c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:35.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.623+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c10049020 con 0x7f3c20074b40 2024-09-05T11:55:35.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:35 smithi059 ceph-mon[30025]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:35.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.882+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3c20062ec0 con 0x7f3c20074b40 2024-09-05T11:55:35.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.888+0000 7f3c16a00640 1 -- 172.21.15.59:0/992523967 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3c100bc220 con 0x7f3c20074b40 2024-09-05T11:55:35.889 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-09-05T11:55:35.889 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQA3nNlmHdK7NBAAe1bpsWaXDFHifxzFgyOGIQ== 2024-09-05T11:55:35.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.890+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 msgr2=0x7f3c040a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:35.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.890+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 0x7f3c040a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c2019b5f0 tx=0x7f3c0c009260 comp rx=0 tx=0).stop 2024-09-05T11:55:35.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.890+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 msgr2=0x7f3c2019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:35.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.890+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c10005940 tx=0x7f3c10033d90 comp rx=0 tx=0).stop 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 shutdown_connections 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3c040a2cc0 0x7f3c040a5180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c20109a70 0x7f3c201a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c200761a0 0x7f3c2019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 --2- 172.21.15.59:0/992523967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c20074b40 0x7f3c2019a0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 >> 172.21.15.59:0/992523967 conn(0x7f3c200fd390 msgr2=0x7f3c20105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 shutdown_connections 2024-09-05T11:55:35.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:35.891+0000 7f3c25000640 1 -- 172.21.15.59:0/992523967 wait complete. 2024-09-05T11:55:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:35 smithi189 ceph-mon[34772]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:35 smithi089 ceph-mon[34499]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:55:36.677 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:55:36.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-05T11:55:36.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-05T11:55:36.760 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T11:55:36.760 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-05T11:55:36.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph mgr dump --format=json 2024-09-05T11:55:36.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:36 smithi059 ceph-mon[30025]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-09-05T11:55:36.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:36 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/992523967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:36.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:36 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:36.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:36 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T11:55:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:36 smithi189 ceph-mon[34772]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-09-05T11:55:37.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:36 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/992523967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:37.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:36 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:37.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:36 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T11:55:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:36 smithi089 ceph-mon[34499]: pgmap v330: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-09-05T11:55:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:36 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/992523967' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:36 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T11:55:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:36 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 -- 172.21.15.59:0/755012954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18801049a0 msgr2=0x7f18801089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 --2- 172.21.15.59:0/755012954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18801049a0 0x7f18801089f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c02a410 comp rx=0 tx=0).stop 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 -- 172.21.15.59:0/755012954 shutdown_connections 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 --2- 172.21.15.59:0/755012954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1880109120 0x7f1880110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 --2- 172.21.15.59:0/755012954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18801049a0 0x7f18801089f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 --2- 172.21.15.59:0/755012954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f18801043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 -- 172.21.15.59:0/755012954 >> 172.21.15.59:0/755012954 conn(0x7f18800ffa90 msgr2=0x7f1880101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 -- 172.21.15.59:0/755012954 shutdown_connections 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.740+0000 7f1888600640 1 -- 172.21.15.59:0/755012954 wait complete. 2024-09-05T11:55:37.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.741+0000 7f1888600640 1 Processor -- start 2024-09-05T11:55:37.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 -- start start 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f188019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18801049a0 0x7f188019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1880114140 con 0x7f1880109120 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1880113fc0 con 0x7f18801049a0 2024-09-05T11:55:37.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1888600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18801142c0 con 0x7f1880103ff0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1886600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1885c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f188019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1886600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42182/0 (socket says 172.21.15.59:42182) 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1886600640 1 -- 172.21.15.59:0/170416890 learned_addr learned my addr 172.21.15.59:0/170416890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.742+0000 7f1885200640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18801049a0 0x7f188019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 -- 172.21.15.59:0/170416890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18801049a0 msgr2=0x7f188019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18801049a0 0x7f188019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 -- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 msgr2=0x7f188019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 -- 172.21.15.59:0/170416890 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188019f5d0 con 0x7f1880103ff0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1886600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f1885c00640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f188019a620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9f0 tx=0x7f187400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:37.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.743+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400ccc0 con 0x7f1880103ff0 2024-09-05T11:55:37.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.744+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188019f8c0 con 0x7f1880103ff0 2024-09-05T11:55:37.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.744+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874005020 con 0x7f1880103ff0 2024-09-05T11:55:37.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.744+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740049e0 con 0x7f1880103ff0 2024-09-05T11:55:37.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.744+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18801a71a0 con 0x7f1880103ff0 2024-09-05T11:55:37.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.745+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18801043d0 con 0x7f1880103ff0 2024-09-05T11:55:37.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.747+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1874005560 con 0x7f1880103ff0 2024-09-05T11:55:37.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.748+0000 7f187e000640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 0x7f18680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.748+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f18740cf5b0 con 0x7f1880103ff0 2024-09-05T11:55:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.748+0000 7f1885200640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 0x7f18680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.749+0000 7f1885200640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 0x7f18680a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f186c009230 tx=0x7f186c0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.750+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18740051d0 con 0x7f1880103ff0 2024-09-05T11:55:37.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.983+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f188019b980 con 0x7f1880103ff0 2024-09-05T11:55:37.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.988+0000 7f187e000640 1 -- 172.21.15.59:0/170416890 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263195 (secure 0 0 0) 0x7f187409c080 con 0x7f1880103ff0 2024-09-05T11:55:37.993 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 msgr2=0x7f18680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 0x7f18680a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f186c009230 tx=0x7f186c0058d0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 msgr2=0x7f188019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f188019a620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9f0 tx=0x7f187400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 shutdown_connections 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f18680a2c10 0x7f18680a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1880109120 0x7f188019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18801049a0 0x7f188019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 --2- 172.21.15.59:0/170416890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1880103ff0 0x7f188019a620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 >> 172.21.15.59:0/170416890 conn(0x7f18800ffa90 msgr2=0x7f1880100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 shutdown_connections 2024-09-05T11:55:37.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:37.993+0000 7f1888600640 1 -- 172.21.15.59:0/170416890 wait complete. 2024-09-05T11:55:38.712 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"flags":0,"active_gid":14146,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":23374249},{"type":"v1","addr":"172.21.15.59:6801","nonce":23374249}]},"active_addr":"172.21.15.59:6801/23374249","active_change":"2024-09-05T11:47:16.761249+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.59:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3486746151}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1760103660}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3884315621}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3791005244}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2092369635}]}]} 2024-09-05T11:55:38.720 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-05T11:55:38.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-05T11:55:38.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:55:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:38 smithi059 ceph-mon[30025]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-05T11:55:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:38 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/170416890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T11:55:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:38 smithi189 ceph-mon[34772]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-05T11:55:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:38 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/170416890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T11:55:39.102 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:38 smithi089 ceph-mon[34499]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-05T11:55:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:38 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/170416890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.750+0000 7ff8f5800640 1 -- 172.21.15.59:0/329091418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074590 msgr2=0x7ff8f0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.750+0000 7ff8f5800640 1 --2- 172.21.15.59:0/329091418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074590 0x7ff8f0074970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009900 tx=0x7ff8e002a480 comp rx=0 tx=0).stop 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 -- 172.21.15.59:0/329091418 shutdown_connections 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 --2- 172.21.15.59:0/329091418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0112b90 0x7ff8f0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 --2- 172.21.15.59:0/329091418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0074eb0 0x7ff8f0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 --2- 172.21.15.59:0/329091418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074590 0x7ff8f0074970 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 -- 172.21.15.59:0/329091418 >> 172.21.15.59:0/329091418 conn(0x7ff8f00fd8c0 msgr2=0x7ff8f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 -- 172.21.15.59:0/329091418 shutdown_connections 2024-09-05T11:55:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.751+0000 7ff8f5800640 1 -- 172.21.15.59:0/329091418 wait complete. 2024-09-05T11:55:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 Processor -- start 2024-09-05T11:55:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 -- start start 2024-09-05T11:55:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0074590 0x7ff8f019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0112b90 0x7ff8f01a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01185c0 con 0x7ff8f0074eb0 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0118440 con 0x7ff8f0074590 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8f5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0118740 con 0x7ff8f0112b90 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42206/0 (socket says 172.21.15.59:42206) 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.752+0000 7ff8e7e00640 1 -- 172.21.15.59:0/2061750158 learned_addr learned my addr 172.21.15.59:0/2061750158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8eea00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0074590 0x7ff8f019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8ef400640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0112b90 0x7ff8f01a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 -- 172.21.15.59:0/2061750158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0112b90 msgr2=0x7ff8f01a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0112b90 0x7ff8f01a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 -- 172.21.15.59:0/2061750158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0074590 msgr2=0x7ff8f019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0074590 0x7ff8f019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 -- 172.21.15.59:0/2061750158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f01a3b80 con 0x7ff8f0074eb0 2024-09-05T11:55:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.753+0000 7ff8e7e00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9f0 tx=0x7ff8d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.754+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800cd80 con 0x7ff8f0074eb0 2024-09-05T11:55:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.754+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01a3e70 con 0x7ff8f0074eb0 2024-09-05T11:55:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.754+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d8005500 con 0x7ff8f0074eb0 2024-09-05T11:55:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.754+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8010690 con 0x7ff8f0074eb0 2024-09-05T11:55:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.754+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f01ab700 con 0x7ff8f0074eb0 2024-09-05T11:55:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.755+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f0074970 con 0x7ff8f0074eb0 2024-09-05T11:55:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.757+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff8d80046c0 con 0x7ff8f0074eb0 2024-09-05T11:55:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.759+0000 7ff8e7400640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.759+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff8d80cf910 con 0x7ff8f0074eb0 2024-09-05T11:55:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.759+0000 7ff8eea00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.760+0000 7ff8eea00640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 0x7ff8d40a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.763+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8d809c350 con 0x7ff8f0074eb0 2024-09-05T11:55:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.956+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8f0104fd0 con 0x7ff8f0074eb0 2024-09-05T11:55:39.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:39 smithi059 ceph-mon[30025]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:55:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.959+0000 7ff8e7400640 1 -- 172.21.15.59:0/2061750158 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24674 (secure 0 0 0) 0x7ff8d809bc50 con 0x7ff8f0074eb0 2024-09-05T11:55:39.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:39.961 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:55:18.740638+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 msgr2=0x7ff8d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 0x7ff8d40a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005b50 comp rx=0 tx=0).stop 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 msgr2=0x7ff8f019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9f0 tx=0x7ff8d800edf0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 shutdown_connections 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f0112b90 0x7ff8f01a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f0074eb0 0x7ff8f019f090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 --2- 172.21.15.59:0/2061750158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8f0074590 0x7ff8f019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.961+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 >> 172.21.15.59:0/2061750158 conn(0x7ff8f00fd8c0 msgr2=0x7ff8f00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.962+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 shutdown_connections 2024-09-05T11:55:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:39.962+0000 7ff8f5800640 1 -- 172.21.15.59:0/2061750158 wait complete. 2024-09-05T11:55:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:39 smithi089 ceph-mon[34499]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:55:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:39 smithi189 ceph-mon[34772]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:55:40.732 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-05T11:55:40.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:55:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:40 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2061750158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:41.110 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:40 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2061750158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:41.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:40 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2061750158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 -- 172.21.15.59:0/2504797507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8112a30 msgr2=0x7fdfb8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 --2- 172.21.15.59:0/2504797507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8112a30 0x7fdfb8114e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ca0 tx=0x7fdfb002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 -- 172.21.15.59:0/2504797507 shutdown_connections 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 --2- 172.21.15.59:0/2504797507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8112a30 0x7fdfb8114e20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 --2- 172.21.15.59:0/2504797507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8077040 0x7fdfb81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 --2- 172.21.15.59:0/2504797507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8076720 0x7fdfb8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 -- 172.21.15.59:0/2504797507 >> 172.21.15.59:0/2504797507 conn(0x7fdfb80ffa90 msgr2=0x7fdfb8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 -- 172.21.15.59:0/2504797507 shutdown_connections 2024-09-05T11:55:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.665+0000 7fdfc0600640 1 -- 172.21.15.59:0/2504797507 wait complete. 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.666+0000 7fdfc0600640 1 Processor -- start 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 -- start start 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8077040 0x7fdfb81a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8112a30 0x7fdfb819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8118580 con 0x7fdfb8076720 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8118400 con 0x7fdfb8112a30 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfc0600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8118700 con 0x7fdfb8077040 2024-09-05T11:55:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfbdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfbdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41080/0 (socket says 172.21.15.59:41080) 2024-09-05T11:55:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfbdc00640 1 -- 172.21.15.59:0/3766681541 learned_addr learned my addr 172.21.15.59:0/3766681541 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfbe600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8112a30 0x7fdfb819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.667+0000 7fdfbd200640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8077040 0x7fdfb81a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 -- 172.21.15.59:0/3766681541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8077040 msgr2=0x7fdfb81a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8077040 0x7fdfb81a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 -- 172.21.15.59:0/3766681541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8112a30 msgr2=0x7fdfb819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8112a30 0x7fdfb819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 -- 172.21.15.59:0/3766681541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb819efe0 con 0x7fdfb8076720 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfbdc00640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00e9f0 tx=0x7fdfac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00cd80 con 0x7fdfb8076720 2024-09-05T11:55:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac005500 con 0x7fdfb8076720 2024-09-05T11:55:41.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb819f2d0 con 0x7fdfb8076720 2024-09-05T11:55:41.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac018630 con 0x7fdfb8076720 2024-09-05T11:55:41.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.668+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81ab520 con 0x7fdfb8076720 2024-09-05T11:55:41.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.669+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb8077e60 con 0x7fdfb8076720 2024-09-05T11:55:41.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.671+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdfac0046c0 con 0x7fdfb8076720 2024-09-05T11:55:41.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.673+0000 7fdfb6000640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 0x7fdfa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:41.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.673+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdfac0cfff0 con 0x7fdfb8076720 2024-09-05T11:55:41.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.673+0000 7fdfbd200640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 0x7fdfa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:41.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.674+0000 7fdfbd200640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 0x7fdfa00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8079300 tx=0x7fdfa4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:41.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.675+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfac09ca50 con 0x7fdfb8076720 2024-09-05T11:55:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.864+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfb8076b00 con 0x7fdfb8076720 2024-09-05T11:55:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.866+0000 7fdfb6000640 1 -- 172.21.15.59:0/3766681541 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24674 (secure 0 0 0) 0x7fdfac09c350 con 0x7fdfb8076720 2024-09-05T11:55:41.869 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:55:41.869 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:55:18.740638+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:55:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.868+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 msgr2=0x7fdfa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.868+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 0x7fdfa00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8079300 tx=0x7fdfa4005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.868+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 msgr2=0x7fdfb81a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.868+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00e9f0 tx=0x7fdfac00edf0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 shutdown_connections 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfa00a2c10 0x7fdfa00a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8112a30 0x7fdfb819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb8077040 0x7fdfb81a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 --2- 172.21.15.59:0/3766681541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfb8076720 0x7fdfb81a46c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 >> 172.21.15.59:0/3766681541 conn(0x7fdfb80ffa90 msgr2=0x7fdfb8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 shutdown_connections 2024-09-05T11:55:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:41.869+0000 7fdfc0600640 1 -- 172.21.15.59:0/3766681541 wait complete. 2024-09-05T11:55:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:41 smithi089 ceph-mon[34499]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-05T11:55:42.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:41 smithi059 ceph-mon[30025]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-05T11:55:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:41 smithi189 ceph-mon[34772]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-05T11:55:42.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-05T11:55:42.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-05T11:55:42.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-05T11:55:42.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-05T11:55:42.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-05T11:55:42.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-05T11:55:42.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-05T11:55:42.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-05T11:55:42.604 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-05T11:55:42.604 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-05T11:55:42.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-05T11:55:42.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-05T11:55:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:42 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3766681541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:43.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3766681541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:42 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3766681541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:55:43.585 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.597 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.768 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.774 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:44.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:43 smithi059 ceph-mon[30025]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:43 smithi089 ceph-mon[34499]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:44.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:43 smithi189 ceph-mon[34772]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 -- 172.21.15.59:0/3057086575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 msgr2=0x7f06c41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 --2- 172.21.15.59:0/3057086575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c41067f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009900 tx=0x7f06b002a410 comp rx=0 tx=0).stop 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 -- 172.21.15.59:0/3057086575 shutdown_connections 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 --2- 172.21.15.59:0/3057086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4106f20 0x7f06c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 --2- 172.21.15.59:0/3057086575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c41067f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 --2- 172.21.15.59:0/3057086575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06c4101df0 0x7f06c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 -- 172.21.15.59:0/3057086575 >> 172.21.15.59:0/3057086575 conn(0x7f06c40fd8b0 msgr2=0x7f06c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 -- 172.21.15.59:0/3057086575 shutdown_connections 2024-09-05T11:55:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.936+0000 7f06cc600640 1 -- 172.21.15.59:0/3057086575 wait complete. 2024-09-05T11:55:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.937+0000 7f06cc600640 1 Processor -- start 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.937+0000 7f06cc600640 1 -- start start 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.937+0000 7f06cc600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4101df0 0x7f06c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.937+0000 7f06cc600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06cc600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06c4106f20 0x7f06c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06cc600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111f80 con 0x7f06c41027a0 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06cc600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111e00 con 0x7f06c4106f20 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06cc600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112100 con 0x7f06c4101df0 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41114/0 (socket says 172.21.15.59:41114) 2024-09-05T11:55:44.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4101df0 0x7f06c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9200640 1 -- 172.21.15.59:0/2511110760 learned_addr learned my addr 172.21.15.59:0/2511110760 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06ca600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06c4106f20 0x7f06c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 -- 172.21.15.59:0/2511110760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06c4106f20 msgr2=0x7f06c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06c4106f20 0x7f06c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 -- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 msgr2=0x7f06c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41027a0 0x7f06c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.938+0000 7f06c9c00640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a6900 con 0x7f06c4101df0 2024-09-05T11:55:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06c9c00640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4101df0 0x7f06c40744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9f0 tx=0x7f06b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ccc0 con 0x7f06c4101df0 2024-09-05T11:55:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8010910 con 0x7f06c4101df0 2024-09-05T11:55:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8018800 con 0x7f06c4101df0 2024-09-05T11:55:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a6bf0 con 0x7f06c4101df0 2024-09-05T11:55:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.939+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a70e0 con 0x7f06c4101df0 2024-09-05T11:55:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.940+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f06c40628c0 con 0x7f06c4101df0 2024-09-05T11:55:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.942+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f06b8010ac0 con 0x7f06c4101df0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.943+0000 7f06c2000640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f06b80cfd20 con 0x7f06c4101df0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c9200640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c2000640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] conn(0x7f06ac0adde0 0x7f06ac0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06b801ae50 con 0x7f06ac0adde0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f06b80d00d0 con 0x7f06c4101df0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06ca600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] conn(0x7f06ac0adde0 0x7f06ac0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.944+0000 7f06c9200640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f06b0031550 tx=0x7f06b0005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:44.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.945+0000 7f06ca600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] conn(0x7f06ac0adde0 0x7f06ac0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:44.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.946+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== osd.8 v2:172.21.15.189:6816/2486650777 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06b801ae50 con 0x7f06ac0adde0 2024-09-05T11:55:44.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.962+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 --> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f06c4062d40 con 0x7f06ac0adde0 2024-09-05T11:55:44.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.962+0000 7f06c2000640 1 -- 172.21.15.59:0/2511110760 <== osd.8 v2:172.21.15.189:6816/2486650777 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f06c4062d40 con 0x7f06ac0adde0 2024-09-05T11:55:44.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] conn(0x7f06ac0adde0 msgr2=0x7f06ac0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] conn(0x7f06ac0adde0 0x7f06ac0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f06ac0a2c10 msgr2=0x7f06ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f06ac0a2c10 0x7f06ac0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f06b0031550 tx=0x7f06b0005a20 comp rx=0 tx=0).stop 2024-09-05T11:55:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4101df0 msgr2=0x7f06c40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 --2- 172.21.15.59:0/2511110760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06c4101df0 0x7f06c40744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06b800e9f0 tx=0x7f06b800edf0 comp rx=0 tx=0).stop 2024-09-05T11:55:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06ca600640 1 -- 172.21.15.59:0/2511110760 reap_dead start 2024-09-05T11:55:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 shutdown_connections 2024-09-05T11:55:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 >> 172.21.15.59:0/2511110760 conn(0x7f06c40fd8b0 msgr2=0x7f06c40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 shutdown_connections 2024-09-05T11:55:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.963+0000 7f06cc600640 1 -- 172.21.15.59:0/2511110760 wait complete. 2024-09-05T11:55:45.186 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:46.101 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:45 smithi059 ceph-mon[30025]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 -- 172.21.15.59:0/880324179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c1049a0 msgr2=0x7fe70c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 --2- 172.21.15.59:0/880324179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c1049a0 0x7fe70c1089f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009900 tx=0x7fe6f802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 -- 172.21.15.59:0/880324179 shutdown_connections 2024-09-05T11:55:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 --2- 172.21.15.59:0/880324179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe70c109120 0x7fe70c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 --2- 172.21.15.59:0/880324179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c1049a0 0x7fe70c1089f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 --2- 172.21.15.59:0/880324179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70c103ff0 0x7fe70c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 -- 172.21.15.59:0/880324179 >> 172.21.15.59:0/880324179 conn(0x7fe70c0ffa90 msgr2=0x7fe70c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 -- 172.21.15.59:0/880324179 shutdown_connections 2024-09-05T11:55:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.121+0000 7fe714600640 1 -- 172.21.15.59:0/880324179 wait complete. 2024-09-05T11:55:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.122+0000 7fe714600640 1 Processor -- start 2024-09-05T11:55:46.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.122+0000 7fe714600640 1 -- start start 2024-09-05T11:55:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 0x7fe70c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe70c1049a0 0x7fe70c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70c109120 0x7fe70c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c114170 con 0x7fe70c103ff0 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c113ff0 con 0x7fe70c1049a0 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe714600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1142f0 con 0x7fe70c109120 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 0x7fe70c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 0x7fe70c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41140/0 (socket says 172.21.15.59:41140) 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711c00640 1 -- 172.21.15.59:0/1452477473 learned_addr learned my addr 172.21.15.59:0/1452477473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711200640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe70c1049a0 0x7fe70c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe712600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70c109120 0x7fe70c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711c00640 1 -- 172.21.15.59:0/1452477473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70c109120 msgr2=0x7fe70c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.123+0000 7fe711c00640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70c109120 0x7fe70c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe711c00640 1 -- 172.21.15.59:0/1452477473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe70c1049a0 msgr2=0x7fe70c19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe711c00640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe70c1049a0 0x7fe70c19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe711c00640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19f680 con 0x7fe70c103ff0 2024-09-05T11:55:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe711c00640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 0x7fe70c19a6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe70000ea60 tx=0x7fe70000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000cd20 con 0x7fe70c103ff0 2024-09-05T11:55:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19f970 con 0x7fe70c103ff0 2024-09-05T11:55:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7000035f0 con 0x7fe70c103ff0 2024-09-05T11:55:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700010690 con 0x7fe70c103ff0 2024-09-05T11:55:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.124+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a7250 con 0x7fe70c103ff0 2024-09-05T11:55:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.125+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe70c062900 con 0x7fe70c103ff0 2024-09-05T11:55:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.127+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe7000037a0 con 0x7fe70c103ff0 2024-09-05T11:55:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.128+0000 7fe70a000640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6f40a2c10 0x7fe6f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7000cfb30 con 0x7fe70c103ff0 2024-09-05T11:55:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe711200640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6f40a2c10 0x7fe6f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe70a000640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] conn(0x7fe6f40adde0 0x7fe6f40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe700023e50 con 0x7fe6f40adde0 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe7000cfee0 con 0x7fe70c103ff0 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.129+0000 7fe712600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] conn(0x7fe6f40adde0 0x7fe6f40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.130+0000 7fe711200640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6f40a2c10 0x7fe6f40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.130+0000 7fe712600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] conn(0x7fe6f40adde0 0x7fe6f40b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.131+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== osd.0 v2:172.21.15.59:6802/2347559609 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe700023e50 con 0x7fe6f40adde0 2024-09-05T11:55:46.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.145+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 --> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe70c062d00 con 0x7fe6f40adde0 2024-09-05T11:55:46.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.145+0000 7fe70a000640 1 -- 172.21.15.59:0/1452477473 <== osd.0 v2:172.21.15.59:6802/2347559609 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe70c062d00 con 0x7fe6f40adde0 2024-09-05T11:55:46.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] conn(0x7fe6f40adde0 msgr2=0x7fe6f40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] conn(0x7fe6f40adde0 0x7fe6f40b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6f40a2c10 msgr2=0x7fe6f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6f40a2c10 0x7fe6f40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8005b50 comp rx=0 tx=0).stop 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 msgr2=0x7fe70c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 --2- 172.21.15.59:0/1452477473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c103ff0 0x7fe70c19a6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe70000ea60 tx=0x7fe70000ee60 comp rx=0 tx=0).stop 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe712600640 1 -- 172.21.15.59:0/1452477473 reap_dead start 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 shutdown_connections 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 >> 172.21.15.59:0/1452477473 conn(0x7fe70c0ffa90 msgr2=0x7fe70c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 shutdown_connections 2024-09-05T11:55:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.146+0000 7fe714600640 1 -- 172.21.15.59:0/1452477473 wait complete. 2024-09-05T11:55:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:45 smithi089 ceph-mon[34499]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 -- 172.21.15.59:0/3356346824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074620 msgr2=0x7fea2c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 --2- 172.21.15.59:0/3356346824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074620 0x7fea2c074a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009900 tx=0x7fea1c02a480 comp rx=0 tx=0).stop 2024-09-05T11:55:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 -- 172.21.15.59:0/3356346824 shutdown_connections 2024-09-05T11:55:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 --2- 172.21.15.59:0/3356346824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c10e3c0 0x7fea2c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 --2- 172.21.15.59:0/3356346824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2c074f40 0x7fea2c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 --2- 172.21.15.59:0/3356346824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074620 0x7fea2c074a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.689+0000 7fea32e00640 1 -- 172.21.15.59:0/3356346824 >> 172.21.15.59:0/3356346824 conn(0x7fea2c0fd8c0 msgr2=0x7fea2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.690+0000 7fea32e00640 1 -- 172.21.15.59:0/3356346824 shutdown_connections 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.690+0000 7fea32e00640 1 -- 172.21.15.59:0/3356346824 wait complete. 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.690+0000 7fea32e00640 1 Processor -- start 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 -- start start 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 0x7fea2c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074f40 0x7fea2c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2c10e3c0 0x7fea2c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1141a0 con 0x7fea2c074f40 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c114020 con 0x7fea2c074620 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea32e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c114320 con 0x7fea2c10e3c0 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074f40 0x7fea2c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 0x7fea2c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 0x7fea2c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40552/0 (socket says 172.21.15.59:40552) 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea2be00640 1 -- 172.21.15.59:0/2222941801 learned_addr learned my addr 172.21.15.59:0/2222941801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.691+0000 7fea30e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2c10e3c0 0x7fea2c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 -- 172.21.15.59:0/2222941801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2c10e3c0 msgr2=0x7fea2c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2c10e3c0 0x7fea2c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 -- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074f40 msgr2=0x7fea2c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074f40 0x7fea2c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19f620 con 0x7fea2c074620 2024-09-05T11:55:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea23400640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea2c074f40 0x7fea2c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea2be00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 0x7fea2c19a670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c004770 tx=0x7fea1c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c03f070 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19f8b0 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c031ca0 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.692+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c035030 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.693+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a71f0 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.693+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea2c062980 con 0x7fea2c074620 2024-09-05T11:55:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.695+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea1c042070 con 0x7fea2c074620 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea29600640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea100a2c10 0x7fea100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fea1c0f0100 con 0x7fea2c074620 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea23400640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea100a2c10 0x7fea100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea29600640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] conn(0x7fea100adde0 0x7fea100b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea1c038e90 con 0x7fea100adde0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fea1c0f04b0 con 0x7fea2c074620 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.697+0000 7fea30e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] conn(0x7fea100adde0 0x7fea100b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.698+0000 7fea23400640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea100a2c10 0x7fea100a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea2c10f7b0 tx=0x7fea14005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.698+0000 7fea30e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] conn(0x7fea100adde0 0x7fea100b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.699+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== osd.1 v2:172.21.15.89:6800/3265971901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea1c038e90 con 0x7fea100adde0 2024-09-05T11:55:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.713+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 --> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea2c062e00 con 0x7fea100adde0 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.714+0000 7fea29600640 1 -- 172.21.15.59:0/2222941801 <== osd.1 v2:172.21.15.89:6800/3265971901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea2c062e00 con 0x7fea100adde0 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.714+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] conn(0x7fea100adde0 msgr2=0x7fea100b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.714+0000 7fea32e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] conn(0x7fea100adde0 0x7fea100b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea100a2c10 msgr2=0x7fea100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea100a2c10 0x7fea100a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea2c10f7b0 tx=0x7fea14005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 msgr2=0x7fea2c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 --2- 172.21.15.59:0/2222941801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea2c074620 0x7fea2c19a670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c004770 tx=0x7fea1c003c10 comp rx=0 tx=0).stop 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea30e00640 1 -- 172.21.15.59:0/2222941801 reap_dead start 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 shutdown_connections 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 >> 172.21.15.59:0/2222941801 conn(0x7fea2c0fd8c0 msgr2=0x7fea2c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 shutdown_connections 2024-09-05T11:55:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:44.715+0000 7fea32e00640 1 -- 172.21.15.59:0/2222941801 wait complete. 2024-09-05T11:55:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:45 smithi189 ceph-mon[34772]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 -- 172.21.15.59:0/3319000541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc074f40 msgr2=0x7f24cc10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 --2- 172.21.15.59:0/3319000541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc074f40 0x7f24cc10de80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a480 comp rx=0 tx=0).stop 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 -- 172.21.15.59:0/3319000541 shutdown_connections 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 --2- 172.21.15.59:0/3319000541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc10e3c0 0x7f24cc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 --2- 172.21.15.59:0/3319000541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc074f40 0x7f24cc10de80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 --2- 172.21.15.59:0/3319000541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 -- 172.21.15.59:0/3319000541 >> 172.21.15.59:0/3319000541 conn(0x7f24cc0fd8c0 msgr2=0x7f24cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 -- 172.21.15.59:0/3319000541 shutdown_connections 2024-09-05T11:55:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.789+0000 7f24d2600640 1 -- 172.21.15.59:0/3319000541 wait complete. 2024-09-05T11:55:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.790+0000 7f24d2600640 1 Processor -- start 2024-09-05T11:55:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 -- start start 2024-09-05T11:55:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc074f40 0x7f24cc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc10e3c0 0x7f24cc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1141b0 con 0x7f24cc10e3c0 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc114030 con 0x7f24cc074f40 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24d2600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc114330 con 0x7f24cc074620 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52262/0 (socket says 172.21.15.59:52262) 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24cb400640 1 -- 172.21.15.59:0/1318904268 learned_addr learned my addr 172.21.15.59:0/1318904268 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.791+0000 7f24cbe00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc10e3c0 0x7f24cc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24caa00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc074f40 0x7f24cc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 -- 172.21.15.59:0/1318904268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc074f40 msgr2=0x7f24cc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24cc074f40 0x7f24cc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 -- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc10e3c0 msgr2=0x7f24cc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc10e3c0 0x7f24cc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc19f660 con 0x7f24cc074620 2024-09-05T11:55:46.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.792+0000 7f24cb400640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc19a6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00eb90 tx=0x7f24bc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.793+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004b20 con 0x7f24cc074620 2024-09-05T11:55:46.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.793+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc19f950 con 0x7f24cc074620 2024-09-05T11:55:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.793+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc004cc0 con 0x7f24cc074620 2024-09-05T11:55:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.793+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1a7230 con 0x7f24cc074620 2024-09-05T11:55:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.793+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc010690 con 0x7f24cc074620 2024-09-05T11:55:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.794+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24cc062940 con 0x7f24cc074620 2024-09-05T11:55:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.796+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f24bc005000 con 0x7f24cc074620 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.797+0000 7f24c3e00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b00a2c10 0x7f24b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.797+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f24bc0cffe0 con 0x7f24cc074620 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.797+0000 7f24caa00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b00a2c10 0x7f24b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.798+0000 7f24c3e00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] conn(0x7f24b00adde0 0x7f24b00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.798+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24bc0047f0 con 0x7f24b00adde0 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.798+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f24bc0d0390 con 0x7f24cc074620 2024-09-05T11:55:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.798+0000 7f24cbe00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] conn(0x7f24b00adde0 0x7f24b00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.798+0000 7f24caa00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b00a2c10 0x7f24b00a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b4005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.799+0000 7f24cbe00640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] conn(0x7f24b00adde0 0x7f24b00b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.799+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== osd.2 v2:172.21.15.189:6800/3282762650 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24bc0047f0 con 0x7f24b00adde0 2024-09-05T11:55:46.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.814+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 --> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24cc062dc0 con 0x7f24b00adde0 2024-09-05T11:55:46.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24c3e00640 1 -- 172.21.15.59:0/1318904268 <== osd.2 v2:172.21.15.189:6800/3282762650 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f24cc062dc0 con 0x7f24b00adde0 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] conn(0x7f24b00adde0 msgr2=0x7f24b00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] conn(0x7f24b00adde0 0x7f24b00b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b00a2c10 msgr2=0x7f24b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b00a2c10 0x7f24b00a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b4005b50 comp rx=0 tx=0).stop 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 msgr2=0x7f24cc19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 --2- 172.21.15.59:0/1318904268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24cc074620 0x7f24cc19a6b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00eb90 tx=0x7f24bc00ef90 comp rx=0 tx=0).stop 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24cbe00640 1 -- 172.21.15.59:0/1318904268 reap_dead start 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 shutdown_connections 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 >> 172.21.15.59:0/1318904268 conn(0x7f24cc0fd8c0 msgr2=0x7f24cc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 shutdown_connections 2024-09-05T11:55:46.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.815+0000 7f24d2600640 1 -- 172.21.15.59:0/1318904268 wait complete. 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 -- 172.21.15.59:0/3582202048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001019c0 msgr2=0x7f2500101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 --2- 172.21.15.59:0/3582202048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001019c0 0x7f2500101e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009930 tx=0x7f24ec02a420 comp rx=0 tx=0).stop 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 -- 172.21.15.59:0/3582202048 shutdown_connections 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 --2- 172.21.15.59:0/3582202048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f250010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 --2- 172.21.15.59:0/3582202048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001019c0 0x7f2500101e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 --2- 172.21.15.59:0/3582202048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25001079c0 0x7f2500107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 -- 172.21.15.59:0/3582202048 >> 172.21.15.59:0/3582202048 conn(0x7f25000fd740 msgr2=0x7f25000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:47.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 -- 172.21.15.59:0/3582202048 shutdown_connections 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.693+0000 7f2505200640 1 -- 172.21.15.59:0/3582202048 wait complete. 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.694+0000 7f2505200640 1 Processor -- start 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.694+0000 7f2505200640 1 -- start start 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25001019c0 0x7f2500198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f25001988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 0x7f250019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2500111de0 con 0x7f25001079c0 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2500111c60 con 0x7f2500102360 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f2505200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2500111f60 con 0x7f25001019c0 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 0x7f250019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24fea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f25001988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:47.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24ff400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25001019c0 0x7f2500198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24fea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f25001988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40616/0 (socket says 172.21.15.59:40616) 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 0x7f250019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41156/0 (socket says 172.21.15.59:41156) 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.695+0000 7f24fea00640 1 -- 172.21.15.59:0/242914651 learned_addr learned my addr 172.21.15.59:0/242914651 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 -- 172.21.15.59:0/242914651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25001019c0 msgr2=0x7f2500198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25001019c0 0x7f2500198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 -- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 msgr2=0x7f25001988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f25001988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250019d310 con 0x7f25001079c0 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24fea00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2500102360 0x7f25001988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24ffe00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 0x7f250019cc30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009fd0 tx=0x7f24f800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800cd60 con 0x7f25001079c0 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250019d600 con 0x7f25001079c0 2024-09-05T11:55:47.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f8005020 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8010690 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.696+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a4ee0 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.697+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2500062940 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.700+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f24f8005540 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24f7e00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24e40a2c90 0x7f24e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f24f80cfdb0 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24ff400640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24e40a2c90 0x7f24e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24f7e00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] conn(0x7f24e40ade60 0x7f24e40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24f801ee50 con 0x7f24e40ade60 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.701+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f24f80d0160 con 0x7f25001079c0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.702+0000 7f24fea00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] conn(0x7f24e40ade60 0x7f24e40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:47.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.702+0000 7f24ff400640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24e40a2c90 0x7f24e40a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009990 tx=0x7f24f0009110 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:47.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.702+0000 7f24fea00640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] conn(0x7f24e40ade60 0x7f24e40b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:47.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.703+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== osd.7 v2:172.21.15.89:6816/2237080020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24f801ee50 con 0x7f24e40ade60 2024-09-05T11:55:47.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.717+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 --> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2500062dc0 con 0x7f24e40ade60 2024-09-05T11:55:47.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.718+0000 7f24f7e00640 1 -- 172.21.15.59:0/242914651 <== osd.7 v2:172.21.15.89:6816/2237080020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2500062dc0 con 0x7f24e40ade60 2024-09-05T11:55:47.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.718+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] conn(0x7f24e40ade60 msgr2=0x7f24e40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] conn(0x7f24e40ade60 0x7f24e40b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24e40a2c90 msgr2=0x7f24e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24e40a2c90 0x7f24e40a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009990 tx=0x7f24f0009110 comp rx=0 tx=0).stop 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 msgr2=0x7f250019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 --2- 172.21.15.59:0/242914651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25001079c0 0x7f250019cc30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009fd0 tx=0x7f24f800ee50 comp rx=0 tx=0).stop 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f24ffe00640 1 -- 172.21.15.59:0/242914651 reap_dead start 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 shutdown_connections 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 >> 172.21.15.59:0/242914651 conn(0x7f25000fd740 msgr2=0x7f25000fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 shutdown_connections 2024-09-05T11:55:47.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:46.719+0000 7f2505200640 1 -- 172.21.15.59:0/242914651 wait complete. 2024-09-05T11:55:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 -- 172.21.15.59:0/3182163461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 msgr2=0x7f3784110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 --2- 172.21.15.59:0/3182163461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f3784110ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 -- 172.21.15.59:0/3182163461 shutdown_connections 2024-09-05T11:55:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 --2- 172.21.15.59:0/3182163461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f3784110ca0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 --2- 172.21.15.59:0/3182163461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37841049a0 0x7f37841089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 --2- 172.21.15.59:0/3182163461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 0x7f37841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 -- 172.21.15.59:0/3182163461 >> 172.21.15.59:0/3182163461 conn(0x7f37840ffa90 msgr2=0x7f3784101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 -- 172.21.15.59:0/3182163461 shutdown_connections 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.002+0000 7f378a200640 1 -- 172.21.15.59:0/3182163461 wait complete. 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.003+0000 7f378a200640 1 Processor -- start 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.003+0000 7f378a200640 1 -- start start 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 0x7f378419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37841049a0 0x7f378419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f378419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3784114180 con 0x7f3784109120 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3784114000 con 0x7f3784103ff0 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f378a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3784114300 con 0x7f37841049a0 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f3783e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f378419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f3783400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 0x7f378419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f3783e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f378419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41198/0 (socket says 172.21.15.59:41198) 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f3783e00640 1 -- 172.21.15.59:0/104875899 learned_addr learned my addr 172.21.15.59:0/104875899 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.004+0000 7f377be00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37841049a0 0x7f378419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 -- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37841049a0 msgr2=0x7f378419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37841049a0 0x7f378419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 -- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 msgr2=0x7f378419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f378419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419f570 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783e00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3784109120 0x7f378419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3783400640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 0x7f378419a5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3774005fd0 tx=0x7f377400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774009c30 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419f860 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3774009dd0 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774010460 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.005+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a7140 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.006+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3784100fd0 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.008+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3774003ef0 con 0x7f3784103ff0 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.009+0000 7f3780c00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37680a2c90 0x7f37680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f37740cf3f0 con 0x7f3784103ff0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f377be00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37680a2c90 0x7f37680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f3780c00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] conn(0x7f37680adde0 0x7f37680b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f377401de50 con 0x7f37680adde0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f37740cf7a0 con 0x7f3784103ff0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.010+0000 7f377be00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37680a2c90 0x7f37680a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3784100340 tx=0x7f376c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.011+0000 7f3783e00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] conn(0x7f37680adde0 0x7f37680b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.011+0000 7f3783e00640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] conn(0x7f37680adde0 0x7f37680b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.012+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== osd.11 v2:172.21.15.189:6824/1305059632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f377401de50 con 0x7f37680adde0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.027+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 --> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37841054d0 con 0x7f37680adde0 2024-09-05T11:55:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.027+0000 7f3780c00640 1 -- 172.21.15.59:0/104875899 <== osd.11 v2:172.21.15.189:6824/1305059632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37841054d0 con 0x7f37680adde0 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] conn(0x7f37680adde0 msgr2=0x7f37680b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] conn(0x7f37680adde0 0x7f37680b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37680a2c90 msgr2=0x7f37680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37680a2c90 0x7f37680a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3784100340 tx=0x7f376c00b410 comp rx=0 tx=0).stop 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 msgr2=0x7f378419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 --2- 172.21.15.59:0/104875899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3784103ff0 0x7f378419a5c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3774005fd0 tx=0x7f377400dde0 comp rx=0 tx=0).stop 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f3783e00640 1 -- 172.21.15.59:0/104875899 reap_dead start 2024-09-05T11:55:48.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 shutdown_connections 2024-09-05T11:55:48.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 >> 172.21.15.59:0/104875899 conn(0x7f37840ffa90 msgr2=0x7f3784100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:48.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 shutdown_connections 2024-09-05T11:55:48.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:47.028+0000 7f378a200640 1 -- 172.21.15.59:0/104875899 wait complete. 2024-09-05T11:55:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:47 smithi089 ceph-mon[34499]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:48.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:47 smithi059 ceph-mon[30025]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:48.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:47 smithi189 ceph-mon[34772]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:55:48.829 INFO:teuthology.orchestra.run.smithi059.stdout:244813135917 2024-09-05T11:55:48.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-05T11:55:49.153 INFO:teuthology.orchestra.run.smithi059.stdout:210453397553 2024-09-05T11:55:49.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-05T11:55:49.469 INFO:teuthology.orchestra.run.smithi059.stdout:326417514524 2024-09-05T11:55:49.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-05T11:55:49.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:49 smithi059 ceph-mon[30025]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 -- 172.21.15.59:0/45232365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c108f60 msgr2=0x7f955c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 --2- 172.21.15.59:0/45232365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c108f60 0x7f955c110ae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9554009ca0 tx=0x7f955402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 -- 172.21.15.59:0/45232365 shutdown_connections 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 --2- 172.21.15.59:0/45232365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c108f60 0x7f955c110ae0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 --2- 172.21.15.59:0/45232365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f955c1047e0 0x7f955c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 --2- 172.21.15.59:0/45232365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c103e30 0x7f955c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 -- 172.21.15.59:0/45232365 >> 172.21.15.59:0/45232365 conn(0x7f955c0ff910 msgr2=0x7f955c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 -- 172.21.15.59:0/45232365 shutdown_connections 2024-09-05T11:55:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.003+0000 7f9560a00640 1 -- 172.21.15.59:0/45232365 wait complete. 2024-09-05T11:55:50.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.004+0000 7f9560a00640 1 Processor -- start 2024-09-05T11:55:50.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.004+0000 7f9560a00640 1 -- start start 2024-09-05T11:55:50.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f955c103e30 0x7f955c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 0x7f955c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c108f60 0x7f955c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f955c113fc0 con 0x7f955c1047e0 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f955c113e40 con 0x7f955c103e30 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f9560a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f955c114140 con 0x7f955c108f60 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f955aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 0x7f955c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f955aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 0x7f955c19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41222/0 (socket says 172.21.15.59:41222) 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f955aa00640 1 -- 172.21.15.59:0/324256790 learned_addr learned my addr 172.21.15.59:0/324256790 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f955b400640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f955c103e30 0x7f955c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.005+0000 7f955be00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c108f60 0x7f955c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 -- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c108f60 msgr2=0x7f955c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c108f60 0x7f955c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 -- 172.21.15.59:0/324256790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f955c103e30 msgr2=0x7f955c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f955c103e30 0x7f955c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c19f3d0 con 0x7f955c1047e0 2024-09-05T11:55:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955be00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c108f60 0x7f955c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f955aa00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 0x7f955c19a960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f954800ea60 tx=0x7f954800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954800cde0 con 0x7f955c1047e0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95480035f0 con 0x7f955c1047e0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c19f6c0 con 0x7f955c1047e0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.006+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548010670 con 0x7f955c1047e0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.007+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1a6fa0 con 0x7f955c1047e0 2024-09-05T11:55:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.007+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f955c062940 con 0x7f955c1047e0 2024-09-05T11:55:50.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.010+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f95480037a0 con 0x7f955c1047e0 2024-09-05T11:55:50.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f9553e00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f95400a2c10 0x7f95400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:50.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f95480cfb00 con 0x7f955c1047e0 2024-09-05T11:55:50.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f955b400640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f95400a2c10 0x7f95400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:50.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f9553e00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] conn(0x7f95400adde0 0x7f95400b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:50.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9548023e50 con 0x7f95400adde0 2024-09-05T11:55:50.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f95480cfeb0 con 0x7f955c1047e0 2024-09-05T11:55:50.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.012+0000 7f955be00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] conn(0x7f95400adde0 0x7f95400b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:50.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.013+0000 7f955b400640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f95400a2c10 0x7f95400a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f954c009bd0 tx=0x7f954c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:50.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.013+0000 7f955be00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] conn(0x7f95400adde0 0x7f95400b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:50.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.014+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== osd.5 v2:172.21.15.189:6808/3482558888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9548023e50 con 0x7f95400adde0 2024-09-05T11:55:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.029+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 --> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f955c062dc0 con 0x7f95400adde0 2024-09-05T11:55:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.029+0000 7f9553e00640 1 -- 172.21.15.59:0/324256790 <== osd.5 v2:172.21.15.189:6808/3482558888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f955c062dc0 con 0x7f95400adde0 2024-09-05T11:55:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] conn(0x7f95400adde0 msgr2=0x7f95400b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] conn(0x7f95400adde0 0x7f95400b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f95400a2c10 msgr2=0x7f95400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f95400a2c10 0x7f95400a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f954c009bd0 tx=0x7f954c005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 msgr2=0x7f955c19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 --2- 172.21.15.59:0/324256790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f955c1047e0 0x7f955c19a960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f954800ea60 tx=0x7f954800ef20 comp rx=0 tx=0).stop 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f955be00640 1 -- 172.21.15.59:0/324256790 reap_dead start 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 shutdown_connections 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 >> 172.21.15.59:0/324256790 conn(0x7f955c0ff910 msgr2=0x7f955c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 shutdown_connections 2024-09-05T11:55:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:50.030+0000 7f9560a00640 1 -- 172.21.15.59:0/324256790 wait complete. 2024-09-05T11:55:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:49 smithi089 ceph-mon[34499]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T11:55:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:49 smithi189 ceph-mon[34772]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T11:55:50.434 INFO:teuthology.orchestra.run.smithi059.stdout:34359738454 2024-09-05T11:55:50.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-05T11:55:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.061+0000 7f40a4a00640 1 -- 172.21.15.59:0/2834213815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0109120 msgr2=0x7f40a0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.061+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2834213815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0109120 0x7f40a0110ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4098009ca0 tx=0x7f409802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 -- 172.21.15.59:0/2834213815 shutdown_connections 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2834213815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0109120 0x7f40a0110ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2834213815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a01049a0 0x7f40a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2834213815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0103ff0 0x7f40a01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 -- 172.21.15.59:0/2834213815 >> 172.21.15.59:0/2834213815 conn(0x7f40a00ffa90 msgr2=0x7f40a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 -- 172.21.15.59:0/2834213815 shutdown_connections 2024-09-05T11:55:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.062+0000 7f40a4a00640 1 -- 172.21.15.59:0/2834213815 wait complete. 2024-09-05T11:55:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 Processor -- start 2024-09-05T11:55:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 -- start start 2024-09-05T11:55:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0103ff0 0x7f40a019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a01049a0 0x7f40a019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 0x7f40a019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40a0114130 con 0x7f40a0103ff0 2024-09-05T11:55:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40a0113fb0 con 0x7f40a01049a0 2024-09-05T11:55:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.063+0000 7f40a4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40a01142b0 con 0x7f40a0109120 2024-09-05T11:55:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.064+0000 7f409e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 0x7f40a019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.064+0000 7f409e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 0x7f40a019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:47896/0 (socket says 172.21.15.59:47896) 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.064+0000 7f409e800640 1 -- 172.21.15.59:0/2880803476 learned_addr learned my addr 172.21.15.59:0/2880803476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.064+0000 7f409de00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0103ff0 0x7f40a019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.064+0000 7f4095400640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a01049a0 0x7f40a019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 -- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a01049a0 msgr2=0x7f40a019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40a01049a0 0x7f40a019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 -- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0103ff0 msgr2=0x7f40a019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0103ff0 0x7f40a019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a019f5a0 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409de00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a0103ff0 0x7f40a019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f409e800640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 0x7f40a019eec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4098031a20 tx=0x7f4098031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098003b70 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a019f830 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4098003d10 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.065+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409802fc40 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.066+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a01a7170 con 0x7f40a0109120 2024-09-05T11:55:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.066+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40a0062900 con 0x7f40a0109120 2024-09-05T11:55:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.069+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4098004050 con 0x7f40a0109120 2024-09-05T11:55:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f4097400640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40840a2c10 0x7f40840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f40980f0940 con 0x7f40a0109120 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f409de00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40840a2c10 0x7f40840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f4097400640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] conn(0x7f40840adde0 0x7f40840b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40840b0950 con 0x7f40840adde0 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.070+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f40980f0cf0 con 0x7f40a0109120 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.071+0000 7f4095400640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] conn(0x7f40840adde0 0x7f40840b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.071+0000 7f409de00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40840a2c10 0x7f40840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4090009c20 tx=0x7f4090005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.071+0000 7f4095400640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] conn(0x7f40840adde0 0x7f40840b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.072+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== osd.4 v2:172.21.15.89:6808/200597957 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40840b0950 con 0x7f40840adde0 2024-09-05T11:55:51.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.087+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 --> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40a0062d00 con 0x7f40840adde0 2024-09-05T11:55:51.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.087+0000 7f4097400640 1 -- 172.21.15.59:0/2880803476 <== osd.4 v2:172.21.15.89:6808/200597957 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f40a0062d00 con 0x7f40840adde0 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] conn(0x7f40840adde0 msgr2=0x7f40840b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] conn(0x7f40840adde0 0x7f40840b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40840a2c10 msgr2=0x7f40840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40840a2c10 0x7f40840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4090009c20 tx=0x7f4090005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 msgr2=0x7f40a019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 --2- 172.21.15.59:0/2880803476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40a0109120 0x7f40a019eec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4098031a20 tx=0x7f4098031a50 comp rx=0 tx=0).stop 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f409e800640 1 -- 172.21.15.59:0/2880803476 reap_dead start 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 shutdown_connections 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 >> 172.21.15.59:0/2880803476 conn(0x7f40a00ffa90 msgr2=0x7f40a0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 shutdown_connections 2024-09-05T11:55:51.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.088+0000 7f40a4a00640 1 -- 172.21.15.59:0/2880803476 wait complete. 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 -- 172.21.15.59:0/2753260378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a581049a0 msgr2=0x7f4a581089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 --2- 172.21.15.59:0/2753260378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a581049a0 0x7f4a581089f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009900 tx=0x7f4a4002a410 comp rx=0 tx=0).stop 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 -- 172.21.15.59:0/2753260378 shutdown_connections 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 --2- 172.21.15.59:0/2753260378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58109120 0x7f4a58110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 --2- 172.21.15.59:0/2753260378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a581049a0 0x7f4a581089f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 --2- 172.21.15.59:0/2753260378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58103ff0 0x7f4a581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 -- 172.21.15.59:0/2753260378 >> 172.21.15.59:0/2753260378 conn(0x7f4a580ffa90 msgr2=0x7f4a58101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 -- 172.21.15.59:0/2753260378 shutdown_connections 2024-09-05T11:55:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.591+0000 7f4a5e400640 1 -- 172.21.15.59:0/2753260378 wait complete. 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.592+0000 7f4a5e400640 1 Processor -- start 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.592+0000 7f4a5e400640 1 -- start start 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.592+0000 7f4a5e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 0x7f4a5819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.592+0000 7f4a5e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a581049a0 0x7f4a5819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a5e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109120 0x7f4a5819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a5e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58114140 con 0x7f4a58109120 2024-09-05T11:55:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a5e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58113fc0 con 0x7f4a58103ff0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a5e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581142c0 con 0x7f4a581049a0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 0x7f4a5819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109120 0x7f4a5819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a4fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a581049a0 0x7f4a5819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 0x7f4a5819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40168/0 (socket says 172.21.15.59:40168) 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 -- 172.21.15.59:0/1586331162 learned_addr learned my addr 172.21.15.59:0/1586331162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 -- 172.21.15.59:0/1586331162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a581049a0 msgr2=0x7f4a5819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a581049a0 0x7f4a5819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 -- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109120 msgr2=0x7f4a5819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a58109120 0x7f4a5819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.593+0000 7f4a57400640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5819f5d0 con 0x7f4a58103ff0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a57400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 0x7f4a5819a620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800cd80 con 0x7f4a58103ff0 2024-09-05T11:55:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5819f8c0 con 0x7f4a58103ff0 2024-09-05T11:55:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48005500 con 0x7f4a58103ff0 2024-09-05T11:55:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48010690 con 0x7f4a58103ff0 2024-09-05T11:55:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.594+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a71a0 con 0x7f4a58103ff0 2024-09-05T11:55:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.595+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a58062980 con 0x7f4a58103ff0 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.597+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a480046c0 con 0x7f4a58103ff0 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.598+0000 7f4a54c00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.598+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4a480cfdd0 con 0x7f4a58103ff0 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a4fe00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a54c00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] conn(0x7f4a3c0adde0 0x7f4a3c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a48005350 con 0x7f4a3c0adde0 2024-09-05T11:55:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4a480d0180 con 0x7f4a58103ff0 2024-09-05T11:55:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a57e00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] conn(0x7f4a3c0adde0 0x7f4a3c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.599+0000 7f4a4fe00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009230 tx=0x7f4a400058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.600+0000 7f4a57e00640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] conn(0x7f4a3c0adde0 0x7f4a3c0b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.601+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== osd.6 v2:172.21.15.59:6818/3110186175 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a48005350 con 0x7f4a3c0adde0 2024-09-05T11:55:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.615+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 --> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a58062e00 con 0x7f4a3c0adde0 2024-09-05T11:55:51.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.615+0000 7f4a54c00640 1 -- 172.21.15.59:0/1586331162 <== osd.6 v2:172.21.15.59:6818/3110186175 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a58062e00 con 0x7f4a3c0adde0 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] conn(0x7f4a3c0adde0 msgr2=0x7f4a3c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] conn(0x7f4a3c0adde0 0x7f4a3c0b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4a3c0a2c10 msgr2=0x7f4a3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009230 tx=0x7f4a400058d0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 msgr2=0x7f4a5819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 --2- 172.21.15.59:0/1586331162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a58103ff0 0x7f4a5819a620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800edf0 comp rx=0 tx=0).stop 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a57e00640 1 -- 172.21.15.59:0/1586331162 reap_dead start 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 shutdown_connections 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 >> 172.21.15.59:0/1586331162 conn(0x7f4a580ffa90 msgr2=0x7f4a58100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 shutdown_connections 2024-09-05T11:55:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:51.616+0000 7f4a5e400640 1 -- 172.21.15.59:0/1586331162 wait complete. 2024-09-05T11:55:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:51 smithi059 ceph-mon[30025]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T11:55:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:51 smithi189 ceph-mon[34772]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T11:55:52.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:51 smithi089 ceph-mon[34499]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T11:55:52.725 INFO:teuthology.orchestra.run.smithi059.stdout:158913790013 2024-09-05T11:55:52.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-05T11:55:53.922 INFO:teuthology.orchestra.run.smithi059.stdout:137438953537 2024-09-05T11:55:53.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 -- 172.21.15.59:0/3669680978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 msgr2=0x7f4d7810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/3669680978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7810e460 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009ca0 tx=0x7f4d7002a950 comp rx=0 tx=0).stop 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 -- 172.21.15.59:0/3669680978 shutdown_connections 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/3669680978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7810e460 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/3669680978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d781034b0 0x7f4d7810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/3669680978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d78102b90 0x7f4d78102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 -- 172.21.15.59:0/3669680978 >> 172.21.15.59:0/3669680978 conn(0x7f4d780fc610 msgr2=0x7f4d780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:54.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:53 smithi059 ceph-mon[30025]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:54.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.136+0000 7f4d7ec00640 1 -- 172.21.15.59:0/3669680978 shutdown_connections 2024-09-05T11:55:54.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.137+0000 7f4d7ec00640 1 -- 172.21.15.59:0/3669680978 wait complete. 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.138+0000 7f4d7ec00640 1 Processor -- start 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 -- start start 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78102b90 0x7f4d7819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d781034b0 0x7f4d7819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78111dd0 con 0x7f4d7810bfc0 2024-09-05T11:55:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78111c50 con 0x7f4d78102b90 2024-09-05T11:55:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78111f50 con 0x7f4d781034b0 2024-09-05T11:55:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d781034b0 0x7f4d7819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46070/0 (socket says 172.21.15.59:46070) 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78102b90 0x7f4d7819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.139+0000 7f4d7e200640 1 -- 172.21.15.59:0/2810565474 learned_addr learned my addr 172.21.15.59:0/2810565474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 -- 172.21.15.59:0/2810565474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d781034b0 msgr2=0x7f4d7819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d781034b0 0x7f4d7819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 -- 172.21.15.59:0/2810565474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78102b90 msgr2=0x7f4d7819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78102b90 0x7f4d7819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7819f420 con 0x7f4d7810bfc0 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7d800640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78102b90 0x7f4d7819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d7e200640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7819ed40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009dc0 tx=0x7f4d70003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.140+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70034040 con 0x7f4d7810bfc0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.141+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7819f6b0 con 0x7f4d7810bfc0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.141+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a6ff0 con 0x7f4d7810bfc0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.141+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70003e50 con 0x7f4d7810bfc0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.141+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d700046c0 con 0x7f4d7810bfc0 2024-09-05T11:55:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.142+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d58001030 con 0x7f4d7810bfc0 2024-09-05T11:55:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.144+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4d70037020 con 0x7f4d7810bfc0 2024-09-05T11:55:54.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.145+0000 7f4d76000640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d600a2c90 0x7f4d600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4d700f0290 con 0x7f4d7810bfc0 2024-09-05T11:55:54.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d7d800640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d600a2c90 0x7f4d600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d76000640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] conn(0x7f4d600ade60 0x7f4d600b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d70033e50 con 0x7f4d600ade60 2024-09-05T11:55:54.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4d700f0640 con 0x7f4d7810bfc0 2024-09-05T11:55:54.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.146+0000 7f4d7ce00640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] conn(0x7f4d600ade60 0x7f4d600b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.147+0000 7f4d7d800640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d600a2c90 0x7f4d600a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005d00 tx=0x7f4d6c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.147+0000 7f4d7ce00640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] conn(0x7f4d600ade60 0x7f4d600b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.148+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== osd.3 v2:172.21.15.59:6810/2051936210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d70033e50 con 0x7f4d600ade60 2024-09-05T11:55:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.163+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 --> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d58002dd0 con 0x7f4d600ade60 2024-09-05T11:55:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.163+0000 7f4d76000640 1 -- 172.21.15.59:0/2810565474 <== osd.3 v2:172.21.15.59:6810/2051936210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d58002dd0 con 0x7f4d600ade60 2024-09-05T11:55:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] conn(0x7f4d600ade60 msgr2=0x7f4d600b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] conn(0x7f4d600ade60 0x7f4d600b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d600a2c90 msgr2=0x7f4d600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d600a2c90 0x7f4d600a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005d00 tx=0x7f4d6c00b410 comp rx=0 tx=0).stop 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 msgr2=0x7f4d7819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 --2- 172.21.15.59:0/2810565474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7810bfc0 0x7f4d7819ed40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009dc0 tx=0x7f4d70003ce0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7e200640 1 -- 172.21.15.59:0/2810565474 reap_dead start 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 shutdown_connections 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 >> 172.21.15.59:0/2810565474 conn(0x7f4d780fc610 msgr2=0x7f4d7810d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 shutdown_connections 2024-09-05T11:55:54.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:54.164+0000 7f4d7ec00640 1 -- 172.21.15.59:0/2810565474 wait complete. 2024-09-05T11:55:54.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:53 smithi189 ceph-mon[34772]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:54.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:53 smithi089 ceph-mon[34499]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:54.668 INFO:teuthology.orchestra.run.smithi059.stdout:55834574929 2024-09-05T11:55:54.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-05T11:55:54.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 -- 172.21.15.59:0/437280571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed8108f60 msgr2=0x7f7ed8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 --2- 172.21.15.59:0/437280571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed8108f60 0x7f7ed8110ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009ca0 tx=0x7f7ed002a8e0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 -- 172.21.15.59:0/437280571 shutdown_connections 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 --2- 172.21.15.59:0/437280571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed8108f60 0x7f7ed8110ae0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 --2- 172.21.15.59:0/437280571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ed81047e0 0x7f7ed8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 --2- 172.21.15.59:0/437280571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8103e30 0x7f7ed8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.325+0000 7f7edda00640 1 -- 172.21.15.59:0/437280571 >> 172.21.15.59:0/437280571 conn(0x7f7ed80ff910 msgr2=0x7f7ed8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.326+0000 7f7edda00640 1 -- 172.21.15.59:0/437280571 shutdown_connections 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.326+0000 7f7edda00640 1 -- 172.21.15.59:0/437280571 wait complete. 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.326+0000 7f7edda00640 1 Processor -- start 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 -- start start 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ed8103e30 0x7f7ed819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 0x7f7ed819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8108f60 0x7f7ed819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8114050 con 0x7f7ed81047e0 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8113ed0 con 0x7f7ed8103e30 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7edda00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81141d0 con 0x7f7ed8108f60 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7ed7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 0x7f7ed819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7ed7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 0x7f7ed819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46058/0 (socket says 172.21.15.59:46058) 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.327+0000 7f7ed7400640 1 -- 172.21.15.59:0/2114149942 learned_addr learned my addr 172.21.15.59:0/2114149942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7e00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ed8103e30 0x7f7ed819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7edd000640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8108f60 0x7f7ed819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 -- 172.21.15.59:0/2114149942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8108f60 msgr2=0x7f7ed819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ed8108f60 0x7f7ed819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 -- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ed8103e30 msgr2=0x7f7ed819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ed8103e30 0x7f7ed819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed819f460 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed7400640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 0x7f7ed819a9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e9f0 tx=0x7f7ec400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400cd80 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec4010910 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed819f750 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.328+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4018990 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.329+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a7030 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.330+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ed8100f60 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.332+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7ec4010ac0 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed4c00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ebc0a2c90 0x7f7ebc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7ec40cff80 con 0x7f7ed81047e0 2024-09-05T11:55:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed7e00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ebc0a2c90 0x7f7ebc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed4c00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] conn(0x7f7ebc0ade60 0x7f7ebc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ec4010760 con 0x7f7ebc0ade60 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.333+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7ec40d0330 con 0x7f7ed81047e0 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.334+0000 7f7edd000640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] conn(0x7f7ebc0ade60 0x7f7ebc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.334+0000 7f7ed7e00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ebc0a2c90 0x7f7ebc0a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006fd0 tx=0x7f7ec80090e0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.334+0000 7f7edd000640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] conn(0x7f7ebc0ade60 0x7f7ebc0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.335+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== osd.10 v2:172.21.15.89:6824/1393916736 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ec4010760 con 0x7f7ebc0ade60 2024-09-05T11:55:54.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.358+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 --> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ed8105310 con 0x7f7ebc0ade60 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7ed4c00640 1 -- 172.21.15.59:0/2114149942 <== osd.10 v2:172.21.15.89:6824/1393916736 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ed8105310 con 0x7f7ebc0ade60 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] conn(0x7f7ebc0ade60 msgr2=0x7f7ebc0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] conn(0x7f7ebc0ade60 0x7f7ebc0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ebc0a2c90 msgr2=0x7f7ebc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7ebc0a2c90 0x7f7ebc0a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8006fd0 tx=0x7f7ec80090e0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 msgr2=0x7f7ed819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 --2- 172.21.15.59:0/2114149942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ed81047e0 0x7f7ed819a9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e9f0 tx=0x7f7ec400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edd000640 1 -- 172.21.15.59:0/2114149942 reap_dead start 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 shutdown_connections 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 >> 172.21.15.59:0/2114149942 conn(0x7f7ed80ff910 msgr2=0x7f7ed8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 shutdown_connections 2024-09-05T11:55:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:52.359+0000 7f7edda00640 1 -- 172.21.15.59:0/2114149942 wait complete. 2024-09-05T11:55:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:55 smithi059 ceph-mon[30025]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:56.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:55 smithi189 ceph-mon[34772]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:55 smithi089 ceph-mon[34499]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:55:56.475 INFO:teuthology.orchestra.run.smithi059.stdout:188978561079 2024-09-05T11:55:56.475 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-05T11:55:56.493 INFO:teuthology.orchestra.run.smithi059.stdout:77309411404 2024-09-05T11:55:56.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-05T11:55:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 -- 172.21.15.59:0/517854267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa801049a0 msgr2=0x7faa801089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 --2- 172.21.15.59:0/517854267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa801049a0 0x7faa801089f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009900 tx=0x7faa6c02a410 comp rx=0 tx=0).stop 2024-09-05T11:55:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 -- 172.21.15.59:0/517854267 shutdown_connections 2024-09-05T11:55:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 --2- 172.21.15.59:0/517854267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80109120 0x7faa80110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 --2- 172.21.15.59:0/517854267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa801049a0 0x7faa801089f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 --2- 172.21.15.59:0/517854267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 0x7faa801043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 -- 172.21.15.59:0/517854267 >> 172.21.15.59:0/517854267 conn(0x7faa800ffa90 msgr2=0x7faa80101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 -- 172.21.15.59:0/517854267 shutdown_connections 2024-09-05T11:55:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.695+0000 7faa85c00640 1 -- 172.21.15.59:0/517854267 wait complete. 2024-09-05T11:55:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.696+0000 7faa85c00640 1 Processor -- start 2024-09-05T11:55:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.696+0000 7faa85c00640 1 -- start start 2024-09-05T11:55:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 0x7faa8019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801049a0 0x7faa8019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 0x7faa8019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa80114170 con 0x7faa80109120 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa80113ff0 con 0x7faa80103ff0 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa85c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa801142f0 con 0x7faa801049a0 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa7f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 0x7faa8019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa7e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801049a0 0x7faa8019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa7f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 0x7faa8019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46078/0 (socket says 172.21.15.59:46078) 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa7f400640 1 -- 172.21.15.59:0/3625238568 learned_addr learned my addr 172.21.15.59:0/3625238568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.697+0000 7faa7ea00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 0x7faa8019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 -- 172.21.15.59:0/3625238568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 msgr2=0x7faa8019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 0x7faa8019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 -- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 msgr2=0x7faa8019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 0x7faa8019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8019f5e0 con 0x7faa801049a0 2024-09-05T11:55:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7f400640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa80109120 0x7faa8019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7ea00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa80103ff0 0x7faa8019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.698+0000 7faa7e000640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801049a0 0x7faa8019ab70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa6c02a930 tx=0x7faa6c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.699+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c03f070 con 0x7faa801049a0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.699+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6c031c90 con 0x7faa801049a0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.699+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8019f870 con 0x7faa801049a0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.699+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c035030 con 0x7faa801049a0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.699+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a71b0 con 0x7faa801049a0 2024-09-05T11:55:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.700+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa80062980 con 0x7faa801049a0 2024-09-05T11:55:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.704+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faa6c042070 con 0x7faa801049a0 2024-09-05T11:55:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa77400640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa640a2c10 0x7faa640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faa6c0efba0 con 0x7faa801049a0 2024-09-05T11:55:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa7ea00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa640a2c10 0x7faa640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa77400640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] conn(0x7faa640adde0 0x7faa640b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa6c005640 con 0x7faa640adde0 2024-09-05T11:55:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.705+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7faa6c0f42a0 con 0x7faa801049a0 2024-09-05T11:55:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.706+0000 7faa7f400640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] conn(0x7faa640adde0 0x7faa640b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.706+0000 7faa7ea00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa640a2c10 0x7faa640a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa70009fc0 tx=0x7faa70005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.707+0000 7faa7f400640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] conn(0x7faa640adde0 0x7faa640b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.707+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== osd.9 v2:172.21.15.59:6826/1620616877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa6c005640 con 0x7faa640adde0 2024-09-05T11:55:56.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.722+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 --> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa80062e00 con 0x7faa640adde0 2024-09-05T11:55:56.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa77400640 1 -- 172.21.15.59:0/3625238568 <== osd.9 v2:172.21.15.59:6826/1620616877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faa80062e00 con 0x7faa640adde0 2024-09-05T11:55:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] conn(0x7faa640adde0 msgr2=0x7faa640b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] conn(0x7faa640adde0 0x7faa640b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa640a2c10 msgr2=0x7faa640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa640a2c10 0x7faa640a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa70009fc0 tx=0x7faa70005f90 comp rx=0 tx=0).stop 2024-09-05T11:55:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801049a0 msgr2=0x7faa8019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.723+0000 7faa85c00640 1 --2- 172.21.15.59:0/3625238568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801049a0 0x7faa8019ab70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa6c02a930 tx=0x7faa6c003b90 comp rx=0 tx=0).stop 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.724+0000 7faa7f400640 1 -- 172.21.15.59:0/3625238568 reap_dead start 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.724+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 shutdown_connections 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.724+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 >> 172.21.15.59:0/3625238568 conn(0x7faa800ffa90 msgr2=0x7faa80100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.724+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 shutdown_connections 2024-09-05T11:55:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:56.724+0000 7faa85c00640 1 -- 172.21.15.59:0/3625238568 wait complete. 2024-09-05T11:55:57.035 INFO:teuthology.orchestra.run.smithi059.stdout:107374182472 2024-09-05T11:55:57.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-05T11:55:57.180 INFO:teuthology.orchestra.run.smithi059.stdout:296352743459 2024-09-05T11:55:57.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-05T11:55:57.994 INFO:teuthology.orchestra.run.smithi059.stdout:274877906986 2024-09-05T11:55:57.994 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-05T11:55:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.069 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.202 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:57 smithi059 ceph-mon[30025]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:55:58.226 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.239 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.239 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.240 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.244 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:55:58.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:57 smithi189 ceph-mon[34772]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:55:58.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:57 smithi089 ceph-mon[34499]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 -- 172.21.15.59:0/2316318823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c102790 msgr2=0x7f847c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 --2- 172.21.15.59:0/2316318823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c102790 0x7f847c1067e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402a410 comp rx=0 tx=0).stop 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 -- 172.21.15.59:0/2316318823 shutdown_connections 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 --2- 172.21.15.59:0/2316318823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c106f10 0x7f847c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 --2- 172.21.15.59:0/2316318823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c102790 0x7f847c1067e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 --2- 172.21.15.59:0/2316318823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c101de0 0x7f847c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 -- 172.21.15.59:0/2316318823 >> 172.21.15.59:0/2316318823 conn(0x7f847c0fd8c0 msgr2=0x7f847c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:59.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 -- 172.21.15.59:0/2316318823 shutdown_connections 2024-09-05T11:55:59.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.517+0000 7f8481c00640 1 -- 172.21.15.59:0/2316318823 wait complete. 2024-09-05T11:55:59.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.518+0000 7f8481c00640 1 Processor -- start 2024-09-05T11:55:59.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.518+0000 7f8481c00640 1 -- start start 2024-09-05T11:55:59.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:59.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c102790 0x7f847c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:59.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 0x7f847c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f847c111fc0 con 0x7f847c101de0 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f847c111e40 con 0x7f847c106f10 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8481c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f847c112140 con 0x7f847c102790 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f847aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f847aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46104/0 (socket says 172.21.15.59:46104) 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f847aa00640 1 -- 172.21.15.59:0/2641368171 learned_addr learned my addr 172.21.15.59:0/2641368171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f847b400640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 0x7f847c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.519+0000 7f8472000640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c102790 0x7f847c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 -- 172.21.15.59:0/2641368171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c102790 msgr2=0x7f847c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c102790 0x7f847c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 -- 172.21.15.59:0/2641368171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 msgr2=0x7f847c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 0x7f847c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 -- 172.21.15.59:0/2641368171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c19f6b0 con 0x7f847c101de0 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847b400640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 0x7f847c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f847aa00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d950 tx=0x7f846c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c017030 con 0x7f847c101de0 2024-09-05T11:55:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c19f9a0 con 0x7f847c101de0 2024-09-05T11:55:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c004020 con 0x7f847c101de0 2024-09-05T11:55:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c0047a0 con 0x7f847c101de0 2024-09-05T11:55:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.520+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c1a7280 con 0x7f847c101de0 2024-09-05T11:55:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.522+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847c1021c0 con 0x7f847c101de0 2024-09-05T11:55:59.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.524+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f846c009a30 con 0x7f847c101de0 2024-09-05T11:55:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.526+0000 7f8473e00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 0x7f84600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:55:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.526+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f846c0cec70 con 0x7f847c101de0 2024-09-05T11:55:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.526+0000 7f8472000640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 0x7f84600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:55:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.527+0000 7f8472000640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 0x7f84600a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8464000c00 tx=0x7f8464005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:55:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.535+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f846c09b6b0 con 0x7f847c101de0 2024-09-05T11:55:59.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.725+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f847c1a74b0 con 0x7f847c101de0 2024-09-05T11:55:59.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.726+0000 7f8473e00640 1 -- 172.21.15.59:0/2641368171 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f846c09afb0 con 0x7f847c101de0 2024-09-05T11:55:59.728 INFO:teuthology.orchestra.run.smithi059.stdout:55834574932 2024-09-05T11:55:59.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 msgr2=0x7f84600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 0x7f84600a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8464000c00 tx=0x7f8464005b50 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 msgr2=0x7f847c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d950 tx=0x7f846c00de10 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 shutdown_connections 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84600a2c10 0x7f84600a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f847c106f10 0x7f847c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f847c102790 0x7f847c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 --2- 172.21.15.59:0/2641368171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f847c101de0 0x7f847c19a700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 >> 172.21.15.59:0/2641368171 conn(0x7f847c0fd8c0 msgr2=0x7f847c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 shutdown_connections 2024-09-05T11:55:59.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:55:59.728+0000 7f8481c00640 1 -- 172.21.15.59:0/2641368171 wait complete. 2024-09-05T11:56:00.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:59 smithi059 ceph-mon[30025]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:00.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:55:59 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2641368171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 -- 172.21.15.59:0/3788260405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8601049a0 msgr2=0x7fb8601089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 --2- 172.21.15.59:0/3788260405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8601049a0 0x7fb8601089f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb848009ca0 tx=0x7fb84802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 -- 172.21.15.59:0/3788260405 shutdown_connections 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 --2- 172.21.15.59:0/3788260405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860109120 0x7fb860110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 --2- 172.21.15.59:0/3788260405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8601049a0 0x7fb8601089f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 --2- 172.21.15.59:0/3788260405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb8601043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 -- 172.21.15.59:0/3788260405 >> 172.21.15.59:0/3788260405 conn(0x7fb8600ffa90 msgr2=0x7fb860101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 -- 172.21.15.59:0/3788260405 shutdown_connections 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.096+0000 7fb865400640 1 -- 172.21.15.59:0/3788260405 wait complete. 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.097+0000 7fb865400640 1 Processor -- start 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.097+0000 7fb865400640 1 -- start start 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8601049a0 0x7fb86019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 0x7fb86019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb860114190 con 0x7fb860109120 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb860114010 con 0x7fb860103ff0 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb865400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb860114310 con 0x7fb8601049a0 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb85e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb85f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 0x7fb86019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb85e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40256/0 (socket says 172.21.15.59:40256) 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb85e800640 1 -- 172.21.15.59:0/1265887518 learned_addr learned my addr 172.21.15.59:0/1265887518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.098+0000 7fb85de00640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8601049a0 0x7fb86019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 -- 172.21.15.59:0/1265887518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8601049a0 msgr2=0x7fb86019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8601049a0 0x7fb86019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 -- 172.21.15.59:0/1265887518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 msgr2=0x7fb86019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 -- 172.21.15.59:0/1265887518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86019f620 con 0x7fb860109120 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85e800640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb85f200640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 0x7fb86019ef40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e970 tx=0x7fb85800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800ccc0 con 0x7fb860109120 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.099+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb858005020 con 0x7fb860109120 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.100+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86019f910 con 0x7fb860109120 2024-09-05T11:56:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.100+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004a00 con 0x7fb860109120 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.100+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a71f0 con 0x7fb860109120 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.101+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8601043d0 con 0x7fb860109120 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.103+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb8580051d0 con 0x7fb860109120 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.105+0000 7fb857400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 0x7fb8400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.105+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb8580cf520 con 0x7fb860109120 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.105+0000 7fb85e800640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 0x7fb8400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.106+0000 7fb85e800640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 0x7fb8400a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb850011550 tx=0x7fb850005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.107+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb85809bff0 con 0x7fb860109120 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 -- 172.21.15.59:0/1920665012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 msgr2=0x7fdf901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 --2- 172.21.15.59:0/1920665012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf901089f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009900 tx=0x7fdf7c02a410 comp rx=0 tx=0).stop 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 -- 172.21.15.59:0/1920665012 shutdown_connections 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 --2- 172.21.15.59:0/1920665012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90109120 0x7fdf90110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 --2- 172.21.15.59:0/1920665012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf901089f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 --2- 172.21.15.59:0/1920665012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90103ff0 0x7fdf901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 -- 172.21.15.59:0/1920665012 >> 172.21.15.59:0/1920665012 conn(0x7fdf900ffa90 msgr2=0x7fdf90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 -- 172.21.15.59:0/1920665012 shutdown_connections 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.210+0000 7fdf97600640 1 -- 172.21.15.59:0/1920665012 wait complete. 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 Processor -- start 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 -- start start 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf9019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90109120 0x7fdf9019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf90114140 con 0x7fdf90109120 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf90113fc0 con 0x7fdf90103ff0 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.211+0000 7fdf97600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf901142c0 con 0x7fdf901049a0 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40268/0 (socket says 172.21.15.59:40268) 2024-09-05T11:56:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 -- 172.21.15.59:0/1476084539 learned_addr learned my addr 172.21.15.59:0/1476084539 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf9019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf95600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90109120 0x7fdf9019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 -- 172.21.15.59:0/1476084539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 msgr2=0x7fdf9019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf9019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 -- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90109120 msgr2=0x7fdf9019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90109120 0x7fdf9019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 -- 172.21.15.59:0/1476084539 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9019f650 con 0x7fdf90103ff0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.212+0000 7fdf94c00640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e9f0 tx=0x7fdf8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.213+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400cd80 con 0x7fdf90103ff0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.213+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf84010910 con 0x7fdf90103ff0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.213+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9019f940 con 0x7fdf90103ff0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.213+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84018990 con 0x7fdf90103ff0 2024-09-05T11:56:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.213+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf901a7220 con 0x7fdf90103ff0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.214+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf901043d0 con 0x7fdf90103ff0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.216+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdf84010ac0 con 0x7fdf90103ff0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.217+0000 7fdf8d600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 0x7fdf780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.217+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdf840cfe10 con 0x7fdf90103ff0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.217+0000 7fdf8fe00640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 0x7fdf780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.218+0000 7fdf8fe00640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 0x7fdf780a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009230 tx=0x7fdf7c0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.219+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf8409c200 con 0x7fdf90103ff0 2024-09-05T11:56:00.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.299+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb86010c3c0 con 0x7fb860109120 2024-09-05T11:56:00.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.299+0000 7fb857400640 1 -- 172.21.15.59:0/1265887518 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb85809b8f0 con 0x7fb860109120 2024-09-05T11:56:00.301 INFO:teuthology.orchestra.run.smithi059.stdout:34359738457 2024-09-05T11:56:00.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 msgr2=0x7fb8400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 0x7fb8400a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb850011550 tx=0x7fb850005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:00.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 msgr2=0x7fb86019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 0x7fb86019ef40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e970 tx=0x7fb85800ee30 comp rx=0 tx=0).stop 2024-09-05T11:56:00.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 shutdown_connections 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb8400a2c10 0x7fb8400a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb860109120 0x7fb86019ef40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8601049a0 0x7fb86019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 --2- 172.21.15.59:0/1265887518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb860103ff0 0x7fb86019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 >> 172.21.15.59:0/1265887518 conn(0x7fb8600ffa90 msgr2=0x7fb860100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.302+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 shutdown_connections 2024-09-05T11:56:00.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.303+0000 7fb865400640 1 -- 172.21.15.59:0/1265887518 wait complete. 2024-09-05T11:56:00.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:59 smithi189 ceph-mon[34772]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:00.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:55:59 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2641368171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T11:56:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.409+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdf9019ba00 con 0x7fdf90103ff0 2024-09-05T11:56:00.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.410+0000 7fdf8d600640 1 -- 172.21.15.59:0/1476084539 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf9019ba00 con 0x7fdf90103ff0 2024-09-05T11:56:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:77309411407 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 msgr2=0x7fdf780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 0x7fdf780a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009230 tx=0x7fdf7c0058d0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 msgr2=0x7fdf9019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e9f0 tx=0x7fdf8400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 shutdown_connections 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdf780a2c10 0x7fdf780a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf90109120 0x7fdf9019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf901049a0 0x7fdf9019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 --2- 172.21.15.59:0/1476084539 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf90103ff0 0x7fdf9019a6a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 >> 172.21.15.59:0/1476084539 conn(0x7fdf900ffa90 msgr2=0x7fdf90100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:00.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 shutdown_connections 2024-09-05T11:56:00.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.412+0000 7fdf97600640 1 -- 172.21.15.59:0/1476084539 wait complete. 2024-09-05T11:56:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:59 smithi089 ceph-mon[34499]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:55:59 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2641368171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T11:56:00.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.816+0000 7fd939000640 1 -- 172.21.15.59:0/3591886480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9341049c0 msgr2=0x7fd934108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.816+0000 7fd939000640 1 --2- 172.21.15.59:0/3591886480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9341049c0 0x7fd934108a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 -- 172.21.15.59:0/3591886480 shutdown_connections 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 --2- 172.21.15.59:0/3591886480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd934109140 0x7fd934110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 --2- 172.21.15.59:0/3591886480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9341049c0 0x7fd934108a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 --2- 172.21.15.59:0/3591886480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 0x7fd9341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 -- 172.21.15.59:0/3591886480 >> 172.21.15.59:0/3591886480 conn(0x7fd9340ffa50 msgr2=0x7fd934101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 -- 172.21.15.59:0/3591886480 shutdown_connections 2024-09-05T11:56:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.817+0000 7fd939000640 1 -- 172.21.15.59:0/3591886480 wait complete. 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 Processor -- start 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 -- start start 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 0x7fd93419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 0x7fd93419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd934114140 con 0x7fd934109140 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113fc0 con 0x7fd9341049c0 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd939000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341142c0 con 0x7fd934104010 2024-09-05T11:56:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.818+0000 7fd932800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 0x7fd93419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd932800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48282/0 (socket says 172.21.15.59:48282) 2024-09-05T11:56:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd932800640 1 -- 172.21.15.59:0/1592504745 learned_addr learned my addr 172.21.15.59:0/1592504745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 0x7fd93419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 -- 172.21.15.59:0/1592504745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 msgr2=0x7fd93419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 0x7fd93419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 -- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 msgr2=0x7fd93419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd931400640 1 -- 172.21.15.59:0/1592504745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419f610 con 0x7fd9341049c0 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.819+0000 7fd932800640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd931400640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 0x7fd93419aba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd920005940 tx=0x7fd9200040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92003f030 con 0x7fd9341049c0 2024-09-05T11:56:00.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9200044c0 con 0x7fd9341049c0 2024-09-05T11:56:00.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419f8a0 con 0x7fd9341049c0 2024-09-05T11:56:00.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920035070 con 0x7fd9341049c0 2024-09-05T11:56:00.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.820+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a71e0 con 0x7fd9341049c0 2024-09-05T11:56:00.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.821+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9341043f0 con 0x7fd9341049c0 2024-09-05T11:56:00.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.823+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd920042030 con 0x7fd9341049c0 2024-09-05T11:56:00.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.824+0000 7fd92aa00640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 0x7fd9180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:00.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.825+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd9200efa60 con 0x7fd9341049c0 2024-09-05T11:56:00.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.825+0000 7fd931e00640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 0x7fd9180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:00.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.825+0000 7fd931e00640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 0x7fd9180a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd924009bd0 tx=0x7fd924005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:00.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:00.827+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9200bbe50 con 0x7fd9341049c0 2024-09-05T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:00 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1265887518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T11:56:00.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:00 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1476084539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T11:56:01.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.026+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd93402d7c0 con 0x7fd9341049c0 2024-09-05T11:56:01.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.027+0000 7fd92aa00640 1 -- 172.21.15.59:0/1592504745 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd9200c0150 con 0x7fd9341049c0 2024-09-05T11:56:01.029 INFO:teuthology.orchestra.run.smithi059.stdout:210453397556 2024-09-05T11:56:01.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.029+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 msgr2=0x7fd9180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:01.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.029+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 0x7fd9180a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd924009bd0 tx=0x7fd924005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:01.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.029+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 msgr2=0x7fd93419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:01.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.029+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 0x7fd93419aba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd920005940 tx=0x7fd9200040c0 comp rx=0 tx=0).stop 2024-09-05T11:56:01.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 shutdown_connections 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd9180a2c90 0x7fd9180a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd934109140 0x7fd93419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9341049c0 0x7fd93419aba0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 --2- 172.21.15.59:0/1592504745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd934104010 0x7fd93419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 >> 172.21.15.59:0/1592504745 conn(0x7fd9340ffa50 msgr2=0x7fd934100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 shutdown_connections 2024-09-05T11:56:01.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.030+0000 7fd939000640 1 -- 172.21.15.59:0/1592504745 wait complete. 2024-09-05T11:56:01.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:00 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1265887518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T11:56:01.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:00 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1476084539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T11:56:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:00 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1265887518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T11:56:01.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:00 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1476084539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T11:56:02.127 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:01 smithi059 ceph-mon[30025]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:02.127 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:01 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1592504745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T11:56:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.123+0000 7fa58da00640 1 -- 172.21.15.59:0/2497295361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 msgr2=0x7fa58810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.123+0000 7fa58da00640 1 --2- 172.21.15.59:0/2497295361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa58810e890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa580009be0 tx=0x7fa58002a6d0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 -- 172.21.15.59:0/2497295361 shutdown_connections 2024-09-05T11:56:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 --2- 172.21.15.59:0/2497295361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa58810e890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 --2- 172.21.15.59:0/2497295361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881019c0 0x7fa588101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 --2- 172.21.15.59:0/2497295361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881079c0 0x7fa588107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 -- 172.21.15.59:0/2497295361 >> 172.21.15.59:0/2497295361 conn(0x7fa5880fd740 msgr2=0x7fa5880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 -- 172.21.15.59:0/2497295361 shutdown_connections 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.124+0000 7fa58da00640 1 -- 172.21.15.59:0/2497295361 wait complete. 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.125+0000 7fa58da00640 1 Processor -- start 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.125+0000 7fa58da00640 1 -- start start 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa588074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881079c0 0x7fa588074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa588111dc0 con 0x7fa5881019c0 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa588111c40 con 0x7fa588102360 2024-09-05T11:56:02.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa58da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa588111f40 con 0x7fa5881079c0 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa587e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa587e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48320/0 (socket says 172.21.15.59:48320) 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa57fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa588074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.126+0000 7fa587e00640 1 -- 172.21.15.59:0/3334845122 learned_addr learned my addr 172.21.15.59:0/3334845122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa58d000640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881079c0 0x7fa588074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 -- 172.21.15.59:0/3334845122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881079c0 msgr2=0x7fa588074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881079c0 0x7fa588074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 -- 172.21.15.59:0/3334845122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 msgr2=0x7fa588074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa588074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 -- 172.21.15.59:0/3334845122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5881a66e0 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa57fe00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa588074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.127+0000 7fa587e00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9c0 tx=0x7fa57800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.128+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57800ccc0 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.128+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa578005500 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.128+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578010430 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.128+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5881a69d0 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.128+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5881a6f10 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.129+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa588101e20 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.132+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa5780046c0 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.134+0000 7fa585600640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 0x7fa56c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.135+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa5780cf8a0 con 0x7fa5881019c0 2024-09-05T11:56:02.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.135+0000 7fa57fe00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 0x7fa56c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.135+0000 7fa57fe00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 0x7fa56c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa588075f80 tx=0x7fa570005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.139+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa57809c2e0 con 0x7fa5881019c0 2024-09-05T11:56:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 -- 172.21.15.59:0/490961661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 msgr2=0x7f5a6c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 --2- 172.21.15.59:0/490961661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c101e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009930 tx=0x7f5a5802a420 comp rx=0 tx=0).stop 2024-09-05T11:56:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 -- 172.21.15.59:0/490961661 shutdown_connections 2024-09-05T11:56:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 --2- 172.21.15.59:0/490961661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c102360 0x7f5a6c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 --2- 172.21.15.59:0/490961661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c101e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 --2- 172.21.15.59:0/490961661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c1079c0 0x7f5a6c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.990+0000 7f5a72000640 1 -- 172.21.15.59:0/490961661 >> 172.21.15.59:0/490961661 conn(0x7f5a6c0fd740 msgr2=0x7f5a6c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.991+0000 7f5a72000640 1 -- 172.21.15.59:0/490961661 shutdown_connections 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.991+0000 7f5a72000640 1 -- 172.21.15.59:0/490961661 wait complete. 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.991+0000 7f5a72000640 1 Processor -- start 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 -- start start 2024-09-05T11:56:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 0x7f5a6c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c1079c0 0x7f5a6c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111e10 con 0x7f5a6c1019c0 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111c90 con 0x7f5a6c102360 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a72000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111f90 con 0x7f5a6c1079c0 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48298/0 (socket says 172.21.15.59:48298) 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a70c00640 1 -- 172.21.15.59:0/746290559 learned_addr learned my addr 172.21.15.59:0/746290559 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a6be00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 0x7f5a6c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.992+0000 7f5a71600640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c1079c0 0x7f5a6c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 -- 172.21.15.59:0/746290559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c1079c0 msgr2=0x7f5a6c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c1079c0 0x7f5a6c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 -- 172.21.15.59:0/746290559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 msgr2=0x7f5a6c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 0x7f5a6c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 -- 172.21.15.59:0/746290559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c19f470 con 0x7f5a6c1019c0 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a6be00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 0x7f5a6c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a70c00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9f0 tx=0x7f5a6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000cd80 con 0x7f5a6c1019c0 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60010910 con 0x7f5a6c1019c0 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c19f760 con 0x7f5a6c1019c0 2024-09-05T11:56:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.993+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60018990 con 0x7f5a6c1019c0 2024-09-05T11:56:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.994+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1a7040 con 0x7f5a6c1019c0 2024-09-05T11:56:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.995+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c101e20 con 0x7f5a6c1019c0 2024-09-05T11:56:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.996+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a60010ac0 con 0x7f5a6c1019c0 2024-09-05T11:56:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.998+0000 7f5a69600640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 0x7f5a540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:02.311 INFO:teuthology.orchestra.run.smithi059.stdout:244813135920 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.998+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5a600cf760 con 0x7f5a6c1019c0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.998+0000 7f5a6be00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 0x7f5a540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:01.999+0000 7f5a6be00640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 0x7f5a540a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006010 tx=0x7f5a58031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.001+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a6009bb50 con 0x7f5a6c1019c0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.196+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5a6c1a7240 con 0x7f5a6c1019c0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.197+0000 7f5a69600640 1 -- 172.21.15.59:0/746290559 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a6009fe50 con 0x7f5a6c1019c0 2024-09-05T11:56:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 msgr2=0x7f5a540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 0x7f5a540a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006010 tx=0x7f5a58031510 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 msgr2=0x7f5a6c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9f0 tx=0x7f5a6000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 shutdown_connections 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a540a2c90 0x7f5a540a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a6c1079c0 0x7f5a6c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a6c102360 0x7f5a6c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 --2- 172.21.15.59:0/746290559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c1019c0 0x7f5a6c19a4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 >> 172.21.15.59:0/746290559 conn(0x7f5a6c0fd740 msgr2=0x7f5a6c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.199+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 shutdown_connections 2024-09-05T11:56:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.200+0000 7f5a72000640 1 -- 172.21.15.59:0/746290559 wait complete. 2024-09-05T11:56:02.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.332+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa5881a7230 con 0x7fa5881019c0 2024-09-05T11:56:02.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.332+0000 7fa585600640 1 -- 172.21.15.59:0/3334845122 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa57809bbe0 con 0x7fa5881019c0 2024-09-05T11:56:02.334 INFO:teuthology.orchestra.run.smithi059.stdout:158913790015 2024-09-05T11:56:02.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:01 smithi089 ceph-mon[34499]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:02.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:01 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1592504745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T11:56:02.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 msgr2=0x7fa56c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 0x7fa56c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa588075f80 tx=0x7fa570005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:02.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 msgr2=0x7fa5880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:02.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9c0 tx=0x7fa57800ee80 comp rx=0 tx=0).stop 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 shutdown_connections 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa56c0a2c10 0x7fa56c0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881079c0 0x7fa588074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa588102360 0x7fa588074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 --2- 172.21.15.59:0/3334845122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5881019c0 0x7fa5880744f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.334+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 >> 172.21.15.59:0/3334845122 conn(0x7fa5880fd740 msgr2=0x7fa5880fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.335+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 shutdown_connections 2024-09-05T11:56:02.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:02.335+0000 7fa58da00640 1 -- 172.21.15.59:0/3334845122 wait complete. 2024-09-05T11:56:02.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:01 smithi189 ceph-mon[34772]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:02.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:01 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1592504745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T11:56:02.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411407 for osd.2 2024-09-05T11:56:02.894 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:03.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/746290559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T11:56:03.107 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3334845122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T11:56:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.296+0000 7f6396c00640 1 -- 172.21.15.59:0/2567003560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 msgr2=0x7f6390108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.296+0000 7f6396c00640 1 --2- 172.21.15.59:0/2567003560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f6390108a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6378009860 tx=0x7f637802a340 comp rx=0 tx=0).stop 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 -- 172.21.15.59:0/2567003560 shutdown_connections 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 --2- 172.21.15.59:0/2567003560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 0x7f6390110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 --2- 172.21.15.59:0/2567003560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f6390108a10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 --2- 172.21.15.59:0/2567003560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 0x7f63901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 -- 172.21.15.59:0/2567003560 >> 172.21.15.59:0/2567003560 conn(0x7f63900ffa50 msgr2=0x7f6390101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 -- 172.21.15.59:0/2567003560 shutdown_connections 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.297+0000 7f6396c00640 1 -- 172.21.15.59:0/2567003560 wait complete. 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 Processor -- start 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 -- start start 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 0x7f639019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:03.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 0x7f639019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6390114140 con 0x7f6390109140 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6390113fc0 con 0x7f63901049c0 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.298+0000 7f6396c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63901142c0 con 0x7f6390104010 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6394c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 0x7f639019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f638fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 0x7f639019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:54020/0 (socket says 172.21.15.59:54020) 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 -- 172.21.15.59:0/489548896 learned_addr learned my addr 172.21.15.59:0/489548896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 -- 172.21.15.59:0/489548896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 msgr2=0x7f639019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 0x7f639019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 -- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 msgr2=0x7f639019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 0x7f639019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 -- 172.21.15.59:0/489548896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639019f600 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.299+0000 7f6387e00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6378035040 tx=0x7f6378033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.300+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637803f020 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.300+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639019f890 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.300+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6378032770 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.300+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378037040 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.300+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63901a71d0 con 0x7f63901049c0 2024-09-05T11:56:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.301+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63901043f0 con 0x7f63901049c0 2024-09-05T11:56:03.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.303+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6378042020 con 0x7f63901049c0 2024-09-05T11:56:03.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.304+0000 7f638d600640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 0x7f63740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:03.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.304+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f63780ef6f0 con 0x7f63901049c0 2024-09-05T11:56:03.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.304+0000 7f638fe00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 0x7f63740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:03.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.305+0000 7f638fe00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 0x7f63740a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6380009bd0 tx=0x7f6380005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:03.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.306+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6378049020 con 0x7f63901049c0 2024-09-05T11:56:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/746290559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T11:56:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3334845122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/746290559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3334845122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T11:56:03.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.496+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f639010c3b0 con 0x7f63901049c0 2024-09-05T11:56:03.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.497+0000 7f638d600640 1 -- 172.21.15.59:0/489548896 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f63780bc240 con 0x7f63901049c0 2024-09-05T11:56:03.499 INFO:teuthology.orchestra.run.smithi059.stdout:296352743461 2024-09-05T11:56:03.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 msgr2=0x7f63740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:03.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 0x7f63740a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6380009bd0 tx=0x7f6380005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:03.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 msgr2=0x7f639019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:03.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6378035040 tx=0x7f6378033d90 comp rx=0 tx=0).stop 2024-09-05T11:56:03.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 shutdown_connections 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63740a2c10 0x7f63740a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6390109140 0x7f639019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63901049c0 0x7f639019ab90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 --2- 172.21.15.59:0/489548896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6390104010 0x7f639019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 >> 172.21.15.59:0/489548896 conn(0x7f63900ffa50 msgr2=0x7f6390100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.499+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 shutdown_connections 2024-09-05T11:56:03.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:03.500+0000 7f6396c00640 1 -- 172.21.15.59:0/489548896 wait complete. 2024-09-05T11:56:04.106 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397553 got 210453397556 for osd.7 2024-09-05T11:56:04.106 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:04.268 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:03 smithi059 ceph-mon[30025]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:04.268 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:03 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/489548896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T11:56:04.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:03 smithi189 ceph-mon[34772]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:04.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:03 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/489548896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T11:56:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 -- 172.21.15.59:0/3954503380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4103e30 msgr2=0x7f55d4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 --2- 172.21.15.59:0/3954503380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4103e30 0x7f55d4104210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009900 tx=0x7f55c802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 -- 172.21.15.59:0/3954503380 shutdown_connections 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 --2- 172.21.15.59:0/3954503380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 0x7f55d4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 --2- 172.21.15.59:0/3954503380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d41047e0 0x7f55d4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 --2- 172.21.15.59:0/3954503380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d4103e30 0x7f55d4104210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 -- 172.21.15.59:0/3954503380 >> 172.21.15.59:0/3954503380 conn(0x7f55d40ff8d0 msgr2=0x7f55d4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.382+0000 7f55da400640 1 -- 172.21.15.59:0/3954503380 shutdown_connections 2024-09-05T11:56:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.383+0000 7f55da400640 1 -- 172.21.15.59:0/3954503380 wait complete. 2024-09-05T11:56:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.383+0000 7f55da400640 1 Processor -- start 2024-09-05T11:56:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 -- start start 2024-09-05T11:56:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 0x7f55d419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 0x7f55d419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4114010 con 0x7f55d41047e0 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4113e90 con 0x7f55d4103e30 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55da400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4114190 con 0x7f55d4108f60 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48354/0 (socket says 172.21.15.59:48354) 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55d3e00640 1 -- 172.21.15.59:0/1119709449 learned_addr learned my addr 172.21.15.59:0/1119709449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55d9000640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 0x7f55d419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.384+0000 7f55d9a00640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 0x7f55d419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 -- 172.21.15.59:0/1119709449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 msgr2=0x7f55d419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 0x7f55d419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 -- 172.21.15.59:0/1119709449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 msgr2=0x7f55d419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 0x7f55d419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 -- 172.21.15.59:0/1119709449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d419f480 con 0x7f55d41047e0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d9000640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 0x7f55d419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d3e00640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c000d950 tx=0x7f55c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0017030 con 0x7f55d41047e0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c0004020 con 0x7f55d41047e0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d419f770 con 0x7f55d41047e0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.385+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00047a0 con 0x7f55d41047e0 2024-09-05T11:56:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.386+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41a7050 con 0x7f55d41047e0 2024-09-05T11:56:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.387+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d4104210 con 0x7f55d41047e0 2024-09-05T11:56:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.389+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f55c00041d0 con 0x7f55d41047e0 2024-09-05T11:56:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.391+0000 7f55d1600640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 0x7f55bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.391+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f55c00cef90 con 0x7f55d41047e0 2024-09-05T11:56:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.391+0000 7f55d9000640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 0x7f55bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.391+0000 7f55d9000640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 0x7f55bc0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.394+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55c009b9d0 con 0x7f55d41047e0 2024-09-05T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:03 smithi089 ceph-mon[34499]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:03 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/489548896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T11:56:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.583+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f55d410bbb0 con 0x7f55d41047e0 2024-09-05T11:56:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.584+0000 7f55d1600640 1 -- 172.21.15.59:0/1119709449 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f55c009b2d0 con 0x7f55d41047e0 2024-09-05T11:56:04.585 INFO:teuthology.orchestra.run.smithi059.stdout:326417514527 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.586+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 msgr2=0x7f55bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.586+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 0x7f55bc0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8031510 comp rx=0 tx=0).stop 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.586+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 msgr2=0x7f55d419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.586+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c000d950 tx=0x7f55c000de10 comp rx=0 tx=0).stop 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 shutdown_connections 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55bc0a2c10 0x7f55bc0a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55d4108f60 0x7f55d419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d41047e0 0x7f55d419aa10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 --2- 172.21.15.59:0/1119709449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55d4103e30 0x7f55d419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 >> 172.21.15.59:0/1119709449 conn(0x7f55d40ff8d0 msgr2=0x7f55d4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 shutdown_connections 2024-09-05T11:56:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:04.587+0000 7f55da400640 1 -- 172.21.15.59:0/1119709449 wait complete. 2024-09-05T11:56:05.221 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:04 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1119709449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T11:56:05.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:04 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1119709449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T11:56:05.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1119709449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T11:56:06.112 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:05 smithi059 ceph-mon[30025]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-05T11:56:06.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:05 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4198477877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T11:56:06.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:05 smithi189 ceph-mon[34772]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-05T11:56:06.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:05 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4198477877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T11:56:06.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 -- 172.21.15.59:0/2405029825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 msgr2=0x7f587810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:06.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 --2- 172.21.15.59:0/2405029825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587810dee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5864009900 tx=0x7f586402a480 comp rx=0 tx=0).stop 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 -- 172.21.15.59:0/2405029825 shutdown_connections 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 --2- 172.21.15.59:0/2405029825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 0x7f5878110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 --2- 172.21.15.59:0/2405029825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587810dee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 --2- 172.21.15.59:0/2405029825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f58780749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 -- 172.21.15.59:0/2405029825 >> 172.21.15.59:0/2405029825 conn(0x7f58780fd8c0 msgr2=0x7f58780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 -- 172.21.15.59:0/2405029825 shutdown_connections 2024-09-05T11:56:06.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.419+0000 7f587ce00640 1 -- 172.21.15.59:0/2405029825 wait complete. 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.420+0000 7f587ce00640 1 Processor -- start 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 -- start start 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f587819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 0x7f587819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5878114160 con 0x7f5878074ee0 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5878113fe0 con 0x7f587810e420 2024-09-05T11:56:06.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f587ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58781142e0 con 0x7f58780745c0 2024-09-05T11:56:06.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.421+0000 7f5875400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:06.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48388/0 (socket says 172.21.15.59:48388) 2024-09-05T11:56:06.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875400640 1 -- 172.21.15.59:0/540599280 learned_addr learned my addr 172.21.15.59:0/540599280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:06.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f587819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:06.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5876800640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 0x7f587819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 -- 172.21.15.59:0/540599280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 msgr2=0x7f587819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 0x7f587819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 -- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 msgr2=0x7f587819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 -- 172.21.15.59:0/540599280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587819f630 con 0x7f58780745c0 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875400640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.422+0000 7f5875e00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f587819a680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f586800d6d0 tx=0x7f586800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.423+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868017070 con 0x7f58780745c0 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.423+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5868009820 con 0x7f58780745c0 2024-09-05T11:56:06.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.423+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868004d00 con 0x7f58780745c0 2024-09-05T11:56:06.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.423+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587819f920 con 0x7f58780745c0 2024-09-05T11:56:06.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.423+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a7200 con 0x7f58780745c0 2024-09-05T11:56:06.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.424+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58780749a0 con 0x7f58780745c0 2024-09-05T11:56:06.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.426+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f58680099d0 con 0x7f58780745c0 2024-09-05T11:56:06.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.427+0000 7f586ea00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 0x7f585c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:06.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.428+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f58680ceff0 con 0x7f58780745c0 2024-09-05T11:56:06.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.428+0000 7f5875400640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 0x7f585c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.428+0000 7f5875400640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 0x7f585c0a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5864031550 tx=0x7f5864005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.432+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f586809b400 con 0x7f58780745c0 2024-09-05T11:56:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:05 smithi089 ceph-mon[34499]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-05T11:56:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:05 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4198477877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T11:56:06.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.624+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5878109600 con 0x7f58780745c0 2024-09-05T11:56:06.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.624+0000 7f586ea00640 1 -- 172.21.15.59:0/540599280 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f586809f700 con 0x7f58780745c0 2024-09-05T11:56:06.626 INFO:teuthology.orchestra.run.smithi059.stdout:188978561082 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.626+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 msgr2=0x7f585c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.626+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 0x7f585c0a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5864031550 tx=0x7f5864005a20 comp rx=0 tx=0).stop 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.626+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 msgr2=0x7f587819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.626+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f587819a680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f586800d6d0 tx=0x7f586800db90 comp rx=0 tx=0).stop 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 shutdown_connections 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f585c0a2c90 0x7f585c0a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587810e420 0x7f587819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5878074ee0 0x7f587819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 --2- 172.21.15.59:0/540599280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58780745c0 0x7f587819a680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 >> 172.21.15.59:0/540599280 conn(0x7f58780fd8c0 msgr2=0x7f58780fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 shutdown_connections 2024-09-05T11:56:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:06.627+0000 7f587ce00640 1 -- 172.21.15.59:0/540599280 wait complete. 2024-09-05T11:56:06.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514524 got 326417514527 for osd.11 2024-09-05T11:56:06.787 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:07.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 -- 172.21.15.59:0/3282408121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 msgr2=0x7fe8e4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:07.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 --2- 172.21.15.59:0/3282408121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e4110cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009be0 tx=0x7fe8dc02a6c0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 -- 172.21.15.59:0/3282408121 shutdown_connections 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 --2- 172.21.15.59:0/3282408121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e4110cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 --2- 172.21.15.59:0/3282408121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 0x7fe8e4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 --2- 172.21.15.59:0/3282408121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 -- 172.21.15.59:0/3282408121 >> 172.21.15.59:0/3282408121 conn(0x7fe8e40ffa50 msgr2=0x7fe8e4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 -- 172.21.15.59:0/3282408121 shutdown_connections 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.558+0000 7fe8eb800640 1 -- 172.21.15.59:0/3282408121 wait complete. 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.559+0000 7fe8eb800640 1 Processor -- start 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.559+0000 7fe8eb800640 1 -- start start 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 0x7fe8e419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4114150 con 0x7fe8e4104010 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4113fd0 con 0x7fe8e4109140 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8eb800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41142d0 con 0x7fe8e41049c0 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8e8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8e8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48370/0 (socket says 172.21.15.59:48370) 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8e8e00640 1 -- 172.21.15.59:0/4198477877 learned_addr learned my addr 172.21.15.59:0/4198477877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.560+0000 7fe8e3e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 0x7fe8e419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e9800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 -- 172.21.15.59:0/4198477877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 msgr2=0x7fe8e419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 0x7fe8e419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 -- 172.21.15.59:0/4198477877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 msgr2=0x7fe8e419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 -- 172.21.15.59:0/4198477877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e419f660 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e9800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e8e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e9c0 tx=0x7fe8d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800ccc0 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.561+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8005500 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.562+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e419f950 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.562+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8010430 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.562+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41a7230 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.563+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e41043f0 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.565+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8d80046c0 con 0x7fe8e4104010 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.567+0000 7fe8e1600640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 0x7fe8cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.567+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe8d80cf910 con 0x7fe8e4104010 2024-09-05T11:56:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.567+0000 7fe8e3e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 0x7fe8cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.568+0000 7fe8e3e00640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 0x7fe8cc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe8e410aa00 tx=0x7fe8d0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.571+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8d809c350 con 0x7fe8e4104010 2024-09-05T11:56:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.764+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe8e410c3e0 con 0x7fe8e4104010 2024-09-05T11:56:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.764+0000 7fe8e1600640 1 -- 172.21.15.59:0/4198477877 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe8d809bc50 con 0x7fe8e4104010 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stdout:137438953540 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.766+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 msgr2=0x7fe8cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.766+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 0x7fe8cc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe8e410aa00 tx=0x7fe8d0005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.766+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 msgr2=0x7fe8e419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.766+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e9c0 tx=0x7fe8d800ee80 comp rx=0 tx=0).stop 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 shutdown_connections 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe8cc0a2c10 0x7fe8cc0a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8e4109140 0x7fe8e419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e41049c0 0x7fe8e419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 --2- 172.21.15.59:0/4198477877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8e4104010 0x7fe8e419a6b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 >> 172.21.15.59:0/4198477877 conn(0x7fe8e40ffa50 msgr2=0x7fe8e4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 shutdown_connections 2024-09-05T11:56:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:05.767+0000 7fe8eb800640 1 -- 172.21.15.59:0/4198477877 wait complete. 2024-09-05T11:56:07.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:06 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/540599280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T11:56:07.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:06 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/540599280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T11:56:07.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:06 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/540599280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T11:56:07.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790013 got 158913790015 for osd.5 2024-09-05T11:56:07.519 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:08.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743459 got 296352743461 for osd.10 2024-09-05T11:56:08.086 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:08.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:07 smithi189 ceph-mon[34772]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:08.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:07 smithi059 ceph-mon[30025]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:08.381 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:08.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574932 for osd.1 2024-09-05T11:56:08.411 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:07 smithi089 ceph-mon[34499]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:08.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738457 for osd.0 2024-09-05T11:56:08.737 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:09.055 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135917 got 244813135920 for osd.8 2024-09-05T11:56:09.055 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:09.946 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561079 got 188978561082 for osd.6 2024-09-05T11:56:09.946 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 -- 172.21.15.59:0/445918002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac101de0 msgr2=0x7f3dac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 --2- 172.21.15.59:0/445918002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac101de0 0x7f3dac1021c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009900 tx=0x7f3da002a410 comp rx=0 tx=0).stop 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 -- 172.21.15.59:0/445918002 shutdown_connections 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 --2- 172.21.15.59:0/445918002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac106f10 0x7f3dac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 --2- 172.21.15.59:0/445918002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac102790 0x7f3dac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 --2- 172.21.15.59:0/445918002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac101de0 0x7f3dac1021c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 -- 172.21.15.59:0/445918002 >> 172.21.15.59:0/445918002 conn(0x7f3dac0fd8c0 msgr2=0x7f3dac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.587+0000 7f3db3e00640 1 -- 172.21.15.59:0/445918002 shutdown_connections 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.588+0000 7f3db3e00640 1 -- 172.21.15.59:0/445918002 wait complete. 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.588+0000 7f3db3e00640 1 Processor -- start 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 -- start start 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 0x7f3dac1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac106f10 0x7f3dac10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac111f70 con 0x7f3dac102790 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac111df0 con 0x7f3dac106f10 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1120f0 con 0x7f3dac101de0 2024-09-05T11:56:10.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48418/0 (socket says 172.21.15.59:48418) 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db0a00640 1 -- 172.21.15.59:0/134356943 learned_addr learned my addr 172.21.15.59:0/134356943 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db1400640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 0x7f3dac1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.589+0000 7f3db1e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac106f10 0x7f3dac10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 -- 172.21.15.59:0/134356943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac106f10 msgr2=0x7f3dac10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac106f10 0x7f3dac10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 -- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 msgr2=0x7f3dac1a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 -- 172.21.15.59:0/134356943 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac10e3c0 con 0x7f3dac101de0 2024-09-05T11:56:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db0a00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3db1400640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 0x7f3dac1a4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005940 tx=0x7f3da0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.590+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da003f030 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.591+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3da0003ad0 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.591+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac10e650 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.591+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0035070 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.591+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac1ab540 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.592+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac1032c0 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.594+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3da0042030 con 0x7f3dac101de0 2024-09-05T11:56:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.595+0000 7f3daa000640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 0x7f3d940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.595+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3da00efd30 con 0x7f3dac101de0 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.595+0000 7f3db0a00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 0x7f3d940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.596+0000 7f3db0a00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 0x7f3d940a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3dac19fff0 tx=0x7f3d9800b4a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.597+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3da00bc120 con 0x7f3dac101de0 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.783+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3dac105ed0 con 0x7f3dac101de0 2024-09-05T11:56:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.783+0000 7f3daa000640 1 -- 172.21.15.59:0/134356943 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3da00c0420 con 0x7f3dac101de0 2024-09-05T11:56:10.025 INFO:teuthology.orchestra.run.smithi059.stdout:107374182475 2024-09-05T11:56:10.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.785+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 msgr2=0x7f3d940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:10.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 0x7f3d940a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3dac19fff0 tx=0x7f3d9800b4a0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 msgr2=0x7f3dac1a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:10.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 0x7f3dac1a4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005940 tx=0x7f3da0004270 comp rx=0 tx=0).stop 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 shutdown_connections 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3d940a2c10 0x7f3d940a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3dac106f10 0x7f3dac10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dac102790 0x7f3dac1a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 --2- 172.21.15.59:0/134356943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dac101de0 0x7f3dac1a4770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 >> 172.21.15.59:0/134356943 conn(0x7f3dac0fd8c0 msgr2=0x7f3dac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 shutdown_connections 2024-09-05T11:56:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:09.786+0000 7f3db3e00640 1 -- 172.21.15.59:0/134356943 wait complete. 2024-09-05T11:56:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:09 smithi059 ceph-mon[30025]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:10.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:09 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/134356943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T11:56:10.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:09 smithi189 ceph-mon[34772]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:10.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:09 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/134356943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T11:56:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:09 smithi089 ceph-mon[34499]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:09 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/134356943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T11:56:10.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953537 got 137438953540 for osd.4 2024-09-05T11:56:10.975 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:11.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182475 for osd.3 2024-09-05T11:56:11.492 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:12.058 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:11 smithi059 ceph-mon[30025]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:12.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 -- 172.21.15.59:0/4056506978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c1049a0 msgr2=0x7fd61c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 --2- 172.21.15.59:0/4056506978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c1049a0 0x7fd61c1089f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd604009900 tx=0x7fd60402a410 comp rx=0 tx=0).stop 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 -- 172.21.15.59:0/4056506978 shutdown_connections 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 --2- 172.21.15.59:0/4056506978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c109120 0x7fd61c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 --2- 172.21.15.59:0/4056506978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c1049a0 0x7fd61c1089f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 --2- 172.21.15.59:0/4056506978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 -- 172.21.15.59:0/4056506978 >> 172.21.15.59:0/4056506978 conn(0x7fd61c0ffa90 msgr2=0x7fd61c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 -- 172.21.15.59:0/4056506978 shutdown_connections 2024-09-05T11:56:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.081+0000 7fd620e00640 1 -- 172.21.15.59:0/4056506978 wait complete. 2024-09-05T11:56:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 Processor -- start 2024-09-05T11:56:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 -- start start 2024-09-05T11:56:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c1049a0 0x7fd61c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 0x7fd61c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c114190 con 0x7fd61c109120 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c114010 con 0x7fd61c103ff0 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.082+0000 7fd620e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c114310 con 0x7fd61c1049a0 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd61a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 0x7fd61c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd61a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 0x7fd61c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35024/0 (socket says 172.21.15.59:35024) 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd61a800640 1 -- 172.21.15.59:0/1758875564 learned_addr learned my addr 172.21.15.59:0/1758875564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd611400640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c1049a0 0x7fd61c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 -- 172.21.15.59:0/1758875564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c1049a0 msgr2=0x7fd61c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c1049a0 0x7fd61c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 -- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 msgr2=0x7fd61c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:12.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 0x7fd61c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 -- 172.21.15.59:0/1758875564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c19f620 con 0x7fd61c103ff0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.083+0000 7fd619e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c19a670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d6d0 tx=0x7fd60c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.084+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd61c103ff0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.084+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c009820 con 0x7fd61c103ff0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.084+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c19f910 con 0x7fd61c103ff0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.084+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004d00 con 0x7fd61c103ff0 2024-09-05T11:56:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.084+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a71f0 con 0x7fd61c103ff0 2024-09-05T11:56:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.085+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c1043d0 con 0x7fd61c103ff0 2024-09-05T11:56:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.087+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd60c005430 con 0x7fd61c103ff0 2024-09-05T11:56:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.088+0000 7fd613400640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 0x7fd6000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.088+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd60c0ced20 con 0x7fd61c103ff0 2024-09-05T11:56:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.088+0000 7fd611400640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 0x7fd6000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.089+0000 7fd611400640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 0x7fd6000a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd604005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:12.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.092+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd60c09b110 con 0x7fd61c103ff0 2024-09-05T11:56:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.276+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd61c10c360 con 0x7fd61c103ff0 2024-09-05T11:56:12.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.277+0000 7fd613400640 1 -- 172.21.15.59:0/1758875564 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd60c09f410 con 0x7fd61c103ff0 2024-09-05T11:56:12.279 INFO:teuthology.orchestra.run.smithi059.stdout:274877906989 2024-09-05T11:56:12.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.279+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 msgr2=0x7fd6000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:12.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.279+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 0x7fd6000a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd604000c00 tx=0x7fd604005b50 comp rx=0 tx=0).stop 2024-09-05T11:56:12.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.279+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 msgr2=0x7fd61c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.279+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c19a670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d6d0 tx=0x7fd60c00db90 comp rx=0 tx=0).stop 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 shutdown_connections 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6000a2c10 0x7fd6000a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61c109120 0x7fd61c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61c1049a0 0x7fd61c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 --2- 172.21.15.59:0/1758875564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd61c103ff0 0x7fd61c19a670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 >> 172.21.15.59:0/1758875564 conn(0x7fd61c0ffa90 msgr2=0x7fd61c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 shutdown_connections 2024-09-05T11:56:12.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:12.280+0000 7fd620e00640 1 -- 172.21.15.59:0/1758875564 wait complete. 2024-09-05T11:56:12.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:11 smithi189 ceph-mon[34772]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:12.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:11 smithi089 ceph-mon[34499]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:12.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906986 got 274877906989 for osd.9 2024-09-05T11:56:12.943 DEBUG:teuthology.parallel:result is None 2024-09-05T11:56:12.943 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T11:56:12.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:56:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:12 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1758875564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T11:56:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:13.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:12 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1758875564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T11:56:13.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:12 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1758875564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T11:56:13.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 -- 172.21.15.59:0/979217998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4074eb0 msgr2=0x7f4df4112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:13.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 --2- 172.21.15.59:0/979217998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4074eb0 0x7f4df4112650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009900 tx=0x7f4de002a480 comp rx=0 tx=0).stop 2024-09-05T11:56:13.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 -- 172.21.15.59:0/979217998 shutdown_connections 2024-09-05T11:56:13.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 --2- 172.21.15.59:0/979217998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4112b90 0x7f4df4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:13.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 --2- 172.21.15.59:0/979217998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4074eb0 0x7f4df4112650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:13.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 --2- 172.21.15.59:0/979217998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074590 0x7f4df4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:13.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 -- 172.21.15.59:0/979217998 >> 172.21.15.59:0/979217998 conn(0x7f4df40fd8c0 msgr2=0x7f4df40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:13.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 -- 172.21.15.59:0/979217998 shutdown_connections 2024-09-05T11:56:13.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.879+0000 7f4dfac00640 1 -- 172.21.15.59:0/979217998 wait complete. 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.880+0000 7f4dfac00640 1 Processor -- start 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.880+0000 7f4dfac00640 1 -- start start 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 0x7f4df419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 0x7f4df419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4112b90 0x7f4df41a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4118560 con 0x7f4df4112b90 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4df41183e0 con 0x7f4df4074eb0 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4dfac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4df41186e0 con 0x7f4df4074590 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4df8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4112b90 0x7f4df41a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4df3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 0x7f4df419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4df3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 0x7f4df419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4df3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 0x7f4df419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-09-05T11:56:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.881+0000 7f4df3e00640 1 -- 172.21.15.59:0/787599859 learned_addr learned my addr 172.21.15.59:0/787599859 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 -- 172.21.15.59:0/787599859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 msgr2=0x7f4df419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 0x7f4df419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 -- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4112b90 msgr2=0x7f4df41a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4112b90 0x7f4df41a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 -- 172.21.15.59:0/787599859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df41a3b50 con 0x7f4df4074eb0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df3400640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 0x7f4df419f060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4de0005940 tx=0x7f4de0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.882+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de003f030 con 0x7f4df4074eb0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.883+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a3de0 con 0x7f4df4074eb0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.883+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0003ad0 con 0x7f4df4074eb0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.883+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0035070 con 0x7f4df4074eb0 2024-09-05T11:56:13.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.883+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41ab6d0 con 0x7f4df4074eb0 2024-09-05T11:56:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.884+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df4074970 con 0x7f4df4074eb0 2024-09-05T11:56:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.886+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4de0042030 con 0x7f4df4074eb0 2024-09-05T11:56:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.887+0000 7f4df0c00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 0x7f4dd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.887+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4de00efd80 con 0x7f4df4074eb0 2024-09-05T11:56:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.887+0000 7f4df3e00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 0x7f4dd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.888+0000 7f4df3e00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 0x7f4dd80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009c90 tx=0x7f4de4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:13.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:13.889+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4de00bc170 con 0x7f4df4074eb0 2024-09-05T11:56:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.069+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4df4062630 con 0x7f4dd80a2c10 2024-09-05T11:56:14.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:13 smithi059 ceph-mon[30025]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.079+0000 7f4df0c00640 1 -- 172.21.15.59:0/787599859 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340289 (secure 0 0 0) 0x7f4df4062630 con 0x7f4dd80a2c10 2024-09-05T11:56:14.085 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 msgr2=0x7f4dd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 0x7f4dd80a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009c90 tx=0x7f4de4005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 msgr2=0x7f4df419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 0x7f4df419f060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4de0005940 tx=0x7f4de0004270 comp rx=0 tx=0).stop 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 shutdown_connections 2024-09-05T11:56:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4dd80a2c10 0x7f4dd80a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df4112b90 0x7f4df41a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4df4074eb0 0x7f4df419f060 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 --2- 172.21.15.59:0/787599859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4df4074590 0x7f4df419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 >> 172.21.15.59:0/787599859 conn(0x7f4df40fd8c0 msgr2=0x7f4df40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 shutdown_connections 2024-09-05T11:56:14.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:14.084+0000 7f4dfac00640 1 -- 172.21.15.59:0/787599859 wait complete. 2024-09-05T11:56:14.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:13 smithi189 ceph-mon[34772]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:13 smithi089 ceph-mon[34499]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:14.862 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-05T11:56:12.876101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656531,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":257,"num_read_kb":241,"num_write":272,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375840,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565984,"statfs":{"total":1151940427776,"available":1151555567616,"internally_reserved":0,"allocated":14434304,"data_stored":8110763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748911+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:55:18.748911+0000","last_peered":"2024-09-05T11:55:18.748911+0000","last_clean":"2024-09-05T11:55:18.748911+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:55:18.748911+0000","last_undegraded":"2024-09-05T11:55:18.748911+0000","last_fullsized":"2024-09-05T11:55:18.748911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959651+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:55:18.959651+0000","last_peered":"2024-09-05T11:55:18.959651+0000","last_clean":"2024-09-05T11:55:18.959651+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:55:18.959651+0000","last_undegraded":"2024-09-05T11:55:18.959651+0000","last_fullsized":"2024-09-05T11:55:18.959651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960937+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:55:18.960937+0000","last_peered":"2024-09-05T11:55:18.960937+0000","last_clean":"2024-09-05T11:55:18.960937+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:55:18.960937+0000","last_undegraded":"2024-09-05T11:55:18.960937+0000","last_fullsized":"2024-09-05T11:55:18.960937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959828+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:55:18.959828+0000","last_peered":"2024-09-05T11:55:18.959828+0000","last_clean":"2024-09-05T11:55:18.959828+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:55:18.959828+0000","last_undegraded":"2024-09-05T11:55:18.959828+0000","last_fullsized":"2024-09-05T11:55:18.959828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829255+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:55:18.829255+0000","last_peered":"2024-09-05T11:55:18.829255+0000","last_clean":"2024-09-05T11:55:18.829255+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:55:18.829255+0000","last_undegraded":"2024-09-05T11:55:18.829255+0000","last_fullsized":"2024-09-05T11:55:18.829255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960868+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:55:18.960868+0000","last_peered":"2024-09-05T11:55:18.960868+0000","last_clean":"2024-09-05T11:55:18.960868+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:55:18.960868+0000","last_undegraded":"2024-09-05T11:55:18.960868+0000","last_fullsized":"2024-09-05T11:55:18.960868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750148+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:55:18.750148+0000","last_peered":"2024-09-05T11:55:18.750148+0000","last_clean":"2024-09-05T11:55:18.750148+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:55:18.750148+0000","last_undegraded":"2024-09-05T11:55:18.750148+0000","last_fullsized":"2024-09-05T11:55:18.750148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061069+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:55:19.061069+0000","last_peered":"2024-09-05T11:55:19.061069+0000","last_clean":"2024-09-05T11:55:19.061069+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:55:19.061069+0000","last_undegraded":"2024-09-05T11:55:19.061069+0000","last_fullsized":"2024-09-05T11:55:19.061069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746730+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:55:18.746730+0000","last_peered":"2024-09-05T11:55:18.746730+0000","last_clean":"2024-09-05T11:55:18.746730+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:55:18.746730+0000","last_undegraded":"2024-09-05T11:55:18.746730+0000","last_fullsized":"2024-09-05T11:55:18.746730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750651+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:55:18.750651+0000","last_peered":"2024-09-05T11:55:18.750651+0000","last_clean":"2024-09-05T11:55:18.750651+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:55:18.750651+0000","last_undegraded":"2024-09-05T11:55:18.750651+0000","last_fullsized":"2024-09-05T11:55:18.750651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749383+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:55:18.749383+0000","last_peered":"2024-09-05T11:55:18.749383+0000","last_clean":"2024-09-05T11:55:18.749383+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:55:18.749383+0000","last_undegraded":"2024-09-05T11:55:18.749383+0000","last_fullsized":"2024-09-05T11:55:18.749383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746451+0000","last_change":"2024-09-05T11:54:38.607159+0000","last_active":"2024-09-05T11:55:18.746451+0000","last_peered":"2024-09-05T11:55:18.746451+0000","last_clean":"2024-09-05T11:55:18.746451+0000","last_became_active":"2024-09-05T11:54:38.606569+0000","last_became_peered":"2024-09-05T11:54:38.606569+0000","last_unstale":"2024-09-05T11:55:18.746451+0000","last_undegraded":"2024-09-05T11:55:18.746451+0000","last_fullsized":"2024-09-05T11:55:18.746451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:28:07.583731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749879+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:55:18.749879+0000","last_peered":"2024-09-05T11:55:18.749879+0000","last_clean":"2024-09-05T11:55:18.749879+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:55:18.749879+0000","last_undegraded":"2024-09-05T11:55:18.749879+0000","last_fullsized":"2024-09-05T11:55:18.749879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749076+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:55:18.749076+0000","last_peered":"2024-09-05T11:55:18.749076+0000","last_clean":"2024-09-05T11:55:18.749076+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:55:18.749076+0000","last_undegraded":"2024-09-05T11:55:18.749076+0000","last_fullsized":"2024-09-05T11:55:18.749076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746802+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:55:18.746802+0000","last_peered":"2024-09-05T11:55:18.746802+0000","last_clean":"2024-09-05T11:55:18.746802+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:55:18.746802+0000","last_undegraded":"2024-09-05T11:55:18.746802+0000","last_fullsized":"2024-09-05T11:55:18.746802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959081+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:55:18.959081+0000","last_peered":"2024-09-05T11:55:18.959081+0000","last_clean":"2024-09-05T11:55:18.959081+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:55:18.959081+0000","last_undegraded":"2024-09-05T11:55:18.959081+0000","last_fullsized":"2024-09-05T11:55:18.959081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744681+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:55:18.744681+0000","last_peered":"2024-09-05T11:55:18.744681+0000","last_clean":"2024-09-05T11:55:18.744681+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:55:18.744681+0000","last_undegraded":"2024-09-05T11:55:18.744681+0000","last_fullsized":"2024-09-05T11:55:18.744681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753048+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:55:18.753048+0000","last_peered":"2024-09-05T11:55:18.753048+0000","last_clean":"2024-09-05T11:55:18.753048+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:55:18.753048+0000","last_undegraded":"2024-09-05T11:55:18.753048+0000","last_fullsized":"2024-09-05T11:55:18.753048+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748420+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:55:18.748420+0000","last_peered":"2024-09-05T11:55:18.748420+0000","last_clean":"2024-09-05T11:55:18.748420+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:55:18.748420+0000","last_undegraded":"2024-09-05T11:55:18.748420+0000","last_fullsized":"2024-09-05T11:55:18.748420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959409+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:55:18.959409+0000","last_peered":"2024-09-05T11:55:18.959409+0000","last_clean":"2024-09-05T11:55:18.959409+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:55:18.959409+0000","last_undegraded":"2024-09-05T11:55:18.959409+0000","last_fullsized":"2024-09-05T11:55:18.959409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748594+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:55:18.748594+0000","last_peered":"2024-09-05T11:55:18.748594+0000","last_clean":"2024-09-05T11:55:18.748594+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:55:18.748594+0000","last_undegraded":"2024-09-05T11:55:18.748594+0000","last_fullsized":"2024-09-05T11:55:18.748594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751886+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:55:18.751886+0000","last_peered":"2024-09-05T11:55:18.751886+0000","last_clean":"2024-09-05T11:55:18.751886+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:55:18.751886+0000","last_undegraded":"2024-09-05T11:55:18.751886+0000","last_fullsized":"2024-09-05T11:55:18.751886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060880+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:55:19.060880+0000","last_peered":"2024-09-05T11:55:19.060880+0000","last_clean":"2024-09-05T11:55:19.060880+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:55:19.060880+0000","last_undegraded":"2024-09-05T11:55:19.060880+0000","last_fullsized":"2024-09-05T11:55:19.060880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746843+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:55:18.746843+0000","last_peered":"2024-09-05T11:55:18.746843+0000","last_clean":"2024-09-05T11:55:18.746843+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:55:18.746843+0000","last_undegraded":"2024-09-05T11:55:18.746843+0000","last_fullsized":"2024-09-05T11:55:18.746843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829155+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:55:18.829155+0000","last_peered":"2024-09-05T11:55:18.829155+0000","last_clean":"2024-09-05T11:55:18.829155+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:55:18.829155+0000","last_undegraded":"2024-09-05T11:55:18.829155+0000","last_fullsized":"2024-09-05T11:55:18.829155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747880+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:55:18.747880+0000","last_peered":"2024-09-05T11:55:18.747880+0000","last_clean":"2024-09-05T11:55:18.747880+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:55:18.747880+0000","last_undegraded":"2024-09-05T11:55:18.747880+0000","last_fullsized":"2024-09-05T11:55:18.747880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744630+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:55:18.744630+0000","last_peered":"2024-09-05T11:55:18.744630+0000","last_clean":"2024-09-05T11:55:18.744630+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:55:18.744630+0000","last_undegraded":"2024-09-05T11:55:18.744630+0000","last_fullsized":"2024-09-05T11:55:18.744630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959726+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:55:18.959726+0000","last_peered":"2024-09-05T11:55:18.959726+0000","last_clean":"2024-09-05T11:55:18.959726+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:55:18.959726+0000","last_undegraded":"2024-09-05T11:55:18.959726+0000","last_fullsized":"2024-09-05T11:55:18.959726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:29.562399+0000","last_change":"2024-09-05T11:54:29.103741+0000","last_active":"2024-09-05T11:55:29.562399+0000","last_peered":"2024-09-05T11:55:29.562399+0000","last_clean":"2024-09-05T11:55:29.562399+0000","last_became_active":"2024-09-05T11:54:29.103285+0000","last_became_peered":"2024-09-05T11:54:29.103285+0000","last_unstale":"2024-09-05T11:55:29.562399+0000","last_undegraded":"2024-09-05T11:55:29.562399+0000","last_fullsized":"2024-09-05T11:55:29.562399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:01:16.713162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.958714+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:55:18.958714+0000","last_peered":"2024-09-05T11:55:18.958714+0000","last_clean":"2024-09-05T11:55:18.958714+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:55:18.958714+0000","last_undegraded":"2024-09-05T11:55:18.958714+0000","last_fullsized":"2024-09-05T11:55:18.958714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744903+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:55:18.744903+0000","last_peered":"2024-09-05T11:55:18.744903+0000","last_clean":"2024-09-05T11:55:18.744903+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:55:18.744903+0000","last_undegraded":"2024-09-05T11:55:18.744903+0000","last_fullsized":"2024-09-05T11:55:18.744903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750269+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:55:18.750269+0000","last_peered":"2024-09-05T11:55:18.750269+0000","last_clean":"2024-09-05T11:55:18.750269+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:55:18.750269+0000","last_undegraded":"2024-09-05T11:55:18.750269+0000","last_fullsized":"2024-09-05T11:55:18.750269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748452+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:55:18.748452+0000","last_peered":"2024-09-05T11:55:18.748452+0000","last_clean":"2024-09-05T11:55:18.748452+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:55:18.748452+0000","last_undegraded":"2024-09-05T11:55:18.748452+0000","last_fullsized":"2024-09-05T11:55:18.748452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745443+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:55:18.745443+0000","last_peered":"2024-09-05T11:55:18.745443+0000","last_clean":"2024-09-05T11:55:18.745443+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:55:18.745443+0000","last_undegraded":"2024-09-05T11:55:18.745443+0000","last_fullsized":"2024-09-05T11:55:18.745443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748502+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:55:18.748502+0000","last_peered":"2024-09-05T11:55:18.748502+0000","last_clean":"2024-09-05T11:55:18.748502+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:55:18.748502+0000","last_undegraded":"2024-09-05T11:55:18.748502+0000","last_fullsized":"2024-09-05T11:55:18.748502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750324+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:55:18.750324+0000","last_peered":"2024-09-05T11:55:18.750324+0000","last_clean":"2024-09-05T11:55:18.750324+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:55:18.750324+0000","last_undegraded":"2024-09-05T11:55:18.750324+0000","last_fullsized":"2024-09-05T11:55:18.750324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745255+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:55:18.745255+0000","last_peered":"2024-09-05T11:55:18.745255+0000","last_clean":"2024-09-05T11:55:18.745255+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:55:18.745255+0000","last_undegraded":"2024-09-05T11:55:18.745255+0000","last_fullsized":"2024-09-05T11:55:18.745255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749954+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:55:18.749954+0000","last_peered":"2024-09-05T11:55:18.749954+0000","last_clean":"2024-09-05T11:55:18.749954+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:55:18.749954+0000","last_undegraded":"2024-09-05T11:55:18.749954+0000","last_fullsized":"2024-09-05T11:55:18.749954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744674+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:55:18.744674+0000","last_peered":"2024-09-05T11:55:18.744674+0000","last_clean":"2024-09-05T11:55:18.744674+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:55:18.744674+0000","last_undegraded":"2024-09-05T11:55:18.744674+0000","last_fullsized":"2024-09-05T11:55:18.744674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751392+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:55:18.751392+0000","last_peered":"2024-09-05T11:55:18.751392+0000","last_clean":"2024-09-05T11:55:18.751392+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:55:18.751392+0000","last_undegraded":"2024-09-05T11:55:18.751392+0000","last_fullsized":"2024-09-05T11:55:18.751392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744437+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:55:18.744437+0000","last_peered":"2024-09-05T11:55:18.744437+0000","last_clean":"2024-09-05T11:55:18.744437+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:55:18.744437+0000","last_undegraded":"2024-09-05T11:55:18.744437+0000","last_fullsized":"2024-09-05T11:55:18.744437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744739+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:55:18.744739+0000","last_peered":"2024-09-05T11:55:18.744739+0000","last_clean":"2024-09-05T11:55:18.744739+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:55:18.744739+0000","last_undegraded":"2024-09-05T11:55:18.744739+0000","last_fullsized":"2024-09-05T11:55:18.744739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744301+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:55:18.744301+0000","last_peered":"2024-09-05T11:55:18.744301+0000","last_clean":"2024-09-05T11:55:18.744301+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:55:18.744301+0000","last_undegraded":"2024-09-05T11:55:18.744301+0000","last_fullsized":"2024-09-05T11:55:18.744301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749802+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:55:18.749802+0000","last_peered":"2024-09-05T11:55:18.749802+0000","last_clean":"2024-09-05T11:55:18.749802+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:55:18.749802+0000","last_undegraded":"2024-09-05T11:55:18.749802+0000","last_fullsized":"2024-09-05T11:55:18.749802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749669+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:55:18.749669+0000","last_peered":"2024-09-05T11:55:18.749669+0000","last_clean":"2024-09-05T11:55:18.749669+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:55:18.749669+0000","last_undegraded":"2024-09-05T11:55:18.749669+0000","last_fullsized":"2024-09-05T11:55:18.749669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959105+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:55:18.959105+0000","last_peered":"2024-09-05T11:55:18.959105+0000","last_clean":"2024-09-05T11:55:18.959105+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:55:18.959105+0000","last_undegraded":"2024-09-05T11:55:18.959105+0000","last_fullsized":"2024-09-05T11:55:18.959105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747099+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:55:18.747099+0000","last_peered":"2024-09-05T11:55:18.747099+0000","last_clean":"2024-09-05T11:55:18.747099+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:55:18.747099+0000","last_undegraded":"2024-09-05T11:55:18.747099+0000","last_fullsized":"2024-09-05T11:55:18.747099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750002+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:55:18.750002+0000","last_peered":"2024-09-05T11:55:18.750002+0000","last_clean":"2024-09-05T11:55:18.750002+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:55:18.750002+0000","last_undegraded":"2024-09-05T11:55:18.750002+0000","last_fullsized":"2024-09-05T11:55:18.750002+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750328+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:55:18.750328+0000","last_peered":"2024-09-05T11:55:18.750328+0000","last_clean":"2024-09-05T11:55:18.750328+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:55:18.750328+0000","last_undegraded":"2024-09-05T11:55:18.750328+0000","last_fullsized":"2024-09-05T11:55:18.750328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750439+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:55:18.750439+0000","last_peered":"2024-09-05T11:55:18.750439+0000","last_clean":"2024-09-05T11:55:18.750439+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:55:18.750439+0000","last_undegraded":"2024-09-05T11:55:18.750439+0000","last_fullsized":"2024-09-05T11:55:18.750439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745209+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:55:18.745209+0000","last_peered":"2024-09-05T11:55:18.745209+0000","last_clean":"2024-09-05T11:55:18.745209+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:55:18.745209+0000","last_undegraded":"2024-09-05T11:55:18.745209+0000","last_fullsized":"2024-09-05T11:55:18.745209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060892+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:55:19.060892+0000","last_peered":"2024-09-05T11:55:19.060892+0000","last_clean":"2024-09-05T11:55:19.060892+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:55:19.060892+0000","last_undegraded":"2024-09-05T11:55:19.060892+0000","last_fullsized":"2024-09-05T11:55:19.060892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750265+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:55:18.750265+0000","last_peered":"2024-09-05T11:55:18.750265+0000","last_clean":"2024-09-05T11:55:18.750265+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:55:18.750265+0000","last_undegraded":"2024-09-05T11:55:18.750265+0000","last_fullsized":"2024-09-05T11:55:18.750265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750840+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:55:18.750840+0000","last_peered":"2024-09-05T11:55:18.750840+0000","last_clean":"2024-09-05T11:55:18.750840+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:55:18.750840+0000","last_undegraded":"2024-09-05T11:55:18.750840+0000","last_fullsized":"2024-09-05T11:55:18.750840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828776+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:55:18.828776+0000","last_peered":"2024-09-05T11:55:18.828776+0000","last_clean":"2024-09-05T11:55:18.828776+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:55:18.828776+0000","last_undegraded":"2024-09-05T11:55:18.828776+0000","last_fullsized":"2024-09-05T11:55:18.828776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750453+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:55:18.750453+0000","last_peered":"2024-09-05T11:55:18.750453+0000","last_clean":"2024-09-05T11:55:18.750453+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:55:18.750453+0000","last_undegraded":"2024-09-05T11:55:18.750453+0000","last_fullsized":"2024-09-05T11:55:18.750453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745094+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:55:18.745094+0000","last_peered":"2024-09-05T11:55:18.745094+0000","last_clean":"2024-09-05T11:55:18.745094+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:55:18.745094+0000","last_undegraded":"2024-09-05T11:55:18.745094+0000","last_fullsized":"2024-09-05T11:55:18.745094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749801+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:55:18.749801+0000","last_peered":"2024-09-05T11:55:18.749801+0000","last_clean":"2024-09-05T11:55:18.749801+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:55:18.749801+0000","last_undegraded":"2024-09-05T11:55:18.749801+0000","last_fullsized":"2024-09-05T11:55:18.749801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749962+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:55:18.749962+0000","last_peered":"2024-09-05T11:55:18.749962+0000","last_clean":"2024-09-05T11:55:18.749962+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:55:18.749962+0000","last_undegraded":"2024-09-05T11:55:18.749962+0000","last_fullsized":"2024-09-05T11:55:18.749962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749212+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:55:18.749212+0000","last_peered":"2024-09-05T11:55:18.749212+0000","last_clean":"2024-09-05T11:55:18.749212+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:55:18.749212+0000","last_undegraded":"2024-09-05T11:55:18.749212+0000","last_fullsized":"2024-09-05T11:55:18.749212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748596+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:55:18.748596+0000","last_peered":"2024-09-05T11:55:18.748596+0000","last_clean":"2024-09-05T11:55:18.748596+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:55:18.748596+0000","last_undegraded":"2024-09-05T11:55:18.748596+0000","last_fullsized":"2024-09-05T11:55:18.748596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749735+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:55:18.749735+0000","last_peered":"2024-09-05T11:55:18.749735+0000","last_clean":"2024-09-05T11:55:18.749735+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:55:18.749735+0000","last_undegraded":"2024-09-05T11:55:18.749735+0000","last_fullsized":"2024-09-05T11:55:18.749735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751378+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:55:18.751378+0000","last_peered":"2024-09-05T11:55:18.751378+0000","last_clean":"2024-09-05T11:55:18.751378+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:55:18.751378+0000","last_undegraded":"2024-09-05T11:55:18.751378+0000","last_fullsized":"2024-09-05T11:55:18.751378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744997+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:55:18.744997+0000","last_peered":"2024-09-05T11:55:18.744997+0000","last_clean":"2024-09-05T11:55:18.744997+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:55:18.744997+0000","last_undegraded":"2024-09-05T11:55:18.744997+0000","last_fullsized":"2024-09-05T11:55:18.744997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749638+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:55:18.749638+0000","last_peered":"2024-09-05T11:55:18.749638+0000","last_clean":"2024-09-05T11:55:18.749638+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:55:18.749638+0000","last_undegraded":"2024-09-05T11:55:18.749638+0000","last_fullsized":"2024-09-05T11:55:18.749638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747345+0000","last_change":"2024-09-05T11:54:29.102616+0000","last_active":"2024-09-05T11:55:18.747345+0000","last_peered":"2024-09-05T11:55:18.747345+0000","last_clean":"2024-09-05T11:55:18.747345+0000","last_became_active":"2024-09-05T11:54:29.102126+0000","last_became_peered":"2024-09-05T11:54:29.102126+0000","last_unstale":"2024-09-05T11:55:18.747345+0000","last_undegraded":"2024-09-05T11:55:18.747345+0000","last_fullsized":"2024-09-05T11:55:18.747345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:09:28.848395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748687+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:55:18.748687+0000","last_peered":"2024-09-05T11:55:18.748687+0000","last_clean":"2024-09-05T11:55:18.748687+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:55:18.748687+0000","last_undegraded":"2024-09-05T11:55:18.748687+0000","last_fullsized":"2024-09-05T11:55:18.748687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750129+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:55:18.750129+0000","last_peered":"2024-09-05T11:55:18.750129+0000","last_clean":"2024-09-05T11:55:18.750129+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:55:18.750129+0000","last_undegraded":"2024-09-05T11:55:18.750129+0000","last_fullsized":"2024-09-05T11:55:18.750129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751077+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:55:18.751077+0000","last_peered":"2024-09-05T11:55:18.751077+0000","last_clean":"2024-09-05T11:55:18.751077+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:55:18.751077+0000","last_undegraded":"2024-09-05T11:55:18.751077+0000","last_fullsized":"2024-09-05T11:55:18.751077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748683+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:55:18.748683+0000","last_peered":"2024-09-05T11:55:18.748683+0000","last_clean":"2024-09-05T11:55:18.748683+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:55:18.748683+0000","last_undegraded":"2024-09-05T11:55:18.748683+0000","last_fullsized":"2024-09-05T11:55:18.748683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744841+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:55:18.744841+0000","last_peered":"2024-09-05T11:55:18.744841+0000","last_clean":"2024-09-05T11:55:18.744841+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:55:18.744841+0000","last_undegraded":"2024-09-05T11:55:18.744841+0000","last_fullsized":"2024-09-05T11:55:18.744841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744915+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:55:18.744915+0000","last_peered":"2024-09-05T11:55:18.744915+0000","last_clean":"2024-09-05T11:55:18.744915+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:55:18.744915+0000","last_undegraded":"2024-09-05T11:55:18.744915+0000","last_fullsized":"2024-09-05T11:55:18.744915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750719+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:55:18.750719+0000","last_peered":"2024-09-05T11:55:18.750719+0000","last_clean":"2024-09-05T11:55:18.750719+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:55:18.750719+0000","last_undegraded":"2024-09-05T11:55:18.750719+0000","last_fullsized":"2024-09-05T11:55:18.750719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749744+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:55:18.749744+0000","last_peered":"2024-09-05T11:55:18.749744+0000","last_clean":"2024-09-05T11:55:18.749744+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:55:18.749744+0000","last_undegraded":"2024-09-05T11:55:18.749744+0000","last_fullsized":"2024-09-05T11:55:18.749744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:56:09.814251+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:09.814251+0000","last_peered":"2024-09-05T11:56:09.814251+0000","last_clean":"2024-09-05T11:56:09.814251+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:09.814251+0000","last_undegraded":"2024-09-05T11:56:09.814251+0000","last_fullsized":"2024-09-05T11:56:09.814251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65131,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":126,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747427+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:55:18.747427+0000","last_peered":"2024-09-05T11:55:18.747427+0000","last_clean":"2024-09-05T11:55:18.747427+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:55:18.747427+0000","last_undegraded":"2024-09-05T11:55:18.747427+0000","last_fullsized":"2024-09-05T11:55:18.747427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061114+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:55:19.061114+0000","last_peered":"2024-09-05T11:55:19.061114+0000","last_clean":"2024-09-05T11:55:19.061114+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:55:19.061114+0000","last_undegraded":"2024-09-05T11:55:19.061114+0000","last_fullsized":"2024-09-05T11:55:19.061114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750985+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:55:18.750985+0000","last_peered":"2024-09-05T11:55:18.750985+0000","last_clean":"2024-09-05T11:55:18.750985+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:55:18.750985+0000","last_undegraded":"2024-09-05T11:55:18.750985+0000","last_fullsized":"2024-09-05T11:55:18.750985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746602+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:55:18.746602+0000","last_peered":"2024-09-05T11:55:18.746602+0000","last_clean":"2024-09-05T11:55:18.746602+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:55:18.746602+0000","last_undegraded":"2024-09-05T11:55:18.746602+0000","last_fullsized":"2024-09-05T11:55:18.746602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829307+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:55:18.829307+0000","last_peered":"2024-09-05T11:55:18.829307+0000","last_clean":"2024-09-05T11:55:18.829307+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:55:18.829307+0000","last_undegraded":"2024-09-05T11:55:18.829307+0000","last_fullsized":"2024-09-05T11:55:18.829307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750531+0000","last_change":"2024-09-05T11:54:29.106596+0000","last_active":"2024-09-05T11:55:18.750531+0000","last_peered":"2024-09-05T11:55:18.750531+0000","last_clean":"2024-09-05T11:55:18.750531+0000","last_became_active":"2024-09-05T11:54:29.106213+0000","last_became_peered":"2024-09-05T11:54:29.106213+0000","last_unstale":"2024-09-05T11:55:18.750531+0000","last_undegraded":"2024-09-05T11:55:18.750531+0000","last_fullsized":"2024-09-05T11:55:18.750531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:24:44.592047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747050+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:55:18.747050+0000","last_peered":"2024-09-05T11:55:18.747050+0000","last_clean":"2024-09-05T11:55:18.747050+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:55:18.747050+0000","last_undegraded":"2024-09-05T11:55:18.747050+0000","last_fullsized":"2024-09-05T11:55:18.747050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061360+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:55:19.061360+0000","last_peered":"2024-09-05T11:55:19.061360+0000","last_clean":"2024-09-05T11:55:19.061360+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:55:19.061360+0000","last_undegraded":"2024-09-05T11:55:19.061360+0000","last_fullsized":"2024-09-05T11:55:19.061360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745488+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:55:18.745488+0000","last_peered":"2024-09-05T11:55:18.745488+0000","last_clean":"2024-09-05T11:55:18.745488+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:55:18.745488+0000","last_undegraded":"2024-09-05T11:55:18.745488+0000","last_fullsized":"2024-09-05T11:55:18.745488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959078+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:55:18.959078+0000","last_peered":"2024-09-05T11:55:18.959078+0000","last_clean":"2024-09-05T11:55:18.959078+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:55:18.959078+0000","last_undegraded":"2024-09-05T11:55:18.959078+0000","last_fullsized":"2024-09-05T11:55:18.959078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829040+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:55:18.829040+0000","last_peered":"2024-09-05T11:55:18.829040+0000","last_clean":"2024-09-05T11:55:18.829040+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:55:18.829040+0000","last_undegraded":"2024-09-05T11:55:18.829040+0000","last_fullsized":"2024-09-05T11:55:18.829040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747393+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:55:18.747393+0000","last_peered":"2024-09-05T11:55:18.747393+0000","last_clean":"2024-09-05T11:55:18.747393+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:55:18.747393+0000","last_undegraded":"2024-09-05T11:55:18.747393+0000","last_fullsized":"2024-09-05T11:55:18.747393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751542+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:55:18.751542+0000","last_peered":"2024-09-05T11:55:18.751542+0000","last_clean":"2024-09-05T11:55:18.751542+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:55:18.751542+0000","last_undegraded":"2024-09-05T11:55:18.751542+0000","last_fullsized":"2024-09-05T11:55:18.751542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749841+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:55:18.749841+0000","last_peered":"2024-09-05T11:55:18.749841+0000","last_clean":"2024-09-05T11:55:18.749841+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:55:18.749841+0000","last_undegraded":"2024-09-05T11:55:18.749841+0000","last_fullsized":"2024-09-05T11:55:18.749841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959536+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:55:18.959536+0000","last_peered":"2024-09-05T11:55:18.959536+0000","last_clean":"2024-09-05T11:55:18.959536+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:55:18.959536+0000","last_undegraded":"2024-09-05T11:55:18.959536+0000","last_fullsized":"2024-09-05T11:55:18.959536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744321+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:55:18.744321+0000","last_peered":"2024-09-05T11:55:18.744321+0000","last_clean":"2024-09-05T11:55:18.744321+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:55:18.744321+0000","last_undegraded":"2024-09-05T11:55:18.744321+0000","last_fullsized":"2024-09-05T11:55:18.744321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745069+0000","last_change":"2024-09-05T11:54:29.101408+0000","last_active":"2024-09-05T11:55:18.745069+0000","last_peered":"2024-09-05T11:55:18.745069+0000","last_clean":"2024-09-05T11:55:18.745069+0000","last_became_active":"2024-09-05T11:54:29.101052+0000","last_became_peered":"2024-09-05T11:54:29.101052+0000","last_unstale":"2024-09-05T11:55:18.745069+0000","last_undegraded":"2024-09-05T11:55:18.745069+0000","last_fullsized":"2024-09-05T11:55:18.745069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:40.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960761+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:55:18.960761+0000","last_peered":"2024-09-05T11:55:18.960761+0000","last_clean":"2024-09-05T11:55:18.960761+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:55:18.960761+0000","last_undegraded":"2024-09-05T11:55:18.960761+0000","last_fullsized":"2024-09-05T11:55:18.960761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744820+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:55:18.744820+0000","last_peered":"2024-09-05T11:55:18.744820+0000","last_clean":"2024-09-05T11:55:18.744820+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:55:18.744820+0000","last_undegraded":"2024-09-05T11:55:18.744820+0000","last_fullsized":"2024-09-05T11:55:18.744820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750928+0000","last_change":"2024-09-05T11:54:29.102371+0000","last_active":"2024-09-05T11:55:18.750928+0000","last_peered":"2024-09-05T11:55:18.750928+0000","last_clean":"2024-09-05T11:55:18.750928+0000","last_became_active":"2024-09-05T11:54:29.102066+0000","last_became_peered":"2024-09-05T11:54:29.102066+0000","last_unstale":"2024-09-05T11:55:18.750928+0000","last_undegraded":"2024-09-05T11:55:18.750928+0000","last_fullsized":"2024-09-05T11:55:18.750928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:55.008088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754810+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:55:18.754810+0000","last_peered":"2024-09-05T11:55:18.754810+0000","last_clean":"2024-09-05T11:55:18.754810+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:55:18.754810+0000","last_undegraded":"2024-09-05T11:55:18.754810+0000","last_fullsized":"2024-09-05T11:55:18.754810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744917+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:55:18.744917+0000","last_peered":"2024-09-05T11:55:18.744917+0000","last_clean":"2024-09-05T11:55:18.744917+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:55:18.744917+0000","last_undegraded":"2024-09-05T11:55:18.744917+0000","last_fullsized":"2024-09-05T11:55:18.744917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748298+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:55:18.748298+0000","last_peered":"2024-09-05T11:55:18.748298+0000","last_clean":"2024-09-05T11:55:18.748298+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:55:18.748298+0000","last_undegraded":"2024-09-05T11:55:18.748298+0000","last_fullsized":"2024-09-05T11:55:18.748298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745098+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:55:18.745098+0000","last_peered":"2024-09-05T11:55:18.745098+0000","last_clean":"2024-09-05T11:55:18.745098+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:55:18.745098+0000","last_undegraded":"2024-09-05T11:55:18.745098+0000","last_fullsized":"2024-09-05T11:55:18.745098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744606+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:55:18.744606+0000","last_peered":"2024-09-05T11:55:18.744606+0000","last_clean":"2024-09-05T11:55:18.744606+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:55:18.744606+0000","last_undegraded":"2024-09-05T11:55:18.744606+0000","last_fullsized":"2024-09-05T11:55:18.744606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828789+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:55:18.828789+0000","last_peered":"2024-09-05T11:55:18.828789+0000","last_clean":"2024-09-05T11:55:18.828789+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:55:18.828789+0000","last_undegraded":"2024-09-05T11:55:18.828789+0000","last_fullsized":"2024-09-05T11:55:18.828789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744825+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:55:18.744825+0000","last_peered":"2024-09-05T11:55:18.744825+0000","last_clean":"2024-09-05T11:55:18.744825+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:55:18.744825+0000","last_undegraded":"2024-09-05T11:55:18.744825+0000","last_fullsized":"2024-09-05T11:55:18.744825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749366+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:55:18.749366+0000","last_peered":"2024-09-05T11:55:18.749366+0000","last_clean":"2024-09-05T11:55:18.749366+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:55:18.749366+0000","last_undegraded":"2024-09-05T11:55:18.749366+0000","last_fullsized":"2024-09-05T11:55:18.749366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745046+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:55:18.745046+0000","last_peered":"2024-09-05T11:55:18.745046+0000","last_clean":"2024-09-05T11:55:18.745046+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:55:18.745046+0000","last_undegraded":"2024-09-05T11:55:18.745046+0000","last_fullsized":"2024-09-05T11:55:18.745046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959692+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:55:18.959692+0000","last_peered":"2024-09-05T11:55:18.959692+0000","last_clean":"2024-09-05T11:55:18.959692+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:55:18.959692+0000","last_undegraded":"2024-09-05T11:55:18.959692+0000","last_fullsized":"2024-09-05T11:55:18.959692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751303+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:55:18.751303+0000","last_peered":"2024-09-05T11:55:18.751303+0000","last_clean":"2024-09-05T11:55:18.751303+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:55:18.751303+0000","last_undegraded":"2024-09-05T11:55:18.751303+0000","last_fullsized":"2024-09-05T11:55:18.751303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959252+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:55:18.959252+0000","last_peered":"2024-09-05T11:55:18.959252+0000","last_clean":"2024-09-05T11:55:18.959252+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:55:18.959252+0000","last_undegraded":"2024-09-05T11:55:18.959252+0000","last_fullsized":"2024-09-05T11:55:18.959252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061014+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:55:19.061014+0000","last_peered":"2024-09-05T11:55:19.061014+0000","last_clean":"2024-09-05T11:55:19.061014+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:55:19.061014+0000","last_undegraded":"2024-09-05T11:55:19.061014+0000","last_fullsized":"2024-09-05T11:55:19.061014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744715+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:55:18.744715+0000","last_peered":"2024-09-05T11:55:18.744715+0000","last_clean":"2024-09-05T11:55:18.744715+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:55:18.744715+0000","last_undegraded":"2024-09-05T11:55:18.744715+0000","last_fullsized":"2024-09-05T11:55:18.744715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754299+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:55:18.754299+0000","last_peered":"2024-09-05T11:55:18.754299+0000","last_clean":"2024-09-05T11:55:18.754299+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:55:18.754299+0000","last_undegraded":"2024-09-05T11:55:18.754299+0000","last_fullsized":"2024-09-05T11:55:18.754299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745216+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:55:18.745216+0000","last_peered":"2024-09-05T11:55:18.745216+0000","last_clean":"2024-09-05T11:55:18.745216+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:55:18.745216+0000","last_undegraded":"2024-09-05T11:55:18.745216+0000","last_fullsized":"2024-09-05T11:55:18.745216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748542+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:55:18.748542+0000","last_peered":"2024-09-05T11:55:18.748542+0000","last_clean":"2024-09-05T11:55:18.748542+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:55:18.748542+0000","last_undegraded":"2024-09-05T11:55:18.748542+0000","last_fullsized":"2024-09-05T11:55:18.748542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750435+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:55:18.750435+0000","last_peered":"2024-09-05T11:55:18.750435+0000","last_clean":"2024-09-05T11:55:18.750435+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:55:18.750435+0000","last_undegraded":"2024-09-05T11:55:18.750435+0000","last_fullsized":"2024-09-05T11:55:18.750435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060771+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:55:19.060771+0000","last_peered":"2024-09-05T11:55:19.060771+0000","last_clean":"2024-09-05T11:55:19.060771+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:55:19.060771+0000","last_undegraded":"2024-09-05T11:55:19.060771+0000","last_fullsized":"2024-09-05T11:55:19.060771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828621+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:55:18.828621+0000","last_peered":"2024-09-05T11:55:18.828621+0000","last_clean":"2024-09-05T11:55:18.828621+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:55:18.828621+0000","last_undegraded":"2024-09-05T11:55:18.828621+0000","last_fullsized":"2024-09-05T11:55:18.828621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828531+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:55:18.828531+0000","last_peered":"2024-09-05T11:55:18.828531+0000","last_clean":"2024-09-05T11:55:18.828531+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:55:18.828531+0000","last_undegraded":"2024-09-05T11:55:18.828531+0000","last_fullsized":"2024-09-05T11:55:18.828531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744676+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:55:18.744676+0000","last_peered":"2024-09-05T11:55:18.744676+0000","last_clean":"2024-09-05T11:55:18.744676+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:55:18.744676+0000","last_undegraded":"2024-09-05T11:55:18.744676+0000","last_fullsized":"2024-09-05T11:55:18.744676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747214+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:55:18.747214+0000","last_peered":"2024-09-05T11:55:18.747214+0000","last_clean":"2024-09-05T11:55:18.747214+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:55:18.747214+0000","last_undegraded":"2024-09-05T11:55:18.747214+0000","last_fullsized":"2024-09-05T11:55:18.747214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750519+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:55:18.750519+0000","last_peered":"2024-09-05T11:55:18.750519+0000","last_clean":"2024-09-05T11:55:18.750519+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:55:18.750519+0000","last_undegraded":"2024-09-05T11:55:18.750519+0000","last_fullsized":"2024-09-05T11:55:18.750519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745888+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:55:18.745888+0000","last_peered":"2024-09-05T11:55:18.745888+0000","last_clean":"2024-09-05T11:55:18.745888+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:55:18.745888+0000","last_undegraded":"2024-09-05T11:55:18.745888+0000","last_fullsized":"2024-09-05T11:55:18.745888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061418+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:55:19.061418+0000","last_peered":"2024-09-05T11:55:19.061418+0000","last_clean":"2024-09-05T11:55:19.061418+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:55:19.061418+0000","last_undegraded":"2024-09-05T11:55:19.061418+0000","last_fullsized":"2024-09-05T11:55:19.061418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753244+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:55:18.753244+0000","last_peered":"2024-09-05T11:55:18.753244+0000","last_clean":"2024-09-05T11:55:18.753244+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:55:18.753244+0000","last_undegraded":"2024-09-05T11:55:18.753244+0000","last_fullsized":"2024-09-05T11:55:18.753244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754442+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:55:18.754442+0000","last_peered":"2024-09-05T11:55:18.754442+0000","last_clean":"2024-09-05T11:55:18.754442+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:55:18.754442+0000","last_undegraded":"2024-09-05T11:55:18.754442+0000","last_fullsized":"2024-09-05T11:55:18.754442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751541+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:55:18.751541+0000","last_peered":"2024-09-05T11:55:18.751541+0000","last_clean":"2024-09-05T11:55:18.751541+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:55:18.751541+0000","last_undegraded":"2024-09-05T11:55:18.751541+0000","last_fullsized":"2024-09-05T11:55:18.751541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748945+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:55:18.748945+0000","last_peered":"2024-09-05T11:55:18.748945+0000","last_clean":"2024-09-05T11:55:18.748945+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:55:18.748945+0000","last_undegraded":"2024-09-05T11:55:18.748945+0000","last_fullsized":"2024-09-05T11:55:18.748945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.752031+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:55:18.752031+0000","last_peered":"2024-09-05T11:55:18.752031+0000","last_clean":"2024-09-05T11:55:18.752031+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:55:18.752031+0000","last_undegraded":"2024-09-05T11:55:18.752031+0000","last_fullsized":"2024-09-05T11:55:18.752031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828990+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:55:18.828990+0000","last_peered":"2024-09-05T11:55:18.828990+0000","last_clean":"2024-09-05T11:55:18.828990+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:55:18.828990+0000","last_undegraded":"2024-09-05T11:55:18.828990+0000","last_fullsized":"2024-09-05T11:55:18.828990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829035+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:55:18.829035+0000","last_peered":"2024-09-05T11:55:18.829035+0000","last_clean":"2024-09-05T11:55:18.829035+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:55:18.829035+0000","last_undegraded":"2024-09-05T11:55:18.829035+0000","last_fullsized":"2024-09-05T11:55:18.829035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750338+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:55:18.750338+0000","last_peered":"2024-09-05T11:55:18.750338+0000","last_clean":"2024-09-05T11:55:18.750338+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:55:18.750338+0000","last_undegraded":"2024-09-05T11:55:18.750338+0000","last_fullsized":"2024-09-05T11:55:18.750338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65929,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":131,"num_read_kb":132,"num_write":89,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":197787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":513188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1712128,"data_stored":1168881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906989,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1108234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":509092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1638400,"data_stored":1103738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":509196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":574257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":574223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:56:14.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:56:15.247 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:15.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:15 smithi059 ceph-mon[30025]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 -- 172.21.15.59:0/3574094969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 msgr2=0x7fae20110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3574094969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae20110b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae18009cd0 tx=0x7fae1802a970 comp rx=0 tx=0).stop 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 -- 172.21.15.59:0/3574094969 shutdown_connections 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3574094969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae20110b00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3574094969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae20109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3574094969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae201000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 -- 172.21.15.59:0/3574094969 >> 172.21.15.59:0/3574094969 conn(0x7fae200fb920 msgr2=0x7fae200fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 -- 172.21.15.59:0/3574094969 shutdown_connections 2024-09-05T11:56:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.754+0000 7fae1fe00640 1 -- 172.21.15.59:0/3574094969 wait complete. 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.755+0000 7fae1fe00640 1 Processor -- start 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.755+0000 7fae1fe00640 1 -- start start 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae2019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae2019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae20113fb0 con 0x7fae2010a160 2024-09-05T11:56:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fae20113e30 con 0x7fae201005f0 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae20114130 con 0x7fae200ffcd0 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46526/0 (socket says 172.21.15.59:46526) 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 -- 172.21.15.59:0/3058277143 learned_addr learned my addr 172.21.15.59:0/3058277143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1f400640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae2019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1e000640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae2019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 -- 172.21.15.59:0/3058277143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 msgr2=0x7fae2019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae2019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 -- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 msgr2=0x7fae2019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.756+0000 7fae1ea00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae2019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae1ea00640 1 -- 172.21.15.59:0/3058277143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019f4a0 con 0x7fae200ffcd0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae1e000640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae2019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae1ea00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000cd80 con 0x7fae200ffcd0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae10010910 con 0x7fae200ffcd0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019f790 con 0x7fae200ffcd0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100188f0 con 0x7fae200ffcd0 2024-09-05T11:56:15.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.757+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a7070 con 0x7fae200ffcd0 2024-09-05T11:56:15.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.758+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae201000b0 con 0x7fae200ffcd0 2024-09-05T11:56:15.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.760+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae10010ac0 con 0x7fae200ffcd0 2024-09-05T11:56:15.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.762+0000 7fae17400640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:15.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.762+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae100cff00 con 0x7fae200ffcd0 2024-09-05T11:56:15.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.762+0000 7fae1e000640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:15.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.763+0000 7fae1e000640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 0x7fae040a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae200fdd80 tx=0x7fae0c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:15.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.764+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae1009c940 con 0x7fae200ffcd0 2024-09-05T11:56:15.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:15 smithi189 ceph-mon[34772]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:15.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:15 smithi089 ceph-mon[34499]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:15.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.945+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae2010c1d0 con 0x7fae040a2c10 2024-09-05T11:56:15.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.955+0000 7fae17400640 1 -- 172.21.15.59:0/3058277143 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340289 (secure 0 0 0) 0x7fae2010c1d0 con 0x7fae040a2c10 2024-09-05T11:56:15.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:15.962 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:56:15.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 msgr2=0x7fae040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:15.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 0x7fae040a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae200fdd80 tx=0x7fae0c005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 msgr2=0x7fae2019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 shutdown_connections 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fae040a2c10 0x7fae040a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae2010a160 0x7fae2019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae201005f0 0x7fae2019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 --2- 172.21.15.59:0/3058277143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae200ffcd0 0x7fae2019a4f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 >> 172.21.15.59:0/3058277143 conn(0x7fae200fb920 msgr2=0x7fae20107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 shutdown_connections 2024-09-05T11:56:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:15.959+0000 7fae1fe00640 1 -- 172.21.15.59:0/3058277143 wait complete. 2024-09-05T11:56:16.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:16 smithi059 ceph-mon[30025]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:16.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:16 smithi189 ceph-mon[34772]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:16.841 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-05T11:56:14.876599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656531,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":257,"num_read_kb":241,"num_write":272,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375840,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565984,"statfs":{"total":1151940427776,"available":1151555567616,"internally_reserved":0,"allocated":14434304,"data_stored":8110763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003298"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748911+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:55:18.748911+0000","last_peered":"2024-09-05T11:55:18.748911+0000","last_clean":"2024-09-05T11:55:18.748911+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:55:18.748911+0000","last_undegraded":"2024-09-05T11:55:18.748911+0000","last_fullsized":"2024-09-05T11:55:18.748911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959651+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:55:18.959651+0000","last_peered":"2024-09-05T11:55:18.959651+0000","last_clean":"2024-09-05T11:55:18.959651+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:55:18.959651+0000","last_undegraded":"2024-09-05T11:55:18.959651+0000","last_fullsized":"2024-09-05T11:55:18.959651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960937+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:55:18.960937+0000","last_peered":"2024-09-05T11:55:18.960937+0000","last_clean":"2024-09-05T11:55:18.960937+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:55:18.960937+0000","last_undegraded":"2024-09-05T11:55:18.960937+0000","last_fullsized":"2024-09-05T11:55:18.960937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959828+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:55:18.959828+0000","last_peered":"2024-09-05T11:55:18.959828+0000","last_clean":"2024-09-05T11:55:18.959828+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:55:18.959828+0000","last_undegraded":"2024-09-05T11:55:18.959828+0000","last_fullsized":"2024-09-05T11:55:18.959828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829255+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:55:18.829255+0000","last_peered":"2024-09-05T11:55:18.829255+0000","last_clean":"2024-09-05T11:55:18.829255+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:55:18.829255+0000","last_undegraded":"2024-09-05T11:55:18.829255+0000","last_fullsized":"2024-09-05T11:55:18.829255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960868+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:55:18.960868+0000","last_peered":"2024-09-05T11:55:18.960868+0000","last_clean":"2024-09-05T11:55:18.960868+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:55:18.960868+0000","last_undegraded":"2024-09-05T11:55:18.960868+0000","last_fullsized":"2024-09-05T11:55:18.960868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750148+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:55:18.750148+0000","last_peered":"2024-09-05T11:55:18.750148+0000","last_clean":"2024-09-05T11:55:18.750148+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:55:18.750148+0000","last_undegraded":"2024-09-05T11:55:18.750148+0000","last_fullsized":"2024-09-05T11:55:18.750148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061069+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:55:19.061069+0000","last_peered":"2024-09-05T11:55:19.061069+0000","last_clean":"2024-09-05T11:55:19.061069+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:55:19.061069+0000","last_undegraded":"2024-09-05T11:55:19.061069+0000","last_fullsized":"2024-09-05T11:55:19.061069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746730+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:55:18.746730+0000","last_peered":"2024-09-05T11:55:18.746730+0000","last_clean":"2024-09-05T11:55:18.746730+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:55:18.746730+0000","last_undegraded":"2024-09-05T11:55:18.746730+0000","last_fullsized":"2024-09-05T11:55:18.746730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750651+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:55:18.750651+0000","last_peered":"2024-09-05T11:55:18.750651+0000","last_clean":"2024-09-05T11:55:18.750651+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:55:18.750651+0000","last_undegraded":"2024-09-05T11:55:18.750651+0000","last_fullsized":"2024-09-05T11:55:18.750651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749383+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:55:18.749383+0000","last_peered":"2024-09-05T11:55:18.749383+0000","last_clean":"2024-09-05T11:55:18.749383+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:55:18.749383+0000","last_undegraded":"2024-09-05T11:55:18.749383+0000","last_fullsized":"2024-09-05T11:55:18.749383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746451+0000","last_change":"2024-09-05T11:54:38.607159+0000","last_active":"2024-09-05T11:55:18.746451+0000","last_peered":"2024-09-05T11:55:18.746451+0000","last_clean":"2024-09-05T11:55:18.746451+0000","last_became_active":"2024-09-05T11:54:38.606569+0000","last_became_peered":"2024-09-05T11:54:38.606569+0000","last_unstale":"2024-09-05T11:55:18.746451+0000","last_undegraded":"2024-09-05T11:55:18.746451+0000","last_fullsized":"2024-09-05T11:55:18.746451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:28:07.583731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749879+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:55:18.749879+0000","last_peered":"2024-09-05T11:55:18.749879+0000","last_clean":"2024-09-05T11:55:18.749879+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:55:18.749879+0000","last_undegraded":"2024-09-05T11:55:18.749879+0000","last_fullsized":"2024-09-05T11:55:18.749879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749076+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:55:18.749076+0000","last_peered":"2024-09-05T11:55:18.749076+0000","last_clean":"2024-09-05T11:55:18.749076+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:55:18.749076+0000","last_undegraded":"2024-09-05T11:55:18.749076+0000","last_fullsized":"2024-09-05T11:55:18.749076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746802+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:55:18.746802+0000","last_peered":"2024-09-05T11:55:18.746802+0000","last_clean":"2024-09-05T11:55:18.746802+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:55:18.746802+0000","last_undegraded":"2024-09-05T11:55:18.746802+0000","last_fullsized":"2024-09-05T11:55:18.746802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959081+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:55:18.959081+0000","last_peered":"2024-09-05T11:55:18.959081+0000","last_clean":"2024-09-05T11:55:18.959081+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:55:18.959081+0000","last_undegraded":"2024-09-05T11:55:18.959081+0000","last_fullsized":"2024-09-05T11:55:18.959081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744681+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:55:18.744681+0000","last_peered":"2024-09-05T11:55:18.744681+0000","last_clean":"2024-09-05T11:55:18.744681+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:55:18.744681+0000","last_undegraded":"2024-09-05T11:55:18.744681+0000","last_fullsized":"2024-09-05T11:55:18.744681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753048+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:55:18.753048+0000","last_peered":"2024-09-05T11:55:18.753048+0000","last_clean":"2024-09-05T11:55:18.753048+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:55:18.753048+0000","last_undegraded":"2024-09-05T11:55:18.753048+0000","last_fullsized":"2024-09-05T11:55:18.753048+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748420+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:55:18.748420+0000","last_peered":"2024-09-05T11:55:18.748420+0000","last_clean":"2024-09-05T11:55:18.748420+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:55:18.748420+0000","last_undegraded":"2024-09-05T11:55:18.748420+0000","last_fullsized":"2024-09-05T11:55:18.748420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959409+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:55:18.959409+0000","last_peered":"2024-09-05T11:55:18.959409+0000","last_clean":"2024-09-05T11:55:18.959409+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:55:18.959409+0000","last_undegraded":"2024-09-05T11:55:18.959409+0000","last_fullsized":"2024-09-05T11:55:18.959409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748594+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:55:18.748594+0000","last_peered":"2024-09-05T11:55:18.748594+0000","last_clean":"2024-09-05T11:55:18.748594+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:55:18.748594+0000","last_undegraded":"2024-09-05T11:55:18.748594+0000","last_fullsized":"2024-09-05T11:55:18.748594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751886+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:55:18.751886+0000","last_peered":"2024-09-05T11:55:18.751886+0000","last_clean":"2024-09-05T11:55:18.751886+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:55:18.751886+0000","last_undegraded":"2024-09-05T11:55:18.751886+0000","last_fullsized":"2024-09-05T11:55:18.751886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060880+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:55:19.060880+0000","last_peered":"2024-09-05T11:55:19.060880+0000","last_clean":"2024-09-05T11:55:19.060880+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:55:19.060880+0000","last_undegraded":"2024-09-05T11:55:19.060880+0000","last_fullsized":"2024-09-05T11:55:19.060880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746843+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:55:18.746843+0000","last_peered":"2024-09-05T11:55:18.746843+0000","last_clean":"2024-09-05T11:55:18.746843+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:55:18.746843+0000","last_undegraded":"2024-09-05T11:55:18.746843+0000","last_fullsized":"2024-09-05T11:55:18.746843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829155+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:55:18.829155+0000","last_peered":"2024-09-05T11:55:18.829155+0000","last_clean":"2024-09-05T11:55:18.829155+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:55:18.829155+0000","last_undegraded":"2024-09-05T11:55:18.829155+0000","last_fullsized":"2024-09-05T11:55:18.829155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747880+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:55:18.747880+0000","last_peered":"2024-09-05T11:55:18.747880+0000","last_clean":"2024-09-05T11:55:18.747880+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:55:18.747880+0000","last_undegraded":"2024-09-05T11:55:18.747880+0000","last_fullsized":"2024-09-05T11:55:18.747880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744630+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:55:18.744630+0000","last_peered":"2024-09-05T11:55:18.744630+0000","last_clean":"2024-09-05T11:55:18.744630+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:55:18.744630+0000","last_undegraded":"2024-09-05T11:55:18.744630+0000","last_fullsized":"2024-09-05T11:55:18.744630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959726+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:55:18.959726+0000","last_peered":"2024-09-05T11:55:18.959726+0000","last_clean":"2024-09-05T11:55:18.959726+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:55:18.959726+0000","last_undegraded":"2024-09-05T11:55:18.959726+0000","last_fullsized":"2024-09-05T11:55:18.959726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:29.562399+0000","last_change":"2024-09-05T11:54:29.103741+0000","last_active":"2024-09-05T11:55:29.562399+0000","last_peered":"2024-09-05T11:55:29.562399+0000","last_clean":"2024-09-05T11:55:29.562399+0000","last_became_active":"2024-09-05T11:54:29.103285+0000","last_became_peered":"2024-09-05T11:54:29.103285+0000","last_unstale":"2024-09-05T11:55:29.562399+0000","last_undegraded":"2024-09-05T11:55:29.562399+0000","last_fullsized":"2024-09-05T11:55:29.562399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:01:16.713162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.958714+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:55:18.958714+0000","last_peered":"2024-09-05T11:55:18.958714+0000","last_clean":"2024-09-05T11:55:18.958714+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:55:18.958714+0000","last_undegraded":"2024-09-05T11:55:18.958714+0000","last_fullsized":"2024-09-05T11:55:18.958714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744903+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:55:18.744903+0000","last_peered":"2024-09-05T11:55:18.744903+0000","last_clean":"2024-09-05T11:55:18.744903+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:55:18.744903+0000","last_undegraded":"2024-09-05T11:55:18.744903+0000","last_fullsized":"2024-09-05T11:55:18.744903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750269+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:55:18.750269+0000","last_peered":"2024-09-05T11:55:18.750269+0000","last_clean":"2024-09-05T11:55:18.750269+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:55:18.750269+0000","last_undegraded":"2024-09-05T11:55:18.750269+0000","last_fullsized":"2024-09-05T11:55:18.750269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748452+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:55:18.748452+0000","last_peered":"2024-09-05T11:55:18.748452+0000","last_clean":"2024-09-05T11:55:18.748452+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:55:18.748452+0000","last_undegraded":"2024-09-05T11:55:18.748452+0000","last_fullsized":"2024-09-05T11:55:18.748452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745443+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:55:18.745443+0000","last_peered":"2024-09-05T11:55:18.745443+0000","last_clean":"2024-09-05T11:55:18.745443+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:55:18.745443+0000","last_undegraded":"2024-09-05T11:55:18.745443+0000","last_fullsized":"2024-09-05T11:55:18.745443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748502+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:55:18.748502+0000","last_peered":"2024-09-05T11:55:18.748502+0000","last_clean":"2024-09-05T11:55:18.748502+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:55:18.748502+0000","last_undegraded":"2024-09-05T11:55:18.748502+0000","last_fullsized":"2024-09-05T11:55:18.748502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750324+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:55:18.750324+0000","last_peered":"2024-09-05T11:55:18.750324+0000","last_clean":"2024-09-05T11:55:18.750324+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:55:18.750324+0000","last_undegraded":"2024-09-05T11:55:18.750324+0000","last_fullsized":"2024-09-05T11:55:18.750324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745255+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:55:18.745255+0000","last_peered":"2024-09-05T11:55:18.745255+0000","last_clean":"2024-09-05T11:55:18.745255+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:55:18.745255+0000","last_undegraded":"2024-09-05T11:55:18.745255+0000","last_fullsized":"2024-09-05T11:55:18.745255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749954+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:55:18.749954+0000","last_peered":"2024-09-05T11:55:18.749954+0000","last_clean":"2024-09-05T11:55:18.749954+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:55:18.749954+0000","last_undegraded":"2024-09-05T11:55:18.749954+0000","last_fullsized":"2024-09-05T11:55:18.749954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744674+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:55:18.744674+0000","last_peered":"2024-09-05T11:55:18.744674+0000","last_clean":"2024-09-05T11:55:18.744674+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:55:18.744674+0000","last_undegraded":"2024-09-05T11:55:18.744674+0000","last_fullsized":"2024-09-05T11:55:18.744674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751392+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:55:18.751392+0000","last_peered":"2024-09-05T11:55:18.751392+0000","last_clean":"2024-09-05T11:55:18.751392+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:55:18.751392+0000","last_undegraded":"2024-09-05T11:55:18.751392+0000","last_fullsized":"2024-09-05T11:55:18.751392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744437+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:55:18.744437+0000","last_peered":"2024-09-05T11:55:18.744437+0000","last_clean":"2024-09-05T11:55:18.744437+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:55:18.744437+0000","last_undegraded":"2024-09-05T11:55:18.744437+0000","last_fullsized":"2024-09-05T11:55:18.744437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744739+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:55:18.744739+0000","last_peered":"2024-09-05T11:55:18.744739+0000","last_clean":"2024-09-05T11:55:18.744739+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:55:18.744739+0000","last_undegraded":"2024-09-05T11:55:18.744739+0000","last_fullsized":"2024-09-05T11:55:18.744739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744301+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:55:18.744301+0000","last_peered":"2024-09-05T11:55:18.744301+0000","last_clean":"2024-09-05T11:55:18.744301+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:55:18.744301+0000","last_undegraded":"2024-09-05T11:55:18.744301+0000","last_fullsized":"2024-09-05T11:55:18.744301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749802+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:55:18.749802+0000","last_peered":"2024-09-05T11:55:18.749802+0000","last_clean":"2024-09-05T11:55:18.749802+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:55:18.749802+0000","last_undegraded":"2024-09-05T11:55:18.749802+0000","last_fullsized":"2024-09-05T11:55:18.749802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749669+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:55:18.749669+0000","last_peered":"2024-09-05T11:55:18.749669+0000","last_clean":"2024-09-05T11:55:18.749669+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:55:18.749669+0000","last_undegraded":"2024-09-05T11:55:18.749669+0000","last_fullsized":"2024-09-05T11:55:18.749669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959105+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:55:18.959105+0000","last_peered":"2024-09-05T11:55:18.959105+0000","last_clean":"2024-09-05T11:55:18.959105+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:55:18.959105+0000","last_undegraded":"2024-09-05T11:55:18.959105+0000","last_fullsized":"2024-09-05T11:55:18.959105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747099+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:55:18.747099+0000","last_peered":"2024-09-05T11:55:18.747099+0000","last_clean":"2024-09-05T11:55:18.747099+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:55:18.747099+0000","last_undegraded":"2024-09-05T11:55:18.747099+0000","last_fullsized":"2024-09-05T11:55:18.747099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750002+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:55:18.750002+0000","last_peered":"2024-09-05T11:55:18.750002+0000","last_clean":"2024-09-05T11:55:18.750002+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:55:18.750002+0000","last_undegraded":"2024-09-05T11:55:18.750002+0000","last_fullsized":"2024-09-05T11:55:18.750002+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750328+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:55:18.750328+0000","last_peered":"2024-09-05T11:55:18.750328+0000","last_clean":"2024-09-05T11:55:18.750328+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:55:18.750328+0000","last_undegraded":"2024-09-05T11:55:18.750328+0000","last_fullsized":"2024-09-05T11:55:18.750328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750439+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:55:18.750439+0000","last_peered":"2024-09-05T11:55:18.750439+0000","last_clean":"2024-09-05T11:55:18.750439+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:55:18.750439+0000","last_undegraded":"2024-09-05T11:55:18.750439+0000","last_fullsized":"2024-09-05T11:55:18.750439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745209+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:55:18.745209+0000","last_peered":"2024-09-05T11:55:18.745209+0000","last_clean":"2024-09-05T11:55:18.745209+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:55:18.745209+0000","last_undegraded":"2024-09-05T11:55:18.745209+0000","last_fullsized":"2024-09-05T11:55:18.745209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060892+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:55:19.060892+0000","last_peered":"2024-09-05T11:55:19.060892+0000","last_clean":"2024-09-05T11:55:19.060892+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:55:19.060892+0000","last_undegraded":"2024-09-05T11:55:19.060892+0000","last_fullsized":"2024-09-05T11:55:19.060892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750265+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:55:18.750265+0000","last_peered":"2024-09-05T11:55:18.750265+0000","last_clean":"2024-09-05T11:55:18.750265+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:55:18.750265+0000","last_undegraded":"2024-09-05T11:55:18.750265+0000","last_fullsized":"2024-09-05T11:55:18.750265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750840+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:55:18.750840+0000","last_peered":"2024-09-05T11:55:18.750840+0000","last_clean":"2024-09-05T11:55:18.750840+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:55:18.750840+0000","last_undegraded":"2024-09-05T11:55:18.750840+0000","last_fullsized":"2024-09-05T11:55:18.750840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828776+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:55:18.828776+0000","last_peered":"2024-09-05T11:55:18.828776+0000","last_clean":"2024-09-05T11:55:18.828776+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:55:18.828776+0000","last_undegraded":"2024-09-05T11:55:18.828776+0000","last_fullsized":"2024-09-05T11:55:18.828776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750453+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:55:18.750453+0000","last_peered":"2024-09-05T11:55:18.750453+0000","last_clean":"2024-09-05T11:55:18.750453+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:55:18.750453+0000","last_undegraded":"2024-09-05T11:55:18.750453+0000","last_fullsized":"2024-09-05T11:55:18.750453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745094+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:55:18.745094+0000","last_peered":"2024-09-05T11:55:18.745094+0000","last_clean":"2024-09-05T11:55:18.745094+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:55:18.745094+0000","last_undegraded":"2024-09-05T11:55:18.745094+0000","last_fullsized":"2024-09-05T11:55:18.745094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749801+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:55:18.749801+0000","last_peered":"2024-09-05T11:55:18.749801+0000","last_clean":"2024-09-05T11:55:18.749801+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:55:18.749801+0000","last_undegraded":"2024-09-05T11:55:18.749801+0000","last_fullsized":"2024-09-05T11:55:18.749801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749962+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:55:18.749962+0000","last_peered":"2024-09-05T11:55:18.749962+0000","last_clean":"2024-09-05T11:55:18.749962+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:55:18.749962+0000","last_undegraded":"2024-09-05T11:55:18.749962+0000","last_fullsized":"2024-09-05T11:55:18.749962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749212+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:55:18.749212+0000","last_peered":"2024-09-05T11:55:18.749212+0000","last_clean":"2024-09-05T11:55:18.749212+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:55:18.749212+0000","last_undegraded":"2024-09-05T11:55:18.749212+0000","last_fullsized":"2024-09-05T11:55:18.749212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748596+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:55:18.748596+0000","last_peered":"2024-09-05T11:55:18.748596+0000","last_clean":"2024-09-05T11:55:18.748596+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:55:18.748596+0000","last_undegraded":"2024-09-05T11:55:18.748596+0000","last_fullsized":"2024-09-05T11:55:18.748596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749735+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:55:18.749735+0000","last_peered":"2024-09-05T11:55:18.749735+0000","last_clean":"2024-09-05T11:55:18.749735+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:55:18.749735+0000","last_undegraded":"2024-09-05T11:55:18.749735+0000","last_fullsized":"2024-09-05T11:55:18.749735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751378+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:55:18.751378+0000","last_peered":"2024-09-05T11:55:18.751378+0000","last_clean":"2024-09-05T11:55:18.751378+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:55:18.751378+0000","last_undegraded":"2024-09-05T11:55:18.751378+0000","last_fullsized":"2024-09-05T11:55:18.751378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744997+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:55:18.744997+0000","last_peered":"2024-09-05T11:55:18.744997+0000","last_clean":"2024-09-05T11:55:18.744997+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:55:18.744997+0000","last_undegraded":"2024-09-05T11:55:18.744997+0000","last_fullsized":"2024-09-05T11:55:18.744997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749638+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:55:18.749638+0000","last_peered":"2024-09-05T11:55:18.749638+0000","last_clean":"2024-09-05T11:55:18.749638+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:55:18.749638+0000","last_undegraded":"2024-09-05T11:55:18.749638+0000","last_fullsized":"2024-09-05T11:55:18.749638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747345+0000","last_change":"2024-09-05T11:54:29.102616+0000","last_active":"2024-09-05T11:55:18.747345+0000","last_peered":"2024-09-05T11:55:18.747345+0000","last_clean":"2024-09-05T11:55:18.747345+0000","last_became_active":"2024-09-05T11:54:29.102126+0000","last_became_peered":"2024-09-05T11:54:29.102126+0000","last_unstale":"2024-09-05T11:55:18.747345+0000","last_undegraded":"2024-09-05T11:55:18.747345+0000","last_fullsized":"2024-09-05T11:55:18.747345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:09:28.848395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748687+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:55:18.748687+0000","last_peered":"2024-09-05T11:55:18.748687+0000","last_clean":"2024-09-05T11:55:18.748687+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:55:18.748687+0000","last_undegraded":"2024-09-05T11:55:18.748687+0000","last_fullsized":"2024-09-05T11:55:18.748687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750129+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:55:18.750129+0000","last_peered":"2024-09-05T11:55:18.750129+0000","last_clean":"2024-09-05T11:55:18.750129+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:55:18.750129+0000","last_undegraded":"2024-09-05T11:55:18.750129+0000","last_fullsized":"2024-09-05T11:55:18.750129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751077+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:55:18.751077+0000","last_peered":"2024-09-05T11:55:18.751077+0000","last_clean":"2024-09-05T11:55:18.751077+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:55:18.751077+0000","last_undegraded":"2024-09-05T11:55:18.751077+0000","last_fullsized":"2024-09-05T11:55:18.751077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748683+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:55:18.748683+0000","last_peered":"2024-09-05T11:55:18.748683+0000","last_clean":"2024-09-05T11:55:18.748683+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:55:18.748683+0000","last_undegraded":"2024-09-05T11:55:18.748683+0000","last_fullsized":"2024-09-05T11:55:18.748683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744841+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:55:18.744841+0000","last_peered":"2024-09-05T11:55:18.744841+0000","last_clean":"2024-09-05T11:55:18.744841+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:55:18.744841+0000","last_undegraded":"2024-09-05T11:55:18.744841+0000","last_fullsized":"2024-09-05T11:55:18.744841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744915+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:55:18.744915+0000","last_peered":"2024-09-05T11:55:18.744915+0000","last_clean":"2024-09-05T11:55:18.744915+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:55:18.744915+0000","last_undegraded":"2024-09-05T11:55:18.744915+0000","last_fullsized":"2024-09-05T11:55:18.744915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750719+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:55:18.750719+0000","last_peered":"2024-09-05T11:55:18.750719+0000","last_clean":"2024-09-05T11:55:18.750719+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:55:18.750719+0000","last_undegraded":"2024-09-05T11:55:18.750719+0000","last_fullsized":"2024-09-05T11:55:18.750719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749744+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:55:18.749744+0000","last_peered":"2024-09-05T11:55:18.749744+0000","last_clean":"2024-09-05T11:55:18.749744+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:55:18.749744+0000","last_undegraded":"2024-09-05T11:55:18.749744+0000","last_fullsized":"2024-09-05T11:55:18.749744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:56:09.814251+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:09.814251+0000","last_peered":"2024-09-05T11:56:09.814251+0000","last_clean":"2024-09-05T11:56:09.814251+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:09.814251+0000","last_undegraded":"2024-09-05T11:56:09.814251+0000","last_fullsized":"2024-09-05T11:56:09.814251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65131,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":126,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747427+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:55:18.747427+0000","last_peered":"2024-09-05T11:55:18.747427+0000","last_clean":"2024-09-05T11:55:18.747427+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:55:18.747427+0000","last_undegraded":"2024-09-05T11:55:18.747427+0000","last_fullsized":"2024-09-05T11:55:18.747427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061114+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:55:19.061114+0000","last_peered":"2024-09-05T11:55:19.061114+0000","last_clean":"2024-09-05T11:55:19.061114+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:55:19.061114+0000","last_undegraded":"2024-09-05T11:55:19.061114+0000","last_fullsized":"2024-09-05T11:55:19.061114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750985+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:55:18.750985+0000","last_peered":"2024-09-05T11:55:18.750985+0000","last_clean":"2024-09-05T11:55:18.750985+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:55:18.750985+0000","last_undegraded":"2024-09-05T11:55:18.750985+0000","last_fullsized":"2024-09-05T11:55:18.750985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.746602+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:55:18.746602+0000","last_peered":"2024-09-05T11:55:18.746602+0000","last_clean":"2024-09-05T11:55:18.746602+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:55:18.746602+0000","last_undegraded":"2024-09-05T11:55:18.746602+0000","last_fullsized":"2024-09-05T11:55:18.746602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829307+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:55:18.829307+0000","last_peered":"2024-09-05T11:55:18.829307+0000","last_clean":"2024-09-05T11:55:18.829307+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:55:18.829307+0000","last_undegraded":"2024-09-05T11:55:18.829307+0000","last_fullsized":"2024-09-05T11:55:18.829307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750531+0000","last_change":"2024-09-05T11:54:29.106596+0000","last_active":"2024-09-05T11:55:18.750531+0000","last_peered":"2024-09-05T11:55:18.750531+0000","last_clean":"2024-09-05T11:55:18.750531+0000","last_became_active":"2024-09-05T11:54:29.106213+0000","last_became_peered":"2024-09-05T11:54:29.106213+0000","last_unstale":"2024-09-05T11:55:18.750531+0000","last_undegraded":"2024-09-05T11:55:18.750531+0000","last_fullsized":"2024-09-05T11:55:18.750531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:24:44.592047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747050+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:55:18.747050+0000","last_peered":"2024-09-05T11:55:18.747050+0000","last_clean":"2024-09-05T11:55:18.747050+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:55:18.747050+0000","last_undegraded":"2024-09-05T11:55:18.747050+0000","last_fullsized":"2024-09-05T11:55:18.747050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061360+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:55:19.061360+0000","last_peered":"2024-09-05T11:55:19.061360+0000","last_clean":"2024-09-05T11:55:19.061360+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:55:19.061360+0000","last_undegraded":"2024-09-05T11:55:19.061360+0000","last_fullsized":"2024-09-05T11:55:19.061360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745488+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:55:18.745488+0000","last_peered":"2024-09-05T11:55:18.745488+0000","last_clean":"2024-09-05T11:55:18.745488+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:55:18.745488+0000","last_undegraded":"2024-09-05T11:55:18.745488+0000","last_fullsized":"2024-09-05T11:55:18.745488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959078+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:55:18.959078+0000","last_peered":"2024-09-05T11:55:18.959078+0000","last_clean":"2024-09-05T11:55:18.959078+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:55:18.959078+0000","last_undegraded":"2024-09-05T11:55:18.959078+0000","last_fullsized":"2024-09-05T11:55:18.959078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829040+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:55:18.829040+0000","last_peered":"2024-09-05T11:55:18.829040+0000","last_clean":"2024-09-05T11:55:18.829040+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:55:18.829040+0000","last_undegraded":"2024-09-05T11:55:18.829040+0000","last_fullsized":"2024-09-05T11:55:18.829040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747393+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:55:18.747393+0000","last_peered":"2024-09-05T11:55:18.747393+0000","last_clean":"2024-09-05T11:55:18.747393+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:55:18.747393+0000","last_undegraded":"2024-09-05T11:55:18.747393+0000","last_fullsized":"2024-09-05T11:55:18.747393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751542+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:55:18.751542+0000","last_peered":"2024-09-05T11:55:18.751542+0000","last_clean":"2024-09-05T11:55:18.751542+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:55:18.751542+0000","last_undegraded":"2024-09-05T11:55:18.751542+0000","last_fullsized":"2024-09-05T11:55:18.751542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749841+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:55:18.749841+0000","last_peered":"2024-09-05T11:55:18.749841+0000","last_clean":"2024-09-05T11:55:18.749841+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:55:18.749841+0000","last_undegraded":"2024-09-05T11:55:18.749841+0000","last_fullsized":"2024-09-05T11:55:18.749841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959536+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:55:18.959536+0000","last_peered":"2024-09-05T11:55:18.959536+0000","last_clean":"2024-09-05T11:55:18.959536+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:55:18.959536+0000","last_undegraded":"2024-09-05T11:55:18.959536+0000","last_fullsized":"2024-09-05T11:55:18.959536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744321+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:55:18.744321+0000","last_peered":"2024-09-05T11:55:18.744321+0000","last_clean":"2024-09-05T11:55:18.744321+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:55:18.744321+0000","last_undegraded":"2024-09-05T11:55:18.744321+0000","last_fullsized":"2024-09-05T11:55:18.744321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745069+0000","last_change":"2024-09-05T11:54:29.101408+0000","last_active":"2024-09-05T11:55:18.745069+0000","last_peered":"2024-09-05T11:55:18.745069+0000","last_clean":"2024-09-05T11:55:18.745069+0000","last_became_active":"2024-09-05T11:54:29.101052+0000","last_became_peered":"2024-09-05T11:54:29.101052+0000","last_unstale":"2024-09-05T11:55:18.745069+0000","last_undegraded":"2024-09-05T11:55:18.745069+0000","last_fullsized":"2024-09-05T11:55:18.745069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:05:40.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.960761+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:55:18.960761+0000","last_peered":"2024-09-05T11:55:18.960761+0000","last_clean":"2024-09-05T11:55:18.960761+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:55:18.960761+0000","last_undegraded":"2024-09-05T11:55:18.960761+0000","last_fullsized":"2024-09-05T11:55:18.960761+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744820+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:55:18.744820+0000","last_peered":"2024-09-05T11:55:18.744820+0000","last_clean":"2024-09-05T11:55:18.744820+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:55:18.744820+0000","last_undegraded":"2024-09-05T11:55:18.744820+0000","last_fullsized":"2024-09-05T11:55:18.744820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750928+0000","last_change":"2024-09-05T11:54:29.102371+0000","last_active":"2024-09-05T11:55:18.750928+0000","last_peered":"2024-09-05T11:55:18.750928+0000","last_clean":"2024-09-05T11:55:18.750928+0000","last_became_active":"2024-09-05T11:54:29.102066+0000","last_became_peered":"2024-09-05T11:54:29.102066+0000","last_unstale":"2024-09-05T11:55:18.750928+0000","last_undegraded":"2024-09-05T11:55:18.750928+0000","last_fullsized":"2024-09-05T11:55:18.750928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:55.008088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754810+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:55:18.754810+0000","last_peered":"2024-09-05T11:55:18.754810+0000","last_clean":"2024-09-05T11:55:18.754810+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:55:18.754810+0000","last_undegraded":"2024-09-05T11:55:18.754810+0000","last_fullsized":"2024-09-05T11:55:18.754810+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744917+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:55:18.744917+0000","last_peered":"2024-09-05T11:55:18.744917+0000","last_clean":"2024-09-05T11:55:18.744917+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:55:18.744917+0000","last_undegraded":"2024-09-05T11:55:18.744917+0000","last_fullsized":"2024-09-05T11:55:18.744917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748298+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:55:18.748298+0000","last_peered":"2024-09-05T11:55:18.748298+0000","last_clean":"2024-09-05T11:55:18.748298+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:55:18.748298+0000","last_undegraded":"2024-09-05T11:55:18.748298+0000","last_fullsized":"2024-09-05T11:55:18.748298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745098+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:55:18.745098+0000","last_peered":"2024-09-05T11:55:18.745098+0000","last_clean":"2024-09-05T11:55:18.745098+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:55:18.745098+0000","last_undegraded":"2024-09-05T11:55:18.745098+0000","last_fullsized":"2024-09-05T11:55:18.745098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744606+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:55:18.744606+0000","last_peered":"2024-09-05T11:55:18.744606+0000","last_clean":"2024-09-05T11:55:18.744606+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:55:18.744606+0000","last_undegraded":"2024-09-05T11:55:18.744606+0000","last_fullsized":"2024-09-05T11:55:18.744606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828789+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:55:18.828789+0000","last_peered":"2024-09-05T11:55:18.828789+0000","last_clean":"2024-09-05T11:55:18.828789+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:55:18.828789+0000","last_undegraded":"2024-09-05T11:55:18.828789+0000","last_fullsized":"2024-09-05T11:55:18.828789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744825+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:55:18.744825+0000","last_peered":"2024-09-05T11:55:18.744825+0000","last_clean":"2024-09-05T11:55:18.744825+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:55:18.744825+0000","last_undegraded":"2024-09-05T11:55:18.744825+0000","last_fullsized":"2024-09-05T11:55:18.744825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.749366+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:55:18.749366+0000","last_peered":"2024-09-05T11:55:18.749366+0000","last_clean":"2024-09-05T11:55:18.749366+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:55:18.749366+0000","last_undegraded":"2024-09-05T11:55:18.749366+0000","last_fullsized":"2024-09-05T11:55:18.749366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745046+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:55:18.745046+0000","last_peered":"2024-09-05T11:55:18.745046+0000","last_clean":"2024-09-05T11:55:18.745046+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:55:18.745046+0000","last_undegraded":"2024-09-05T11:55:18.745046+0000","last_fullsized":"2024-09-05T11:55:18.745046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959692+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:55:18.959692+0000","last_peered":"2024-09-05T11:55:18.959692+0000","last_clean":"2024-09-05T11:55:18.959692+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:55:18.959692+0000","last_undegraded":"2024-09-05T11:55:18.959692+0000","last_fullsized":"2024-09-05T11:55:18.959692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751303+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:55:18.751303+0000","last_peered":"2024-09-05T11:55:18.751303+0000","last_clean":"2024-09-05T11:55:18.751303+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:55:18.751303+0000","last_undegraded":"2024-09-05T11:55:18.751303+0000","last_fullsized":"2024-09-05T11:55:18.751303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.959252+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:55:18.959252+0000","last_peered":"2024-09-05T11:55:18.959252+0000","last_clean":"2024-09-05T11:55:18.959252+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:55:18.959252+0000","last_undegraded":"2024-09-05T11:55:18.959252+0000","last_fullsized":"2024-09-05T11:55:18.959252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061014+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:55:19.061014+0000","last_peered":"2024-09-05T11:55:19.061014+0000","last_clean":"2024-09-05T11:55:19.061014+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:55:19.061014+0000","last_undegraded":"2024-09-05T11:55:19.061014+0000","last_fullsized":"2024-09-05T11:55:19.061014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744715+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:55:18.744715+0000","last_peered":"2024-09-05T11:55:18.744715+0000","last_clean":"2024-09-05T11:55:18.744715+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:55:18.744715+0000","last_undegraded":"2024-09-05T11:55:18.744715+0000","last_fullsized":"2024-09-05T11:55:18.744715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754299+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:55:18.754299+0000","last_peered":"2024-09-05T11:55:18.754299+0000","last_clean":"2024-09-05T11:55:18.754299+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:55:18.754299+0000","last_undegraded":"2024-09-05T11:55:18.754299+0000","last_fullsized":"2024-09-05T11:55:18.754299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745216+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:55:18.745216+0000","last_peered":"2024-09-05T11:55:18.745216+0000","last_clean":"2024-09-05T11:55:18.745216+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:55:18.745216+0000","last_undegraded":"2024-09-05T11:55:18.745216+0000","last_fullsized":"2024-09-05T11:55:18.745216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748542+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:55:18.748542+0000","last_peered":"2024-09-05T11:55:18.748542+0000","last_clean":"2024-09-05T11:55:18.748542+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:55:18.748542+0000","last_undegraded":"2024-09-05T11:55:18.748542+0000","last_fullsized":"2024-09-05T11:55:18.748542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750435+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:55:18.750435+0000","last_peered":"2024-09-05T11:55:18.750435+0000","last_clean":"2024-09-05T11:55:18.750435+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:55:18.750435+0000","last_undegraded":"2024-09-05T11:55:18.750435+0000","last_fullsized":"2024-09-05T11:55:18.750435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.060771+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:55:19.060771+0000","last_peered":"2024-09-05T11:55:19.060771+0000","last_clean":"2024-09-05T11:55:19.060771+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:55:19.060771+0000","last_undegraded":"2024-09-05T11:55:19.060771+0000","last_fullsized":"2024-09-05T11:55:19.060771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828621+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:55:18.828621+0000","last_peered":"2024-09-05T11:55:18.828621+0000","last_clean":"2024-09-05T11:55:18.828621+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:55:18.828621+0000","last_undegraded":"2024-09-05T11:55:18.828621+0000","last_fullsized":"2024-09-05T11:55:18.828621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828531+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:55:18.828531+0000","last_peered":"2024-09-05T11:55:18.828531+0000","last_clean":"2024-09-05T11:55:18.828531+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:55:18.828531+0000","last_undegraded":"2024-09-05T11:55:18.828531+0000","last_fullsized":"2024-09-05T11:55:18.828531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.744676+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:55:18.744676+0000","last_peered":"2024-09-05T11:55:18.744676+0000","last_clean":"2024-09-05T11:55:18.744676+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:55:18.744676+0000","last_undegraded":"2024-09-05T11:55:18.744676+0000","last_fullsized":"2024-09-05T11:55:18.744676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.747214+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:55:18.747214+0000","last_peered":"2024-09-05T11:55:18.747214+0000","last_clean":"2024-09-05T11:55:18.747214+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:55:18.747214+0000","last_undegraded":"2024-09-05T11:55:18.747214+0000","last_fullsized":"2024-09-05T11:55:18.747214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750519+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:55:18.750519+0000","last_peered":"2024-09-05T11:55:18.750519+0000","last_clean":"2024-09-05T11:55:18.750519+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:55:18.750519+0000","last_undegraded":"2024-09-05T11:55:18.750519+0000","last_fullsized":"2024-09-05T11:55:18.750519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.745888+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:55:18.745888+0000","last_peered":"2024-09-05T11:55:18.745888+0000","last_clean":"2024-09-05T11:55:18.745888+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:55:18.745888+0000","last_undegraded":"2024-09-05T11:55:18.745888+0000","last_fullsized":"2024-09-05T11:55:18.745888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:19.061418+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:55:19.061418+0000","last_peered":"2024-09-05T11:55:19.061418+0000","last_clean":"2024-09-05T11:55:19.061418+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:55:19.061418+0000","last_undegraded":"2024-09-05T11:55:19.061418+0000","last_fullsized":"2024-09-05T11:55:19.061418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.753244+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:55:18.753244+0000","last_peered":"2024-09-05T11:55:18.753244+0000","last_clean":"2024-09-05T11:55:18.753244+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:55:18.753244+0000","last_undegraded":"2024-09-05T11:55:18.753244+0000","last_fullsized":"2024-09-05T11:55:18.753244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.754442+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:55:18.754442+0000","last_peered":"2024-09-05T11:55:18.754442+0000","last_clean":"2024-09-05T11:55:18.754442+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:55:18.754442+0000","last_undegraded":"2024-09-05T11:55:18.754442+0000","last_fullsized":"2024-09-05T11:55:18.754442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.751541+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:55:18.751541+0000","last_peered":"2024-09-05T11:55:18.751541+0000","last_clean":"2024-09-05T11:55:18.751541+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:55:18.751541+0000","last_undegraded":"2024-09-05T11:55:18.751541+0000","last_fullsized":"2024-09-05T11:55:18.751541+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.748945+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:55:18.748945+0000","last_peered":"2024-09-05T11:55:18.748945+0000","last_clean":"2024-09-05T11:55:18.748945+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:55:18.748945+0000","last_undegraded":"2024-09-05T11:55:18.748945+0000","last_fullsized":"2024-09-05T11:55:18.748945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.752031+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:55:18.752031+0000","last_peered":"2024-09-05T11:55:18.752031+0000","last_clean":"2024-09-05T11:55:18.752031+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:55:18.752031+0000","last_undegraded":"2024-09-05T11:55:18.752031+0000","last_fullsized":"2024-09-05T11:55:18.752031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.828990+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:55:18.828990+0000","last_peered":"2024-09-05T11:55:18.828990+0000","last_clean":"2024-09-05T11:55:18.828990+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:55:18.828990+0000","last_undegraded":"2024-09-05T11:55:18.828990+0000","last_fullsized":"2024-09-05T11:55:18.828990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.829035+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:55:18.829035+0000","last_peered":"2024-09-05T11:55:18.829035+0000","last_clean":"2024-09-05T11:55:18.829035+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:55:18.829035+0000","last_undegraded":"2024-09-05T11:55:18.829035+0000","last_fullsized":"2024-09-05T11:55:18.829035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-05T11:55:18.750338+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:55:18.750338+0000","last_peered":"2024-09-05T11:55:18.750338+0000","last_clean":"2024-09-05T11:55:18.750338+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:55:18.750338+0000","last_undegraded":"2024-09-05T11:55:18.750338+0000","last_fullsized":"2024-09-05T11:55:18.750338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65929,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":131,"num_read_kb":132,"num_write":89,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":197787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":513188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37016,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708136,"statfs":{"total":95995035648,"available":95957131264,"internally_reserved":0,"allocated":1712128,"data_stored":1168881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1108234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":509092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1638400,"data_stored":1103738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":513210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":509196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":574257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":574223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:56:16.847 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T11:56:16.847 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T11:56:16.848 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-05T11:56:16.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph health --format=json 2024-09-05T11:56:16.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:16 smithi089 ceph-mon[34499]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:17.226 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:17.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:17 smithi059 ceph-mon[30025]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:17.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.823+0000 7f563c600640 1 -- 172.21.15.59:0/1132770516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634074590 msgr2=0x7f5634074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:17.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.823+0000 7f563c600640 1 --2- 172.21.15.59:0/1132770516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634074590 0x7f5634074970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f562802a480 comp rx=0 tx=0).stop 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.823+0000 7f563c600640 1 -- 172.21.15.59:0/1132770516 shutdown_connections 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.823+0000 7f563c600640 1 --2- 172.21.15.59:0/1132770516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634112b90 0x7f5634114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.824+0000 7f563c600640 1 --2- 172.21.15.59:0/1132770516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 0x7f5634112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.824+0000 7f563c600640 1 --2- 172.21.15.59:0/1132770516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634074590 0x7f5634074970 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.824+0000 7f563c600640 1 -- 172.21.15.59:0/1132770516 >> 172.21.15.59:0/1132770516 conn(0x7f56340fd8c0 msgr2=0x7f56340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.824+0000 7f563c600640 1 -- 172.21.15.59:0/1132770516 shutdown_connections 2024-09-05T11:56:17.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.824+0000 7f563c600640 1 -- 172.21.15.59:0/1132770516 wait complete. 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 Processor -- start 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 -- start start 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 0x7f56341a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634112b90 0x7f563419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5634118550 con 0x7f5634112b90 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56341183d0 con 0x7f5634074590 2024-09-05T11:56:17.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563c600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56341186d0 con 0x7f5634074eb0 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f5639c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f563a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634112b90 0x7f563419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f5639c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36418/0 (socket says 172.21.15.59:36418) 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f5639c00640 1 -- 172.21.15.59:0/2965963137 learned_addr learned my addr 172.21.15.59:0/2965963137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.825+0000 7f5639200640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 0x7f56341a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 -- 172.21.15.59:0/2965963137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 msgr2=0x7f56341a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 0x7f56341a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 -- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634112b90 msgr2=0x7f563419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634112b90 0x7f563419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 -- 172.21.15.59:0/2965963137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563419efd0 con 0x7f5634074590 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5639c00640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5628006010 tx=0x7f5628004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:17.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562803f030 con 0x7f5634074590 2024-09-05T11:56:17.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563419f260 con 0x7f5634074590 2024-09-05T11:56:17.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628004440 con 0x7f5634074590 2024-09-05T11:56:17.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.826+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628035070 con 0x7f5634074590 2024-09-05T11:56:17.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.827+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341ab5b0 con 0x7f5634074590 2024-09-05T11:56:17.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.828+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634074970 con 0x7f5634074590 2024-09-05T11:56:17.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.830+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5628042030 con 0x7f5634074590 2024-09-05T11:56:17.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.831+0000 7f5632000640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 0x7f561c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:17.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.831+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f56280efae0 con 0x7f5634074590 2024-09-05T11:56:17.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.831+0000 7f5639200640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 0x7f561c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:17.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.832+0000 7f5639200640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 0x7f561c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56341000e0 tx=0x7f5620005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:17.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:17.833+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56280bbf30 con 0x7f5634074590 2024-09-05T11:56:17.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:17 smithi189 ceph-mon[34772]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:17.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:17 smithi089 ceph-mon[34499]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:56:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.057+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5634104fd0 con 0x7f5634074590 2024-09-05T11:56:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.057+0000 7f5632000640 1 -- 172.21.15.59:0/2965963137 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f56280c0230 con 0x7f5634074590 2024-09-05T11:56:18.059 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:18.059 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 msgr2=0x7f561c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 0x7f561c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56341000e0 tx=0x7f5620005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 msgr2=0x7f56341a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5628006010 tx=0x7f5628004040 comp rx=0 tx=0).stop 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 shutdown_connections 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f561c0a2c10 0x7f561c0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5634112b90 0x7f563419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634074eb0 0x7f56341a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 --2- 172.21.15.59:0/2965963137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634074590 0x7f56341a46e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.060+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 >> 172.21.15.59:0/2965963137 conn(0x7f56340fd8c0 msgr2=0x7f56340ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.061+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 shutdown_connections 2024-09-05T11:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:18.061+0000 7f563c600640 1 -- 172.21.15.59:0/2965963137 wait complete. 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-05T11:56:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:18 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2965963137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T11:56:18.799 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-05T11:56:18.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-05T11:56:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:18 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2965963137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T11:56:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-05T11:56:18.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:18 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2965963137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T11:56:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 -- 172.21.15.59:0/2452945025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f01047e0 msgr2=0x7f23f0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 --2- 172.21.15.59:0/2452945025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f01047e0 0x7f23f0108830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009900 tx=0x7f23d802a410 comp rx=0 tx=0).stop 2024-09-05T11:56:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 -- 172.21.15.59:0/2452945025 shutdown_connections 2024-09-05T11:56:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 --2- 172.21.15.59:0/2452945025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f0108f60 0x7f23f0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 --2- 172.21.15.59:0/2452945025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f01047e0 0x7f23f0108830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 --2- 172.21.15.59:0/2452945025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 0x7f23f0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 -- 172.21.15.59:0/2452945025 >> 172.21.15.59:0/2452945025 conn(0x7f23f00ff8d0 msgr2=0x7f23f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 -- 172.21.15.59:0/2452945025 shutdown_connections 2024-09-05T11:56:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.724+0000 7f23ef200640 1 -- 172.21.15.59:0/2452945025 wait complete. 2024-09-05T11:56:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.725+0000 7f23ef200640 1 Processor -- start 2024-09-05T11:56:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.725+0000 7f23ef200640 1 -- start start 2024-09-05T11:56:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.725+0000 7f23ef200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 0x7f23f019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ef200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ef200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 0x7f23f019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ef200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0114010 con 0x7f23f0108f60 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ef200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0113e90 con 0x7f23f0103e30 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ef200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0114190 con 0x7f23f01047e0 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ee800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 0x7f23f019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ede00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 0x7f23f019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46546/0 (socket says 172.21.15.59:46546) 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 -- 172.21.15.59:0/1305131056 learned_addr learned my addr 172.21.15.59:0/1305131056 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 -- 172.21.15.59:0/1305131056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 msgr2=0x7f23f019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 0x7f23f019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 -- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 msgr2=0x7f23f019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 0x7f23f019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23e7e00640 1 -- 172.21.15.59:0/1305131056 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f019f420 con 0x7f23f01047e0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.726+0000 7f23ee800640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 0x7f23f019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23d8004770 tx=0x7f23d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d803f070 con 0x7f23f01047e0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f019f6b0 con 0x7f23f01047e0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d8031ca0 con 0x7f23f01047e0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a6ff0 con 0x7f23f01047e0 2024-09-05T11:56:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.727+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8049060 con 0x7f23f01047e0 2024-09-05T11:56:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.728+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f0104210 con 0x7f23f01047e0 2024-09-05T11:56:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.731+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23d8042070 con 0x7f23f01047e0 2024-09-05T11:56:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.732+0000 7f23e6a00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 0x7f23d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.733+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f23d80eff10 con 0x7f23f01047e0 2024-09-05T11:56:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.733+0000 7f23ede00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 0x7f23d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.733+0000 7f23ede00640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 0x7f23d40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23e0005d00 tx=0x7f23e000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.734+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23d80bc250 con 0x7f23f01047e0 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-05T11:56:19.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:19 smithi189 ceph-mon[34772]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T11:56:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T11:56:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-05T11:56:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-05T11:56:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-05T11:56:19.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-05T11:56:19.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-05T11:56:19.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:19 smithi089 ceph-mon[34499]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T11:56:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.956+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f010c170 con 0x7f23f01047e0 2024-09-05T11:56:19.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-05T11:56:19.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:19 smithi059 ceph-mon[30025]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T11:56:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.957+0000 7f23e6a00640 1 -- 172.21.15.59:0/1305131056 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f23d80c0550 con 0x7f23f01047e0 2024-09-05T11:56:19.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:19.959 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 msgr2=0x7f23d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 0x7f23d40a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23e0005d00 tx=0x7f23e000b410 comp rx=0 tx=0).stop 2024-09-05T11:56:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 msgr2=0x7f23f019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23d8004770 tx=0x7f23d8003c10 comp rx=0 tx=0).stop 2024-09-05T11:56:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 shutdown_connections 2024-09-05T11:56:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23d40a2c10 0x7f23d40a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23f0108f60 0x7f23f019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f01047e0 0x7f23f019a9b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 --2- 172.21.15.59:0/1305131056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f0103e30 0x7f23f019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.959+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 >> 172.21.15.59:0/1305131056 conn(0x7f23f00ff8d0 msgr2=0x7f23f0100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.960+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 shutdown_connections 2024-09-05T11:56:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:19.960+0000 7f23ef200640 1 -- 172.21.15.59:0/1305131056 wait complete. 2024-09-05T11:56:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:20 smithi059 ceph-mon[30025]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:56:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:20 smithi059 ceph-mon[30025]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T11:56:20.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:20 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1305131056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:20.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:20.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:20 smithi189 ceph-mon[34772]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:56:20.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:20 smithi189 ceph-mon[34772]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T11:56:20.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:20 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1305131056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:20 smithi089 ceph-mon[34499]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:56:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:20 smithi089 ceph-mon[34499]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T11:56:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:20 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1305131056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 -- 172.21.15.59:0/548661862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054112a50 msgr2=0x7f7054114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 --2- 172.21.15.59:0/548661862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054112a50 0x7f7054114e40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f704c009be0 tx=0x7f704c02a6c0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 -- 172.21.15.59:0/548661862 shutdown_connections 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 --2- 172.21.15.59:0/548661862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054112a50 0x7f7054114e40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 --2- 172.21.15.59:0/548661862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f7054112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 --2- 172.21.15.59:0/548661862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054076720 0x7f7054076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 -- 172.21.15.59:0/548661862 >> 172.21.15.59:0/548661862 conn(0x7f70540ffa50 msgr2=0x7f7054101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 -- 172.21.15.59:0/548661862 shutdown_connections 2024-09-05T11:56:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.716+0000 7f705b200640 1 -- 172.21.15.59:0/548661862 wait complete. 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 Processor -- start 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 -- start start 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 0x7f70541a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054112a50 0x7f705419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70541185a0 con 0x7f7054076720 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7054118420 con 0x7f7054077040 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.717+0000 7f705b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7054118720 con 0x7f7054112a50 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f7053e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 0x7f70541a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:21.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59674/0 (socket says 172.21.15.59:59674) 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 -- 172.21.15.59:0/2651305330 learned_addr learned my addr 172.21.15.59:0/2651305330 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f7053e00640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 0x7f70541a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59874/0 (socket says 172.21.15.59:59874) 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f7059200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054112a50 0x7f705419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 -- 172.21.15.59:0/2651305330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054112a50 msgr2=0x7f705419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054112a50 0x7f705419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 -- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 msgr2=0x7f70541a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 0x7f70541a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.718+0000 7f704b400640 1 -- 172.21.15.59:0/2651305330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705419efc0 con 0x7f7054077040 2024-09-05T11:56:21.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f704b400640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e9c0 tx=0x7f703c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c00cde0 con 0x7f7054077040 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705419f2b0 con 0x7f7054077040 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c0035f0 con 0x7f7054077040 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010650 con 0x7f7054077040 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.719+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541ab5a0 con 0x7f7054077040 2024-09-05T11:56:21.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.720+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7054076b00 con 0x7f7054077040 2024-09-05T11:56:21.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.722+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f703c0037a0 con 0x7f7054077040 2024-09-05T11:56:21.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.723+0000 7f7051600640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 0x7f70380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:21.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.723+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f703c0cf8e0 con 0x7f7054077040 2024-09-05T11:56:21.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.724+0000 7f7053e00640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 0x7f70380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:21.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.724+0000 7f7053e00640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 0x7f70380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f705419f9d0 tx=0x7f7044005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:21.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.728+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f703c09c270 con 0x7f7054077040 2024-09-05T11:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.948+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70541ab8c0 con 0x7f7054077040 2024-09-05T11:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.949+0000 7f7051600640 1 -- 172.21.15.59:0/2651305330 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f703c09bb70 con 0x7f7054077040 2024-09-05T11:56:21.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:21.951 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 msgr2=0x7f70380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 0x7f70380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f705419f9d0 tx=0x7f7044005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 msgr2=0x7f70541a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e9c0 tx=0x7f703c00ee80 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 shutdown_connections 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70380a2c10 0x7f70380a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054112a50 0x7f705419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7054077040 0x7f70541a4c10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 --2- 172.21.15.59:0/2651305330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7054076720 0x7f70541a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 >> 172.21.15.59:0/2651305330 conn(0x7f70540ffa50 msgr2=0x7f7054101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:21.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 shutdown_connections 2024-09-05T11:56:21.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:21.952+0000 7f705b200640 1 -- 172.21.15.59:0/2651305330 wait complete. 2024-09-05T11:56:22.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:22 smithi059 ceph-mon[30025]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:22.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:22 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2651305330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:22.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:22 smithi189 ceph-mon[34772]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:22.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:22 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2651305330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:22.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:56:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:22 smithi089 ceph-mon[34499]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:22 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2651305330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:23.304 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:23.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 -- 172.21.15.59:0/1124773960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838102360 msgr2=0x7fa83810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:23.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1124773960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838102360 0x7fa83810e890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa830009c10 tx=0x7fa83002a6f0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 -- 172.21.15.59:0/1124773960 shutdown_connections 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1124773960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838102360 0x7fa83810e890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1124773960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381019c0 0x7fa838101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1124773960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8381079c0 0x7fa838107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 -- 172.21.15.59:0/1124773960 >> 172.21.15.59:0/1124773960 conn(0x7fa8380fd740 msgr2=0x7fa8380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.841+0000 7fa83ee00640 1 -- 172.21.15.59:0/1124773960 shutdown_connections 2024-09-05T11:56:23.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.842+0000 7fa83ee00640 1 -- 172.21.15.59:0/1124773960 wait complete. 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.842+0000 7fa83ee00640 1 Processor -- start 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.842+0000 7fa83ee00640 1 -- start start 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8381019c0 0x7fa83819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381079c0 0x7fa83819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:23.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa838111dc0 con 0x7fa838102360 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa838111c40 con 0x7fa8381079c0 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa838111f40 con 0x7fa8381019c0 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59898/0 (socket says 172.21.15.59:59898) 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83d000640 1 -- 172.21.15.59:0/1983425365 learned_addr learned my addr 172.21.15.59:0/1983425365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83da00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8381019c0 0x7fa83819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.843+0000 7fa83e400640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381079c0 0x7fa83819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 -- 172.21.15.59:0/1983425365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8381019c0 msgr2=0x7fa83819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8381019c0 0x7fa83819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 -- 172.21.15.59:0/1983425365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381079c0 msgr2=0x7fa83819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381079c0 0x7fa83819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 -- 172.21.15.59:0/1983425365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83819f410 con 0x7fa838102360 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83d000640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa82400e9f0 tx=0x7fa82400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82400cd80 con 0x7fa838102360 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83819f700 con 0x7fa838102360 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824005500 con 0x7fa838102360 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824010690 con 0x7fa838102360 2024-09-05T11:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.844+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a6fe0 con 0x7fa838102360 2024-09-05T11:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.846+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838101e20 con 0x7fa838102360 2024-09-05T11:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.848+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8240046c0 con 0x7fa838102360 2024-09-05T11:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.849+0000 7fa836000640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 0x7fa8200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.849+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa8240cfc50 con 0x7fa838102360 2024-09-05T11:56:23.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.849+0000 7fa83da00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 0x7fa8200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:23.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.850+0000 7fa83da00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 0x7fa8200a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0096c0 tx=0x7fa82c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:23.852+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8240541b0 con 0x7fa838102360 2024-09-05T11:56:24.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.038+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa83819b7c0 con 0x7fa838102360 2024-09-05T11:56:24.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.040+0000 7fa836000640 1 -- 172.21.15.59:0/1983425365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7fa82409c5e0 con 0x7fa838102360 2024-09-05T11:56:24.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:24.043 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 msgr2=0x7fa8200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 0x7fa8200a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0096c0 tx=0x7fa82c009260 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 msgr2=0x7fa83819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa82400e9f0 tx=0x7fa82400edf0 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 shutdown_connections 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8200a2c10 0x7fa8200a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8381079c0 0x7fa83819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa838102360 0x7fa83819a9a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 --2- 172.21.15.59:0/1983425365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8381019c0 0x7fa83819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 >> 172.21.15.59:0/1983425365 conn(0x7fa8380fd740 msgr2=0x7fa83810cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.043+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 shutdown_connections 2024-09-05T11:56:24.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:24.044+0000 7fa83ee00640 1 -- 172.21.15.59:0/1983425365 wait complete. 2024-09-05T11:56:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:24 smithi059 ceph-mon[30025]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:24.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:24 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1983425365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:24.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:24 smithi189 ceph-mon[34772]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:24.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:24 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1983425365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:24.876 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:24 smithi089 ceph-mon[34499]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-05T11:56:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:24 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1983425365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:25.252 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 -- 172.21.15.59:0/1745176775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94076720 msgr2=0x7f7c94076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 --2- 172.21.15.59:0/1745176775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94076720 0x7f7c94076b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009900 tx=0x7f7c8402a410 comp rx=0 tx=0).stop 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 -- 172.21.15.59:0/1745176775 shutdown_connections 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 --2- 172.21.15.59:0/1745176775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94112a30 0x7f7c94114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 --2- 172.21.15.59:0/1745176775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 0x7f7c941124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 --2- 172.21.15.59:0/1745176775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94076720 0x7f7c94076b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 -- 172.21.15.59:0/1745176775 >> 172.21.15.59:0/1745176775 conn(0x7f7c940ffa90 msgr2=0x7f7c94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 -- 172.21.15.59:0/1745176775 shutdown_connections 2024-09-05T11:56:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.822+0000 7f7c99a00640 1 -- 172.21.15.59:0/1745176775 wait complete. 2024-09-05T11:56:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.823+0000 7f7c99a00640 1 Processor -- start 2024-09-05T11:56:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 -- start start 2024-09-05T11:56:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94076720 0x7f7c941a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 0x7f7c941a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94118610 con 0x7f7c94112a30 2024-09-05T11:56:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94118490 con 0x7f7c94076720 2024-09-05T11:56:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c99a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94118790 con 0x7f7c94077040 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c93400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c93400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59922/0 (socket says 172.21.15.59:59922) 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c93400640 1 -- 172.21.15.59:0/3303462973 learned_addr learned my addr 172.21.15.59:0/3303462973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.824+0000 7f7c92a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94076720 0x7f7c941a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c92000640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 0x7f7c941a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 -- 172.21.15.59:0/3303462973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 msgr2=0x7f7c941a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 0x7f7c941a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 -- 172.21.15.59:0/3303462973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94076720 msgr2=0x7f7c941a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94076720 0x7f7c941a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 -- 172.21.15.59:0/3303462973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9419f010 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.825+0000 7f7c93400640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00dac0 tx=0x7f7c8c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.826+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c017030 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.826+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9419f300 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.826+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c003e50 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.826+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c010460 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.826+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941ab5f0 con 0x7f7c94112a30 2024-09-05T11:56:25.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.827+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c94076b00 con 0x7f7c94112a30 2024-09-05T11:56:25.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.829+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c8c01a070 con 0x7f7c94112a30 2024-09-05T11:56:25.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.830+0000 7f7c8b400640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 0x7f7c780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:25.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.830+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7c8c0cf280 con 0x7f7c94112a30 2024-09-05T11:56:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.830+0000 7f7c92a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 0x7f7c780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.831+0000 7f7c92a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 0x7f7c780a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006010 tx=0x7f7c84000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:25.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:25.834+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c8c09bc10 con 0x7f7c94112a30 2024-09-05T11:56:26.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.061+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c9402d7c0 con 0x7f7c94112a30 2024-09-05T11:56:26.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.062+0000 7f7c8b400640 1 -- 172.21.15.59:0/3303462973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f7c8c09b510 con 0x7f7c94112a30 2024-09-05T11:56:26.064 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:26.064 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:26.064 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:26.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 msgr2=0x7f7c780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:26.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 0x7f7c780a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006010 tx=0x7f7c84000bc0 comp rx=0 tx=0).stop 2024-09-05T11:56:26.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 msgr2=0x7f7c9419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00dac0 tx=0x7f7c8c00df80 comp rx=0 tx=0).stop 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 shutdown_connections 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c780a2c90 0x7f7c780a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c94112a30 0x7f7c9419e8e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c94077040 0x7f7c941a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 --2- 172.21.15.59:0/3303462973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c94076720 0x7f7c941a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:26.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 >> 172.21.15.59:0/3303462973 conn(0x7f7c940ffa90 msgr2=0x7f7c94101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 shutdown_connections 2024-09-05T11:56:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:26.065+0000 7f7c99a00640 1 -- 172.21.15.59:0/3303462973 wait complete. 2024-09-05T11:56:26.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:26 smithi059 ceph-mon[30025]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:26.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:26 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3303462973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:26.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T11:54:32.492433+0000', 'modified': '2024-09-05T11:55:14.725014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/2748476304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 2748476304}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 2748476304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4050620246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4050620246}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4050620246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24545]} 2024-09-05T11:56:26.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T11:56:26.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T11:56:26.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:26 smithi189 ceph-mon[34772]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:26.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:26 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3303462973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:26 smithi089 ceph-mon[34499]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:26 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3303462973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:27.197 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.745+0000 7fa22fe00640 1 -- 172.21.15.59:0/1883839391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 msgr2=0x7fa230104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.745+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1883839391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa230104210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a410 comp rx=0 tx=0).stop 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 -- 172.21.15.59:0/1883839391 shutdown_connections 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1883839391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa230108f60 0x7fa230110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1883839391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2301047e0 0x7fa230108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1883839391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa230104210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 -- 172.21.15.59:0/1883839391 >> 172.21.15.59:0/1883839391 conn(0x7fa2300ff910 msgr2=0x7fa230101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 -- 172.21.15.59:0/1883839391 shutdown_connections 2024-09-05T11:56:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.746+0000 7fa22fe00640 1 -- 172.21.15.59:0/1883839391 wait complete. 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 Processor -- start 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 -- start start 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 0x7fa23019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa230113fd0 con 0x7fa230103e30 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa230113e50 con 0x7fa230108f60 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.747+0000 7fa22fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa230114150 con 0x7fa2301047e0 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59926/0 (socket says 172.21.15.59:59926) 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 0x7fa23019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22ea00640 1 -- 172.21.15.59:0/1526888630 learned_addr learned my addr 172.21.15.59:0/1526888630 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59718/0 (socket says 172.21.15.59:59718) 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 -- 172.21.15.59:0/1526888630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 msgr2=0x7fa23019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 0x7fa23019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 -- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 msgr2=0x7fa23019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22f400640 1 -- 172.21.15.59:0/1526888630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23019f4d0 con 0x7fa230108f60 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22e000640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 0x7fa23019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.748+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa22f400640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22800da10 tx=0x7fa22800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa227400640 1 -- 172.21.15.59:0/1526888630 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800bd70 con 0x7fa230108f60 2024-09-05T11:56:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23019f7c0 con 0x7fa230108f60 2024-09-05T11:56:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa227400640 1 -- 172.21.15.59:0/1526888630 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa228003e50 con 0x7fa230108f60 2024-09-05T11:56:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa227400640 1 -- 172.21.15.59:0/1526888630 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280106a0 con 0x7fa230108f60 2024-09-05T11:56:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.749+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a70a0 con 0x7fa230108f60 2024-09-05T11:56:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.753+0000 7fa2385c7740 1 Processor -- start 2024-09-05T11:56:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.753+0000 7fa227400640 1 -- 172.21.15.59:0/1526888630 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa228003970 con 0x7fa230108f60 2024-09-05T11:56:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.753+0000 7fa2385c7740 1 -- start start 2024-09-05T11:56:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 0x559bb12278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb1232180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb1234ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb118d0b0 con 0x559bb12326c0 2024-09-05T11:56:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559bb1134ab0 con 0x559bb1227e00 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa2385c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bb1189070 con 0x559bb12274e0 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa22e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb1234ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa22e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb1234ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59928/0 (socket says 172.21.15.59:59928) 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa22e000640 1 -- 172.21.15.59:0/1801024515 learned_addr learned my addr 172.21.15.59:0/1801024515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb1232180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.754+0000 7fa22e000640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 0x559bb12278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 -- 172.21.15.59:0/1801024515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 msgr2=0x559bb12278c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 0x559bb12278c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 -- 172.21.15.59:0/1801024515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 msgr2=0x559bb1234ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb1234ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 -- 172.21.15.59:0/1801024515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb11414a0 con 0x559bb1227e00 2024-09-05T11:56:27.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22e000640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 0x559bb12278c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb1232180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa220031550 tx=0x7fa220004470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959f9c5a6d27e6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa227400640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 0x7fa2140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa227400640 1 -- 172.21.15.59:0/1526888630 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa2280cf980 con 0x7fa230108f60 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.755+0000 7fa22e000640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 0x7fa2140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa21be00640 1 -- 172.21.15.59:0/1801024515 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220040070 con 0x559bb1227e00 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa21be00640 1 -- 172.21.15.59:0/1801024515 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2200047a0 con 0x559bb1227e00 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa21be00640 1 -- 172.21.15.59:0/1801024515 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035040 con 0x559bb1227e00 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 -- 172.21.15.59:0/1801024515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 msgr2=0x559bb1232180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa22e000640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 0x7fa2140a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x559bb1224df0 tx=0x7fa21c00bb70 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb1232180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa220031550 tx=0x7fa220004470 comp rx=0 tx=0).stop 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 -- 172.21.15.59:0/1801024515 shutdown_connections 2024-09-05T11:56:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb1234ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb1232180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1801024515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12274e0 0x559bb12278c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.756+0000 7fa2385c7740 1 -- 172.21.15.59:0/1801024515 >> 172.21.15.59:0/1801024515 conn(0x559bb1223a70 msgr2=0x559bb1223e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.757+0000 7fa2385c7740 1 -- 172.21.15.59:0/1801024515 shutdown_connections 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.757+0000 7fa2385c7740 1 -- 172.21.15.59:0/1801024515 wait complete. 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.757+0000 7fa2385c7740 1 -- start start 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.757+0000 7fa2385c7740 1 Processor -- start 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb122f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb12352d0 con 0x559bb12274e0 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559bb1115920 con 0x559bb12326c0 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bb1235450 con 0x559bb1227e00 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa2385c7740 10 client.0 _ll_register_callbacks cb 0x559bb11398a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb122f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59736/0 (socket says 172.21.15.59:59736) 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb122f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59936/0 (socket says 172.21.15.59:59936) 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22f400640 1 -- 172.21.15.59:0/1344474895 learned_addr learned my addr 172.21.15.59:0/1344474895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.758+0000 7fa22e000640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 -- 172.21.15.59:0/1344474895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 msgr2=0x559bb122fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 -- 172.21.15.59:0/1344474895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 msgr2=0x559bb122f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb122f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 -- 172.21.15.59:0/1344474895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb0efe2b0 con 0x559bb12326c0 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa22f400640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2280a4350 tx=0x7fa22809c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e2d227031b71ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa218c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:56:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a4880 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa2280042e0 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.759+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2200047a0 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa228003c60 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa2385c7740 10 client.24980 fetch_fsmap learned FSMap version 21 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559bb1241010 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2200047a0 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa228024060 con 0x559bb12326c0 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa2385c7740 10 client.24980 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa2385c7740 10 client.24980 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.760+0000 7fa2385c7740 20 client.24980 populate_metadata read hostname 'smithi059' 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.761+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12436c0 0x559bb1244b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.761+0000 7fa2385c7740 4 client.24980 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:56:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.761+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bb119a620 con 0x559bb12436c0 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.761+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12436c0 0x559bb1244b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.762+0000 7fa22ea00640 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12436c0 0x559bb1244b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d122dc954b9040 server_cookie=89c03c16b0533ad8 in_seq=0 out_seq=0 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.762+0000 7fa218c00640 10 client.24980 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.762+0000 7fa218c00640 1 -- 172.21.15.59:0/1344474895 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bb119a620 con 0x559bb12436c0 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.762+0000 7fa218c00640 10 client.24980 handle_command_reply: tid=0 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.762+0000 7fa2385c7740 1 client.24980 shutdown 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 msgr2=0x559bb1230020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2280a4350 tx=0x7fa22809c3d0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 shutdown_connections 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12326c0 0x559bb1230020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12436c0 0x559bb1244b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 --2- 172.21.15.59:0/1344474895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb122f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa218c00640 0 client.24980 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 >> 172.21.15.59:0/1344474895 conn(0x559bb1223a70 msgr2=0x559bb1229750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 shutdown_connections 2024-09-05T11:56:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 1 -- 172.21.15.59:0/1344474895 wait complete. 2024-09-05T11:56:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.763+0000 7fa2385c7740 20 client.24980 trim_cache size 0 max 16384 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 Processor -- start 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 -- start start 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 0x559bb122de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb122c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb1242e70 con 0x559bb12326c0 2024-09-05T11:56:27.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559bb1243540 con 0x559bb12274e0 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.782+0000 7fa2385c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bb1243c10 con 0x559bb1227e00 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb122c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 0x559bb122de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb122c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59950/0 (socket says 172.21.15.59:59950) 2024-09-05T11:56:27.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22f400640 1 -- 172.21.15.59:0/2046106325 learned_addr learned my addr 172.21.15.59:0/2046106325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:27.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 -- 172.21.15.59:0/2046106325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 msgr2=0x559bb122e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 -- 172.21.15.59:0/2046106325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 msgr2=0x559bb122c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb122c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 -- 172.21.15.59:0/2046106325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb123ecf0 con 0x559bb12274e0 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.783+0000 7fa22ea00640 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 0x559bb122de70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa220005c40 tx=0x7fa220047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de478fb8df44d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa21b400640 1 -- 172.21.15.59:0/2046106325 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220040040 con 0x559bb12274e0 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa21b400640 1 -- 172.21.15.59:0/2046106325 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa220003d90 con 0x559bb12274e0 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa21b400640 1 -- 172.21.15.59:0/2046106325 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035040 con 0x559bb12274e0 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 -- 172.21.15.59:0/2046106325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 msgr2=0x559bb122de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 0x559bb122de70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa220005c40 tx=0x7fa220047040 comp rx=0 tx=0).stop 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 -- 172.21.15.59:0/2046106325 shutdown_connections 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12326c0 0x559bb122c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb1227e00 0x559bb122e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 --2- 172.21.15.59:0/2046106325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb12274e0 0x559bb122de70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 -- 172.21.15.59:0/2046106325 >> 172.21.15.59:0/2046106325 conn(0x559bb1223a70 msgr2=0x559bb1223e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 -- 172.21.15.59:0/2046106325 shutdown_connections 2024-09-05T11:56:27.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.784+0000 7fa2385c7740 1 -- 172.21.15.59:0/2046106325 wait complete. 2024-09-05T11:56:27.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.785+0000 7fa2385c7740 1 -- start start 2024-09-05T11:56:27.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.785+0000 7fa2385c7740 1 Processor -- start 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.785+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb124d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12326c0 0x559bb1246d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb12449b0 con 0x559bb12274e0 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559bb12442e0 con 0x559bb1227e00 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bb1245080 con 0x559bb12326c0 2024-09-05T11:56:27.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa2385c7740 10 client.0 _ll_register_callbacks cb 0x7fa22809b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59958/0 (socket says 172.21.15.59:59958) 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22ea00640 1 -- 172.21.15.59:0/839380814 learned_addr learned my addr 172.21.15.59:0/839380814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb124d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22f400640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12326c0 0x559bb1246d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 -- 172.21.15.59:0/839380814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12326c0 msgr2=0x559bb1246d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12326c0 0x559bb1246d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 -- 172.21.15.59:0/839380814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 msgr2=0x559bb124d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22e000640 1 -- 172.21.15.59:0/839380814 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb12475c0 con 0x559bb1227e00 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.786+0000 7fa22ea00640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:27.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa22e000640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb124d590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00be70 tx=0x7fa21c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6151e22b88694f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa218c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c020a20 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa220003d90 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa21c020bc0 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa21c038050 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa2385c7740 10 client.24986 fetch_fsmap learned FSMap version 21 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559bb124ca20 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.787+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa220003d90 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa21c02a3f0 con 0x559bb1227e00 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 10 client.24986 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 10 client.24986 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 20 client.24986 populate_metadata read hostname 'smithi059' 2024-09-05T11:56:27.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12b25a0 0x559bb12b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 4 client.24986 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "status"}) -- 0x559bb1194b30 con 0x559bb12b25a0 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.788+0000 7fa22ea00640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12b25a0 0x559bb12b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa22ea00640 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12b25a0 0x559bb12b12b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fc1e87ea80d1e0 server_cookie=cb0041f1cfd50506 in_seq=0 out_seq=0 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa218c00640 10 client.24986 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa218c00640 1 -- 172.21.15.59:0/839380814 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559bb1194b30 con 0x559bb12b25a0 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa218c00640 10 client.24986 handle_command_reply: tid=0 2024-09-05T11:56:27.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 client.24986 shutdown 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 msgr2=0x559bb124d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb124d590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00be70 tx=0x7fa21c004730 comp rx=0 tx=0).stop 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 shutdown_connections 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bb12326c0 0x559bb1246d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559bb1227e00 0x559bb124d590 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559bb12b25a0 0x559bb12b12b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 --2- 172.21.15.59:0/839380814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb12274e0 0x559bb124d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 >> 172.21.15.59:0/839380814 conn(0x559bb1223a70 msgr2=0x559bb123d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa218c00640 0 client.24986 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 shutdown_connections 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 1 -- 172.21.15.59:0/839380814 wait complete. 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.789+0000 7fa2385c7740 20 client.24986 trim_cache size 0 max 16384 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "53d560d4-6b7c-11ef-bcd6-c7b262605968", 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-09-05T11:56:27.795 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24545, 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 115.24310979400001, 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 21, 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 93, 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 89, 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 127.93768596 2024-09-05T11:56:27.796 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 msgr2=0x7fa2140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 0x7fa2140a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x559bb1224df0 tx=0x7fa21c00bb70 comp rx=0 tx=0).stop 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 msgr2=0x7fa23019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa22800da10 tx=0x7fa22800ded0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 shutdown_connections 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa2140a2c10 0x7fa2140a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa230108f60 0x7fa23019edf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2301047e0 0x7fa23019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 --2- 172.21.15.59:0/1526888630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa230103e30 0x7fa23019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:27.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 >> 172.21.15.59:0/1526888630 conn(0x7fa2300ff910 msgr2=0x7fa230100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:27.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 shutdown_connections 2024-09-05T11:56:27.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:27.790+0000 7fa22fe00640 1 -- 172.21.15.59:0/1526888630 wait complete. 2024-09-05T11:56:28.577 INFO:tasks.cephadm:Setup complete, yielding 2024-09-05T11:56:28.577 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T11:56:28.587 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T11:56:28.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch status' 2024-09-05T11:56:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:28 smithi089 ceph-mon[34499]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:28.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:28 smithi059 ceph-mon[30025]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:28.965 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:28 smithi189 ceph-mon[34772]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 -- 172.21.15.59:0/3945538395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8105810 msgr2=0x7fdac810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 --2- 172.21.15.59:0/3945538395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8105810 0x7fdac810c0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009ca0 tx=0x7fdac002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 -- 172.21.15.59:0/3945538395 shutdown_connections 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 --2- 172.21.15.59:0/3945538395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8105810 0x7fdac810c0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 --2- 172.21.15.59:0/3945538395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 0x7fdac81052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 --2- 172.21.15.59:0/3945538395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8103c50 0x7fdac8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 -- 172.21.15.59:0/3945538395 >> 172.21.15.59:0/3945538395 conn(0x7fdac80ff3e0 msgr2=0x7fdac8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 -- 172.21.15.59:0/3945538395 shutdown_connections 2024-09-05T11:56:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.513+0000 7fdacce00640 1 -- 172.21.15.59:0/3945538395 wait complete. 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 Processor -- start 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 -- start start 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 0x7fdac819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8105810 0x7fdac81a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810f8b0 con 0x7fdac8104e50 2024-09-05T11:56:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810f730 con 0x7fdac8103c50 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.514+0000 7fdacce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810fa30 con 0x7fdac8105810 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 0x7fdac819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59776/0 (socket says 172.21.15.59:59776) 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 -- 172.21.15.59:0/256807713 learned_addr learned my addr 172.21.15.59:0/256807713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7e00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8105810 0x7fdac81a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 -- 172.21.15.59:0/256807713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8105810 msgr2=0x7fdac81a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8105810 0x7fdac81a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 -- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 msgr2=0x7fdac819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 0x7fdac819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 -- 172.21.15.59:0/256807713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81a1ad0 con 0x7fdac8103c50 2024-09-05T11:56:29.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.515+0000 7fdac7400640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.516+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017020 con 0x7fdac8103c50 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.516+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab8009c30 con 0x7fdac8103c50 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.516+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab800f720 con 0x7fdac8103c50 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.516+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81a1dc0 con 0x7fdac8103c50 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.516+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a2270 con 0x7fdac8103c50 2024-09-05T11:56:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.517+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac810b020 con 0x7fdac8103c50 2024-09-05T11:56:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.519+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdab801a020 con 0x7fdac8103c50 2024-09-05T11:56:29.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.520+0000 7fdabfe00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 0x7fdaac0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.520+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdab80cf660 con 0x7fdac8103c50 2024-09-05T11:56:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.520+0000 7fdac6a00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 0x7fdaac0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.521+0000 7fdac6a00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 0x7fdaac0a5180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdac819b3c0 tx=0x7fdab400b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.522+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdab8021020 con 0x7fdac8103c50 2024-09-05T11:56:29.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.724+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdac8063020 con 0x7fdaac0a2cc0 2024-09-05T11:56:29.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.725+0000 7fdabfe00640 1 -- 172.21.15.59:0/256807713 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdac8063020 con 0x7fdaac0a2cc0 2024-09-05T11:56:29.727 INFO:teuthology.orchestra.run.smithi059.stdout:Backend: cephadm 2024-09-05T11:56:29.727 INFO:teuthology.orchestra.run.smithi059.stdout:Available: Yes 2024-09-05T11:56:29.727 INFO:teuthology.orchestra.run.smithi059.stdout:Paused: No 2024-09-05T11:56:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 msgr2=0x7fdaac0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 0x7fdaac0a5180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdac819b3c0 tx=0x7fdab400b390 comp rx=0 tx=0).stop 2024-09-05T11:56:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 msgr2=0x7fdac8199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).stop 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 shutdown_connections 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdaac0a2cc0 0x7fdaac0a5180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdac8105810 0x7fdac81a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.727+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac8104e50 0x7fdac819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.728+0000 7fdacce00640 1 --2- 172.21.15.59:0/256807713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdac8103c50 0x7fdac8199ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.728+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 >> 172.21.15.59:0/256807713 conn(0x7fdac80ff3e0 msgr2=0x7fdac81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.728+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 shutdown_connections 2024-09-05T11:56:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:29.728+0000 7fdacce00640 1 -- 172.21.15.59:0/256807713 wait complete. 2024-09-05T11:56:30.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-05T11:56:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:30 smithi059 ceph-mon[30025]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:30 smithi059 ceph-mon[30025]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:30 smithi089 ceph-mon[34499]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:30 smithi089 ceph-mon[34499]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:30 smithi189 ceph-mon[34772]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:30 smithi189 ceph-mon[34772]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 -- 172.21.15.59:0/2359738498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778102e00 msgr2=0x7f6778103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 --2- 172.21.15.59:0/2359738498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778102e00 0x7f6778103280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6764009900 tx=0x7f676402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 -- 172.21.15.59:0/2359738498 shutdown_connections 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 --2- 172.21.15.59:0/2359738498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f677810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 --2- 172.21.15.59:0/2359738498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778102e00 0x7f6778103280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 --2- 172.21.15.59:0/2359738498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778101c00 0x7f6778102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.438+0000 7f677da00640 1 -- 172.21.15.59:0/2359738498 >> 172.21.15.59:0/2359738498 conn(0x7f67780fd390 msgr2=0x7f67780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.439+0000 7f677da00640 1 -- 172.21.15.59:0/2359738498 shutdown_connections 2024-09-05T11:56:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.439+0000 7f677da00640 1 -- 172.21.15.59:0/2359738498 wait complete. 2024-09-05T11:56:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.439+0000 7f677da00640 1 Processor -- start 2024-09-05T11:56:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.439+0000 7f677da00640 1 -- start start 2024-09-05T11:56:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 0x7f6778075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 0x7f67780745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f6778074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d880 con 0x7f6778101c00 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d700 con 0x7f67781037c0 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f677da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f677810da00 con 0x7f6778102e00 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f6776000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 0x7f67780745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f6776000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 0x7f67780745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:53210/0 (socket says 172.21.15.59:53210) 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f6776000640 1 -- 172.21.15.59:0/3743566390 learned_addr learned my addr 172.21.15.59:0/3743566390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f6777400640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f6778074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.440+0000 7f6776a00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 0x7f6778075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 -- 172.21.15.59:0/3743566390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 msgr2=0x7f67780745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 0x7f67780745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 -- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 msgr2=0x7f6778075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 0x7f6778075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 -- 172.21.15.59:0/3743566390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67780753e0 con 0x7f67781037c0 2024-09-05T11:56:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6776a00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 0x7f6778075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f6777400640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f6778074b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fd0 tx=0x7f677000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000ce20 con 0x7f67781037c0 2024-09-05T11:56:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770005020 con 0x7f67781037c0 2024-09-05T11:56:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770010720 con 0x7f67781037c0 2024-09-05T11:56:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.441+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781a2090 con 0x7f67781037c0 2024-09-05T11:56:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.442+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781a2550 con 0x7f67781037c0 2024-09-05T11:56:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.443+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6778109000 con 0x7f67781037c0 2024-09-05T11:56:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.444+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6770005540 con 0x7f67781037c0 2024-09-05T11:56:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.446+0000 7f676f400640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 0x7f675c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.446+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f67700cfd50 con 0x7f67781037c0 2024-09-05T11:56:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.446+0000 7f6776a00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 0x7f675c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.447+0000 7f6776a00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 0x7f675c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6768005fd0 tx=0x7f676800b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.449+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f677009c6e0 con 0x7f67781037c0 2024-09-05T11:56:31.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.657+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6778062630 con 0x7f675c0a2c10 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.670+0000 7f676f400640 1 -- 172.21.15.59:0/3743566390 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f6778062630 con 0x7f675c0a2c10 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stdout:mds.a smithi059 running (2m) 61s ago 2m 12.6M - 19.1.1-261-ge856d6b2 1d96423d1332 7acf1b0de9af 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stdout:mds.b smithi089 running (2m) 63s ago 2m 26.4M - 19.1.1-261-ge856d6b2 1d96423d1332 9d21701c70f7 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stdout:mds.c smithi189 running (2m) 63s ago 2m 14.2M - 19.1.1-261-ge856d6b2 1d96423d1332 b69f493c97a9 2024-09-05T11:56:31.672 INFO:teuthology.orchestra.run.smithi059.stdout:mds.d smithi059 running (2m) 61s ago 2m 12.9M - 19.1.1-261-ge856d6b2 1d96423d1332 f8eccdd37f66 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.e smithi089 running (2m) 63s ago 2m 14.4M - 19.1.1-261-ge856d6b2 1d96423d1332 f73f702ee2a3 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.f smithi189 running (2m) 63s ago 2m 13.4M - 19.1.1-261-ge856d6b2 1d96423d1332 0f28ac9dc9b2 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.g smithi059 running (2m) 61s ago 2m 12.8M - 19.1.1-261-ge856d6b2 1d96423d1332 30aa7770c156 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.h smithi089 running (2m) 63s ago 2m 16.6M - 19.1.1-261-ge856d6b2 1d96423d1332 1b01f8ddc938 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.i smithi189 running (2m) 63s ago 2m 15.8M - 19.1.1-261-ge856d6b2 1d96423d1332 4e17eda66df8 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.j smithi059 running (2m) 61s ago 2m 12.7M - 19.1.1-261-ge856d6b2 1d96423d1332 ede2494b26fa 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.k smithi089 running (2m) 63s ago 2m 14.5M - 19.1.1-261-ge856d6b2 1d96423d1332 e7d671bed9dd 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mds.l smithi189 running (2m) 63s ago 2m 15.9M - 19.1.1-261-ge856d6b2 1d96423d1332 7881afaa67bf 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.x smithi059 *:9283,8765 running (10m) 61s ago 9m 507M - 19.1.1-261-ge856d6b2 1d96423d1332 48d3a55566b0 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.y smithi089 *:8443,8765 running (8m) 63s ago 8m 442M - 19.1.1-261-ge856d6b2 1d96423d1332 62c3ab2e75c4 2024-09-05T11:56:31.673 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.z smithi189 *:8443,8765 running (8m) 63s ago 8m 442M - 19.1.1-261-ge856d6b2 1d96423d1332 19ca017281fc 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a smithi059 running (10m) 61s ago 10m 50.0M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 080e238eff3c 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b smithi089 running (8m) 63s ago 8m 40.5M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 576bd771f338 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c smithi189 running (8m) 63s ago 8m 40.8M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 71c5ef535066 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 smithi059 running (7m) 61s ago 7m 43.9M 4273M 19.1.1-261-ge856d6b2 1d96423d1332 0e1779df18c6 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 smithi089 running (7m) 63s ago 7m 49.4M 4271M 19.1.1-261-ge856d6b2 1d96423d1332 e056e7cb8fb0 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 smithi189 running (7m) 63s ago 7m 48.3M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 0f47b40b6384 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 smithi059 running (6m) 61s ago 6m 51.9M 4273M 19.1.1-261-ge856d6b2 1d96423d1332 43d2ff916f99 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 smithi089 running (6m) 63s ago 6m 52.1M 4271M 19.1.1-261-ge856d6b2 1d96423d1332 5f701c526ae6 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 smithi189 running (5m) 63s ago 5m 53.4M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 6874a77f2840 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 smithi059 running (5m) 61s ago 5m 42.8M 4273M 19.1.1-261-ge856d6b2 1d96423d1332 92c7c646493a 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 smithi089 running (4m) 63s ago 4m 44.5M 4271M 19.1.1-261-ge856d6b2 1d96423d1332 f6f02dc77dea 2024-09-05T11:56:31.674 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 smithi189 running (4m) 63s ago 4m 54.4M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 9528e2c16f84 2024-09-05T11:56:31.675 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 smithi059 running (3m) 61s ago 3m 49.5M 4273M 19.1.1-261-ge856d6b2 1d96423d1332 bdd4efd86691 2024-09-05T11:56:31.675 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 smithi089 running (3m) 63s ago 3m 51.0M 4271M 19.1.1-261-ge856d6b2 1d96423d1332 215703ad25d2 2024-09-05T11:56:31.675 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 smithi189 running (3m) 63s ago 3m 41.8M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 ea5578454a13 2024-09-05T11:56:31.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 msgr2=0x7f675c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 0x7f675c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6768005fd0 tx=0x7f676800b390 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 msgr2=0x7f6778074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f6778074b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fd0 tx=0x7f677000ef10 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 shutdown_connections 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f675c0a2c10 0x7f675c0a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67781037c0 0x7f6778074b20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6778102e00 0x7f67780745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 --2- 172.21.15.59:0/3743566390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6778101c00 0x7f6778075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 >> 172.21.15.59:0/3743566390 conn(0x7f67780fd390 msgr2=0x7f67781061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 shutdown_connections 2024-09-05T11:56:31.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:31.673+0000 7f677da00640 1 -- 172.21.15.59:0/3743566390 wait complete. 2024-09-05T11:56:31.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:31 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:56:31.891 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:31 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:56:31.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:31 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:32 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-05T11:56:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:32.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:32 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:32 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.004+0000 7fd0de800640 1 -- 172.21.15.59:0/3327304917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 msgr2=0x7fd0d8078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.004+0000 7fd0de800640 1 --2- 172.21.15.59:0/3327304917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d8078250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009900 tx=0x7fd0c402a410 comp rx=0 tx=0).stop 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 -- 172.21.15.59:0/3327304917 shutdown_connections 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 --2- 172.21.15.59:0/3327304917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8078790 0x7fd0d8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 --2- 172.21.15.59:0/3327304917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d8078250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 --2- 172.21.15.59:0/3327304917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8076ba0 0x7fd0d8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 -- 172.21.15.59:0/3327304917 >> 172.21.15.59:0/3327304917 conn(0x7fd0d80ff3e0 msgr2=0x7fd0d8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 -- 172.21.15.59:0/3327304917 shutdown_connections 2024-09-05T11:56:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.005+0000 7fd0de800640 1 -- 172.21.15.59:0/3327304917 wait complete. 2024-09-05T11:56:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.006+0000 7fd0de800640 1 Processor -- start 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.006+0000 7fd0de800640 1 -- start start 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.006+0000 7fd0de800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.006+0000 7fd0de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.006+0000 7fd0de800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 0x7fd0d81a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0de800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8113d60 con 0x7fd0d8077df0 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0de800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8113be0 con 0x7fd0d8076ba0 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0de800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8113ee0 con 0x7fd0d8078790 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0dd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0dca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0dd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:33016/0 (socket says 172.21.15.59:33016) 2024-09-05T11:56:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0dd400640 1 -- 172.21.15.59:0/568675489 learned_addr learned my addr 172.21.15.59:0/568675489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.007+0000 7fd0dde00640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 0x7fd0d81a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 -- 172.21.15.59:0/568675489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 msgr2=0x7fd0d81a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 0x7fd0d81a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 -- 172.21.15.59:0/568675489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 msgr2=0x7fd0d819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 -- 172.21.15.59:0/568675489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d81a5f40 con 0x7fd0d8077df0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dde00640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 0x7fd0d81a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dd400640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0dca00640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d819e850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0c402a930 tx=0x7fd0c4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c403f030 con 0x7fd0d8077df0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c4003ce0 con 0x7fd0d8077df0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d81a61d0 con 0x7fd0d8077df0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4035070 con 0x7fd0d8077df0 2024-09-05T11:56:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.008+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d81a66d0 con 0x7fd0d8077df0 2024-09-05T11:56:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.010+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d81105d0 con 0x7fd0d8077df0 2024-09-05T11:56:34.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.011+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd0c4042030 con 0x7fd0d8077df0 2024-09-05T11:56:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.014+0000 7fd0d6000640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 0x7fd0c00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.014+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd0c40ef5e0 con 0x7fd0d8077df0 2024-09-05T11:56:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.014+0000 7fd0dd400640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 0x7fd0c00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.015+0000 7fd0dd400640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 0x7fd0c00a5180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc011500 tx=0x7fd0cc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.017+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0c40bb920 con 0x7fd0d8077df0 2024-09-05T11:56:34.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.243+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd0d8062630 con 0x7fd0c00a2cc0 2024-09-05T11:56:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.250+0000 7fd0d6000640 1 -- 172.21.15.59:0/568675489 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd0d8062630 con 0x7fd0c00a2cc0 2024-09-05T11:56:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-05T11:56:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:mds.all 12/12 64s ago 2m smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-05T11:56:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:mgr 3/3 64s ago 8m smithi059=x;smithi089=y;smithi189=z;count:3 2024-09-05T11:56:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:mon 3/3 64s ago 8m smithi059:172.21.15.59=a;smithi089:172.21.15.89=b;smithi189:172.21.15.189=c;count:3 2024-09-05T11:56:34.252 INFO:teuthology.orchestra.run.smithi059.stdout:osd 12 64s ago - 2024-09-05T11:56:34.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 msgr2=0x7fd0c00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:34.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 0x7fd0c00a5180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc011500 tx=0x7fd0cc005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:34.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 msgr2=0x7fd0d819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:34.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d819e850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0c402a930 tx=0x7fd0c4032ce0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 shutdown_connections 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0c00a2cc0 0x7fd0c00a5180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0d8078790 0x7fd0d81a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0d8077df0 0x7fd0d819e850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 --2- 172.21.15.59:0/568675489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd0d8076ba0 0x7fd0d819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.252+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 >> 172.21.15.59:0/568675489 conn(0x7fd0d80ff3e0 msgr2=0x7fd0d8100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.253+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 shutdown_connections 2024-09-05T11:56:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:34.253+0000 7fd0de800640 1 -- 172.21.15.59:0/568675489 wait complete. 2024-09-05T11:56:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:34 smithi089 ceph-mon[34499]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:34.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:34 smithi059 ceph-mon[30025]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:35.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-05T11:56:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:34 smithi189 ceph-mon[34772]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:56:35.873 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:56:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:56:35.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 -- 172.21.15.59:0/2742665506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010103e00 msgr2=0x7f9010104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 --2- 172.21.15.59:0/2742665506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010103e00 0x7f9010104200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9000009900 tx=0x7f900002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 -- 172.21.15.59:0/2742665506 shutdown_connections 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 --2- 172.21.15.59:0/2742665506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010105920 0x7f901010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 --2- 172.21.15.59:0/2742665506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010104f60 0x7f90101053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 --2- 172.21.15.59:0/2742665506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010103e00 0x7f9010104200 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 -- 172.21.15.59:0/2742665506 >> 172.21.15.59:0/2742665506 conn(0x7f90100ff5b0 msgr2=0x7f90101019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:35.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 -- 172.21.15.59:0/2742665506 shutdown_connections 2024-09-05T11:56:35.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.987+0000 7f9014e00640 1 -- 172.21.15.59:0/2742665506 wait complete. 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.988+0000 7f9014e00640 1 Processor -- start 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.988+0000 7f9014e00640 1 -- start start 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010104f60 0x7f901019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 0x7f90101a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f901010fb00 con 0x7f9010105920 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f901010f980 con 0x7f9010104f60 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9014e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f901010fc80 con 0x7f9010103e00 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f900de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f900e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 0x7f90101a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f900de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:53254/0 (socket says 172.21.15.59:53254) 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f900de00640 1 -- 172.21.15.59:0/573528399 learned_addr learned my addr 172.21.15.59:0/573528399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.989+0000 7f9007e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010104f60 0x7f901019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 -- 172.21.15.59:0/573528399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010104f60 msgr2=0x7f901019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010104f60 0x7f901019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 -- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 msgr2=0x7f90101a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 0x7f90101a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 -- 172.21.15.59:0/573528399 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101a1d30 con 0x7f9010103e00 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900e800640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 0x7f90101a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f900de00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9000031440 tx=0x7f9000032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900003f030 con 0x7f9010103e00 2024-09-05T11:56:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9000003ce0 con 0x7f9010103e00 2024-09-05T11:56:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.990+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90101a1fc0 con 0x7f9010103e00 2024-09-05T11:56:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.991+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000035070 con 0x7f9010103e00 2024-09-05T11:56:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.991+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101a23b0 con 0x7f9010103e00 2024-09-05T11:56:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.992+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901010b1a0 con 0x7f9010103e00 2024-09-05T11:56:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.994+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9000042030 con 0x7f9010103e00 2024-09-05T11:56:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.995+0000 7f9006a00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 0x7f8ff40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.995+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f90000efae0 con 0x7f9010103e00 2024-09-05T11:56:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.995+0000 7f9007e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 0x7f8ff40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.996+0000 7f9007e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 0x7f8ff40a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f901019b620 tx=0x7f8ff8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:35.997+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90000bbe20 con 0x7f9010103e00 2024-09-05T11:56:36.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:56:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:56:36.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:56:36.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.189+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9010062630 con 0x7f8ff40a2c40 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.190+0000 7f9006a00640 1 -- 172.21.15.59:0/573528399 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9010062630 con 0x7f8ff40a2c40 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:HOST ADDR LABELS STATUS 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 172.21.15.59 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:smithi089 172.21.15.89 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:smithi189 172.21.15.189 2024-09-05T11:56:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:3 hosts in cluster 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 msgr2=0x7f8ff40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 0x7f8ff40a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f901019b620 tx=0x7f8ff8005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 msgr2=0x7f901019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9000031440 tx=0x7f9000032ce0 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 shutdown_connections 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ff40a2c40 0x7f8ff40a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9010105920 0x7f90101a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9010104f60 0x7f901019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 --2- 172.21.15.59:0/573528399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9010103e00 0x7f901019a100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 >> 172.21.15.59:0/573528399 conn(0x7f90100ff5b0 msgr2=0x7f9010108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 shutdown_connections 2024-09-05T11:56:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:36.193+0000 7f9014e00640 1 -- 172.21.15.59:0/573528399 wait complete. 2024-09-05T11:56:36.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:36 smithi059 ceph-mon[30025]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:36.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-05T11:56:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:36 smithi189 ceph-mon[34772]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:36 smithi089 ceph-mon[34499]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-05T11:56:37.269 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:37.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 -- 172.21.15.59:0/1997014757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040103e00 msgr2=0x7f4040104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:37.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 --2- 172.21.15.59:0/1997014757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040103e00 0x7f4040104200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4030009900 tx=0x7f403002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 -- 172.21.15.59:0/1997014757 shutdown_connections 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 --2- 172.21.15.59:0/1997014757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040105920 0x7f404010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 --2- 172.21.15.59:0/1997014757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040104f60 0x7f40401053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 --2- 172.21.15.59:0/1997014757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040103e00 0x7f4040104200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 -- 172.21.15.59:0/1997014757 >> 172.21.15.59:0/1997014757 conn(0x7f40400ff5b0 msgr2=0x7f40401019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 -- 172.21.15.59:0/1997014757 shutdown_connections 2024-09-05T11:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.867+0000 7f4046600640 1 -- 172.21.15.59:0/1997014757 wait complete. 2024-09-05T11:56:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 Processor -- start 2024-09-05T11:56:37.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 -- start start 2024-09-05T11:56:37.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:37.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040104f60 0x7f404019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:37.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 0x7f40401a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:37.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f404010fab0 con 0x7f4040104f60 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f404010f930 con 0x7f4040103e00 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.868+0000 7f4046600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f404010fc30 con 0x7f4040105920 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:33064/0 (socket says 172.21.15.59:33064) 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 -- 172.21.15.59:0/1783596954 learned_addr learned my addr 172.21.15.59:0/1783596954 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403fe00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 0x7f40401a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403ea00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040104f60 0x7f404019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 -- 172.21.15.59:0/1783596954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 msgr2=0x7f40401a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 0x7f40401a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 -- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040104f60 msgr2=0x7f404019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040104f60 0x7f404019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403f400640 1 -- 172.21.15.59:0/1783596954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40401a1d30 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.869+0000 7f403fe00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 0x7f40401a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f403f400640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f40300042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403003f030 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4030003ce0 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030035070 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40401a1fc0 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.870+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40401a23c0 con 0x7f4040103e00 2024-09-05T11:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.871+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404010b240 con 0x7f4040103e00 2024-09-05T11:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.873+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4030042030 con 0x7f4040103e00 2024-09-05T11:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.874+0000 7f4037e00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 0x7f40240a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.874+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f40300efc30 con 0x7f4040103e00 2024-09-05T11:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.875+0000 7f403ea00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 0x7f40240a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.875+0000 7f403ea00640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 0x7f40240a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f404019b620 tx=0x7f402c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:37.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:37.877+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40300bbf70 con 0x7f4040103e00 2024-09-05T11:56:37.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:37 smithi059 ceph-mon[30025]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:38.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.071+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4040062630 con 0x7f40240a2c40 2024-09-05T11:56:38.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.072+0000 7f4037e00640 1 -- 172.21.15.59:0/1783596954 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4040062630 con 0x7f40240a2c40 2024-09-05T11:56:38.074 INFO:teuthology.orchestra.run.smithi059.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-05T11:56:38.074 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T11:56:38.074 INFO:teuthology.orchestra.run.smithi059.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T11:56:38.074 INFO:teuthology.orchestra.run.smithi059.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 msgr2=0x7f40240a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 0x7f40240a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f404019b620 tx=0x7f402c005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 msgr2=0x7f404019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4030009230 tx=0x7f40300042f0 comp rx=0 tx=0).stop 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 shutdown_connections 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40240a2c40 0x7f40240a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:38.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4040105920 0x7f40401a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:38.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4040104f60 0x7f404019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:38.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 --2- 172.21.15.59:0/1783596954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4040103e00 0x7f404019a100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:38.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.074+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 >> 172.21.15.59:0/1783596954 conn(0x7f40400ff5b0 msgr2=0x7f4040108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:38.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.075+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 shutdown_connections 2024-09-05T11:56:38.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:38.075+0000 7f4046600640 1 -- 172.21.15.59:0/1783596954 wait complete. 2024-09-05T11:56:38.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:37 smithi189 ceph-mon[34772]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:37 smithi089 ceph-mon[34499]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:38.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:38 smithi059 ceph-mon[30025]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:38.834 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T11:56:38.844 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T11:56:38.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-05T11:56:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:38 smithi189 ceph-mon[34772]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:38 smithi089 ceph-mon[34499]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:39.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 -- 172.21.15.59:0/2084905505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10102e00 msgr2=0x7f0c10103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:39.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 --2- 172.21.15.59:0/2084905505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10102e00 0x7f0c10103280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009900 tx=0x7f0bfc02a410 comp rx=0 tx=0).stop 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 -- 172.21.15.59:0/2084905505 shutdown_connections 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 --2- 172.21.15.59:0/2084905505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c101037c0 0x7f0c1010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 --2- 172.21.15.59:0/2084905505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10102e00 0x7f0c10103280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 --2- 172.21.15.59:0/2084905505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c10101c00 0x7f0c10102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.779+0000 7f0c17200640 1 -- 172.21.15.59:0/2084905505 >> 172.21.15.59:0/2084905505 conn(0x7f0c100fd390 msgr2=0x7f0c100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.780+0000 7f0c17200640 1 -- 172.21.15.59:0/2084905505 shutdown_connections 2024-09-05T11:56:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.780+0000 7f0c17200640 1 -- 172.21.15.59:0/2084905505 wait complete. 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.781+0000 7f0c17200640 1 Processor -- start 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.781+0000 7f0c17200640 1 -- start start 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.781+0000 7f0c17200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.781+0000 7f0c17200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 0x7f0c100745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.781+0000 7f0c17200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c101037c0 0x7f0c10074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c17200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010d8d0 con 0x7f0c10101c00 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c17200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010d750 con 0x7f0c10102e00 2024-09-05T11:56:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c17200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010da50 con 0x7f0c101037c0 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c15200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c101037c0 0x7f0c10074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 0x7f0c100745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37956/0 (socket says 172.21.15.59:37956) 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0fe00640 1 -- 172.21.15.59:0/731245711 learned_addr learned my addr 172.21.15.59:0/731245711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 -- 172.21.15.59:0/731245711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c101037c0 msgr2=0x7f0c10074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c101037c0 0x7f0c10074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 -- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 msgr2=0x7f0c10075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0f400640 1 -- 172.21.15.59:0/731245711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c100753e0 con 0x7f0c10102e00 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.782+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c0f400640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 0x7f0c100745e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02a930 tx=0x7f0bfc032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc03f030 con 0x7f0c10102e00 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101a1ff0 con 0x7f0c10102e00 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bfc003ce0 con 0x7f0c10102e00 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc035070 con 0x7f0c10102e00 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.783+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a2440 con 0x7f0c10102e00 2024-09-05T11:56:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.784+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10108f30 con 0x7f0c10102e00 2024-09-05T11:56:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.786+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0bfc042030 con 0x7f0c10102e00 2024-09-05T11:56:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.787+0000 7f0c0cc00640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 0x7f0bf40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.787+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0bfc0efa10 con 0x7f0c10102e00 2024-09-05T11:56:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.788+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 0x7f0bf40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.788+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 0x7f0bf40a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005fd0 tx=0x7f0c0000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:39.792+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bfc0bbd50 con 0x7f0c10102e00 2024-09-05T11:56:39.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:39 smithi059 ceph-mon[30025]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.016+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0c1002d9e0 con 0x7f0c10102e00 2024-09-05T11:56:40.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.017+0000 7f0c0cc00640 1 -- 172.21.15.59:0/731245711 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3046 (secure 0 0 0) 0x7f0bfc0c0050 con 0x7f0c10102e00 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:e21 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:btime 2024-09-05T11:55:15:722116+0000 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:legacy client fscid: 1 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:Filesystem 'cephfs' (1) 2024-09-05T11:56:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:fs_name cephfs 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 21 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-09-05T11:54:32.492433+0000 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-09-05T11:55:14.725014+0000 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:tableserver 0 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:root 0 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:session_timeout 300 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:session_autoclose 300 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:max_file_size 1099511627776 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:max_xattr_size 65536 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:required_client_features {} 2024-09-05T11:56:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure 0 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure_osd_epoch 0 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:max_mds 1 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:in 0 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:up {0=24545} 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:failed 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:damaged 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:stopped 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:data_pools [3,4] 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:metadata_pool 2 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:inline_data disabled 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:balancer 2024-09-05T11:56:40.021 INFO:teuthology.orchestra.run.smithi059.stdout:bal_rank_mask -1 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:standby_count_wanted 0 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:qdb_cluster leader: 24545 members: 24545 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b{0:24545} state up:active seq 5 addr [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.h{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.89:6836/2748476304,v1:172.21.15.89:6837/2748476304] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:Standby daemons: 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.j{-1:14478} state up:standby seq 1 addr [v2:172.21.15.59:6834/1152129733,v1:172.21.15.59:6835/1152129733] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d{-1:14490} state up:standby seq 1 addr [v2:172.21.15.59:6836/1365446032,v1:172.21.15.59:6837/1365446032] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.g{-1:14520} state up:standby seq 1 addr [v2:172.21.15.59:6838/3325685690,v1:172.21.15.59:6839/3325685690] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a{-1:14532} state up:standby seq 1 addr [v2:172.21.15.59:6840/2374945780,v1:172.21.15.59:6841/2374945780] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.022 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.189:6832/3760489514,v1:172.21.15.189:6833/3760489514] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.023 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.89:6832/2738174591,v1:172.21.15.89:6833/2738174591] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.023 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.f{-1:24472} state up:standby seq 1 addr [v2:172.21.15.189:6834/4161006032,v1:172.21.15.189:6835/4161006032] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.023 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c{-1:24478} state up:standby seq 1 addr [v2:172.21.15.189:6836/827690495,v1:172.21.15.189:6837/827690495] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.023 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.e{-1:24485} state up:standby seq 1 addr [v2:172.21.15.89:6834/318370648,v1:172.21.15.89:6835/318370648] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.023 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.i{-1:24499} state up:standby seq 1 addr [v2:172.21.15.189:6838/3241376158,v1:172.21.15.189:6839/3241376158] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 msgr2=0x7f0bf40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 0x7f0bf40a5150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c00005fd0 tx=0x7f0c0000b410 comp rx=0 tx=0).stop 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 msgr2=0x7f0c100745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 0x7f0c100745e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02a930 tx=0x7f0bfc032ce0 comp rx=0 tx=0).stop 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 shutdown_connections 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bf40a2c90 0x7f0bf40a5150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c101037c0 0x7f0c10074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c10102e00 0x7f0c100745e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:40.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 --2- 172.21.15.59:0/731245711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c10101c00 0x7f0c10075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 >> 172.21.15.59:0/731245711 conn(0x7f0c100fd390 msgr2=0x7f0c10106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 shutdown_connections 2024-09-05T11:56:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:40.019+0000 7f0c17200640 1 -- 172.21.15.59:0/731245711 wait complete. 2024-09-05T11:56:40.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:39 smithi189 ceph-mon[34772]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:39 smithi089 ceph-mon[34499]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T11:56:40.701 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-05T11:56:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:40 smithi059 ceph-mon[30025]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:40 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/731245711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T11:56:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:40 smithi189 ceph-mon[34772]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:40 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/731245711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T11:56:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:40 smithi089 ceph-mon[34499]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:40 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/731245711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T11:56:41.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 -- 172.21.15.59:0/2580568897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0104e50 msgr2=0x7f2dc01052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 --2- 172.21.15.59:0/2580568897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0104e50 0x7f2dc01052d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009900 tx=0x7f2dac02a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 -- 172.21.15.59:0/2580568897 shutdown_connections 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 --2- 172.21.15.59:0/2580568897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 --2- 172.21.15.59:0/2580568897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0104e50 0x7f2dc01052d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 --2- 172.21.15.59:0/2580568897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0103c50 0x7f2dc0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 -- 172.21.15.59:0/2580568897 >> 172.21.15.59:0/2580568897 conn(0x7f2dc00ff3e0 msgr2=0x7f2dc0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 -- 172.21.15.59:0/2580568897 shutdown_connections 2024-09-05T11:56:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.630+0000 7f2dc6600640 1 -- 172.21.15.59:0/2580568897 wait complete. 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 Processor -- start 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 -- start start 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0103c50 0x7f2dc0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010f8b0 con 0x7f2dc0104e50 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010f730 con 0x7f2dc0105810 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.631+0000 7f2dc6600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010fa30 con 0x7f2dc0103c50 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56698/0 (socket says 172.21.15.59:56698) 2024-09-05T11:56:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 -- 172.21.15.59:0/96835338 learned_addr learned my addr 172.21.15.59:0/96835338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dc5c00640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dc5200640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0103c50 0x7f2dc0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 -- 172.21.15.59:0/96835338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0103c50 msgr2=0x7f2dc0199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0103c50 0x7f2dc0199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 -- 172.21.15.59:0/96835338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 msgr2=0x7f2dc01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 -- 172.21.15.59:0/96835338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc01a1b10 con 0x7f2dc0104e50 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dc5c00640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.632+0000 7f2dbfe00640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009230 tx=0x7f2dac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.633+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03f030 con 0x7f2dc0104e50 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.633+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac003ce0 con 0x7f2dc0104e50 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.633+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac035070 con 0x7f2dc0104e50 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.633+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc01a1da0 con 0x7f2dc0104e50 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.633+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc01a2220 con 0x7f2dc0104e50 2024-09-05T11:56:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.634+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc010b050 con 0x7f2dc0104e50 2024-09-05T11:56:41.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.636+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2dac042030 con 0x7f2dc0104e50 2024-09-05T11:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.637+0000 7f2dbd600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 0x7f2da80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.638+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2dac0ef750 con 0x7f2dc0104e50 2024-09-05T11:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.638+0000 7f2dc5200640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 0x7f2da80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.638+0000 7f2dc5200640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 0x7f2da80a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2db40095f0 tx=0x7f2db4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:41.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.642+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dac0bba90 con 0x7f2dc0104e50 2024-09-05T11:56:41.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.832+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2dc002da30 con 0x7f2dc0104e50 2024-09-05T11:56:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.833+0000 7f2dbd600640 1 -- 172.21.15.59:0/96835338 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5754 (secure 0 0 0) 0x7f2dac0bfd90 con 0x7f2dc0104e50 2024-09-05T11:56:41.834 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 93 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-09-05T11:46:24.063849+0000 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-09-05T11:56:19.566525+0000 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 27 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client luminous 2024-09-05T11:56:41.835 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release squid 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 12 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6802/2347559609,v1:172.21.15.59:6803/2347559609] [v2:172.21.15.59:6804/2347559609,v1:172.21.15.59:6805/2347559609] exists,up 8d274feb-cee7-4082-a14e-2c4c9ed0bc40 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/3265971901,v1:172.21.15.89:6801/3265971901] [v2:172.21.15.89:6802/3265971901,v1:172.21.15.89:6803/3265971901] exists,up 02b53955-a3d9-45d3-9a4f-c36d00907942 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/3282762650,v1:172.21.15.189:6801/3282762650] [v2:172.21.15.189:6802/3282762650,v1:172.21.15.189:6803/3282762650] exists,up b03ca7ba-23bc-4f72-b35e-1784080f9550 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6810/2051936210,v1:172.21.15.59:6811/2051936210] [v2:172.21.15.59:6812/2051936210,v1:172.21.15.59:6813/2051936210] exists,up fe023f0b-749c-4eed-ae29-f58180c46a1a 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 32 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/200597957,v1:172.21.15.89:6809/200597957] [v2:172.21.15.89:6810/200597957,v1:172.21.15.89:6811/200597957] exists,up 1ff44594-2df9-4bb2-8154-402f258fab7c 2024-09-05T11:56:41.836 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 37 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/3482558888,v1:172.21.15.189:6809/3482558888] [v2:172.21.15.189:6810/3482558888,v1:172.21.15.189:6811/3482558888] exists,up 89d7954d-f1d1-4ac8-b1df-e6c542fe0104 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 up in weight 1 up_from 44 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6818/3110186175,v1:172.21.15.59:6819/3110186175] [v2:172.21.15.59:6820/3110186175,v1:172.21.15.59:6821/3110186175] exists,up 3748a7e4-df25-42ae-b0cf-a50a43bb3ed6 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 up in weight 1 up_from 49 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2237080020,v1:172.21.15.89:6817/2237080020] [v2:172.21.15.89:6818/2237080020,v1:172.21.15.89:6819/2237080020] exists,up 20387607-714b-47c7-af78-daf2be8da827 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 up in weight 1 up_from 57 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/2486650777,v1:172.21.15.189:6817/2486650777] [v2:172.21.15.189:6818/2486650777,v1:172.21.15.189:6819/2486650777] exists,up 7405b92c-3969-427a-b954-31e466aef539 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 up in weight 1 up_from 64 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6826/1620616877,v1:172.21.15.59:6827/1620616877] [v2:172.21.15.59:6828/1620616877,v1:172.21.15.59:6829/1620616877] exists,up 85bf4014-3af1-4004-ac79-add151bbd6df 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 up in weight 1 up_from 69 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6824/1393916736,v1:172.21.15.89:6825/1393916736] [v2:172.21.15.89:6826/1393916736,v1:172.21.15.89:6827/1393916736] exists,up f83d4779-7bf1-47e6-93a7-e54379bc692f 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 up in weight 1 up_from 76 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/1305059632,v1:172.21.15.189:6825/1305059632] [v2:172.21.15.189:6826/1305059632,v1:172.21.15.189:6827/1305059632] exists,up fedf8ed4-fc52-4603-b9cd-19837038c324 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.6 [8,11] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.11 [7,11] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.16 [2,4] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1a [8,1] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1b [2,4] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.1a [5,6] 2024-09-05T11:56:41.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.1b [4,8] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.2c [4,9] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.38 [3,9] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.39 [5,8] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.3a [4,7] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.3c [4,9] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.c [9,10] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.18 [9,8] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.19 [9,8] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.1a [5,8] 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/2057764847 expires 2024-09-06T11:47:16.761138+0000 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3566679520 expires 2024-09-06T11:47:16.761138+0000 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1131370065 expires 2024-09-06T11:47:16.761138+0000 2024-09-05T11:56:41.838 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1895165557 expires 2024-09-06T11:47:16.761138+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/2307158254 expires 2024-09-06T11:46:49.566628+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/2307158254 expires 2024-09-06T11:46:49.566628+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3602565252 expires 2024-09-06T11:46:49.566628+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1380942177 expires 2024-09-06T11:46:49.566628+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/2057764847 expires 2024-09-06T11:47:16.761138+0000 2024-09-05T11:56:41.839 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1592551985 expires 2024-09-06T11:46:49.566628+0000 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 msgr2=0x7f2da80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 0x7f2da80a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2db40095f0 tx=0x7f2db4005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 msgr2=0x7f2dc019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009230 tx=0x7f2dac0042f0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 shutdown_connections 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2da80a2c40 0x7f2da80a5100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dc0105810 0x7f2dc01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dc0104e50 0x7f2dc019a420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 --2- 172.21.15.59:0/96835338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dc0103c50 0x7f2dc0199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.835+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 >> 172.21.15.59:0/96835338 conn(0x7f2dc00ff3e0 msgr2=0x7f2dc0108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.836+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 shutdown_connections 2024-09-05T11:56:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:41.836+0000 7f2dc6600640 1 -- 172.21.15.59:0/96835338 wait complete. 2024-09-05T11:56:42.576 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-05T11:56:42.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:42 smithi059 ceph-mon[30025]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/96835338' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T11:56:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:42 smithi189 ceph-mon[34772]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:42 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/96835338' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T11:56:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:42 smithi089 ceph-mon[34499]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:42 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/96835338' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T11:56:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 -- 172.21.15.59:0/3836090384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac103ff0 msgr2=0x7fa9ac1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 --2- 172.21.15.59:0/3836090384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac1043d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009900 tx=0x7fa9a002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 -- 172.21.15.59:0/3836090384 shutdown_connections 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 --2- 172.21.15.59:0/3836090384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac109120 0x7fa9ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 --2- 172.21.15.59:0/3836090384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 --2- 172.21.15.59:0/3836090384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac1043d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 -- 172.21.15.59:0/3836090384 >> 172.21.15.59:0/3836090384 conn(0x7fa9ac0ffa90 msgr2=0x7fa9ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 -- 172.21.15.59:0/3836090384 shutdown_connections 2024-09-05T11:56:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.528+0000 7fa9b4200640 1 -- 172.21.15.59:0/3836090384 wait complete. 2024-09-05T11:56:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 Processor -- start 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 -- start start 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 0x7fa9ac19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac114140 con 0x7fa9ac103ff0 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac113fc0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.529+0000 7fa9b4200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac1142c0 con 0x7fa9ac109120 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:43.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56724/0 (socket says 172.21.15.59:56724) 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b1800640 1 -- 172.21.15.59:0/763556787 learned_addr learned my addr 172.21.15.59:0/763556787 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b2200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 0x7fa9ac19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 -- 172.21.15.59:0/763556787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 msgr2=0x7fa9ac19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 0x7fa9ac19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 -- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 msgr2=0x7fa9ac19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 -- 172.21.15.59:0/763556787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac19f5b0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b2200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 0x7fa9ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:43.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.530+0000 7fa9b0e00640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac19ab40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e9f0 tx=0x7fa99800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.531+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99800ccc0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.531+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa998005020 con 0x7fa9ac1049a0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.531+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980049e0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.531+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac19f8a0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.531+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac1a7180 con 0x7fa9ac1049a0 2024-09-05T11:56:43.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.532+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac1043d0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.534+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa998005560 con 0x7fa9ac1049a0 2024-09-05T11:56:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.536+0000 7fa9aa000640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.536+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa9980cf5c0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.536+0000 7fa9b1800640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.537+0000 7fa9b1800640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 0x7fa9940a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00096c0 tx=0x7fa9a0005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:43.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.537+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9980051d0 con 0x7fa9ac1049a0 2024-09-05T11:56:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.760+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9ac19b960 con 0x7fa9ac1049a0 2024-09-05T11:56:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.761+0000 7fa9aa000640 1 -- 172.21.15.59:0/763556787 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fa99809bf50 con 0x7fa9ac1049a0 2024-09-05T11:56:43.762 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:43.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 msgr2=0x7fa9940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 0x7fa9940a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00096c0 tx=0x7fa9a0005a20 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 msgr2=0x7fa9ac19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac19ab40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e9f0 tx=0x7fa99800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 shutdown_connections 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9ac109120 0x7fa9ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9ac1049a0 0x7fa9ac19ab40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 --2- 172.21.15.59:0/763556787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9ac103ff0 0x7fa9ac19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 >> 172.21.15.59:0/763556787 conn(0x7fa9ac0ffa90 msgr2=0x7fa9ac100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 shutdown_connections 2024-09-05T11:56:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:43.763+0000 7fa9b4200640 1 -- 172.21.15.59:0/763556787 wait complete. 2024-09-05T11:56:44.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:44.852 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:44.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:44 smithi059 ceph-mon[30025]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:44.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:44 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/763556787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:44 smithi189 ceph-mon[34772]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:44 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/763556787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:44 smithi089 ceph-mon[34499]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:44 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/763556787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:45.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 -- 172.21.15.59:0/911711616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d41049a0 msgr2=0x7ff5d41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 --2- 172.21.15.59:0/911711616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d41049a0 0x7ff5d41089f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009900 tx=0x7ff5c002a410 comp rx=0 tx=0).stop 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 -- 172.21.15.59:0/911711616 shutdown_connections 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 --2- 172.21.15.59:0/911711616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4109120 0x7ff5d4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 --2- 172.21.15.59:0/911711616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d41049a0 0x7ff5d41089f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 --2- 172.21.15.59:0/911711616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d4103ff0 0x7ff5d41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.410+0000 7ff5dc200640 1 -- 172.21.15.59:0/911711616 >> 172.21.15.59:0/911711616 conn(0x7ff5d40ffa90 msgr2=0x7ff5d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.411+0000 7ff5dc200640 1 -- 172.21.15.59:0/911711616 shutdown_connections 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.411+0000 7ff5dc200640 1 -- 172.21.15.59:0/911711616 wait complete. 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.412+0000 7ff5dc200640 1 Processor -- start 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.412+0000 7ff5dc200640 1 -- start start 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.412+0000 7ff5dc200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5dc200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d41049a0 0x7ff5d419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5dc200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4109120 0x7ff5d419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5dc200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4114170 con 0x7ff5d4109120 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5dc200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4113ff0 con 0x7ff5d4103ff0 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5dc200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41142f0 con 0x7ff5d41049a0 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5da200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4109120 0x7ff5d419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:35594/0 (socket says 172.21.15.59:35594) 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d41049a0 0x7ff5d419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 -- 172.21.15.59:0/2292217939 learned_addr learned my addr 172.21.15.59:0/2292217939 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 -- 172.21.15.59:0/2292217939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d41049a0 msgr2=0x7ff5d419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d41049a0 0x7ff5d419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 -- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4109120 msgr2=0x7ff5d419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4109120 0x7ff5d419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.413+0000 7ff5d9800640 1 -- 172.21.15.59:0/2292217939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d419f600 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5d9800640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009d50 tx=0x7ff5c800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8003b70 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d419f8f0 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8003d10 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8010a20 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.414+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a71d0 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.415+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d41043d0 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.417+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff5c8010bc0 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.419+0000 7ff5d2000640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 0x7ff5bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.419+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff5c80cfd70 con 0x7ff5d4103ff0 2024-09-05T11:56:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.419+0000 7ff5d8e00640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 0x7ff5bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:45.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.419+0000 7ff5d8e00640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 0x7ff5bc0a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:45.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.421+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5c809c790 con 0x7ff5d4103ff0 2024-09-05T11:56:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.645+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5d410c360 con 0x7ff5d4103ff0 2024-09-05T11:56:45.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.646+0000 7ff5d2000640 1 -- 172.21.15.59:0/2292217939 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7ff5c809c090 con 0x7ff5d4103ff0 2024-09-05T11:56:45.648 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:45.648 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 msgr2=0x7ff5bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 0x7ff5bc0a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0005b50 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 msgr2=0x7ff5d419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009d50 tx=0x7ff5c800c6a0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 shutdown_connections 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff5bc0a2c10 0x7ff5bc0a50d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5d4109120 0x7ff5d419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d41049a0 0x7ff5d419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 --2- 172.21.15.59:0/2292217939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5d4103ff0 0x7ff5d419a650 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 >> 172.21.15.59:0/2292217939 conn(0x7ff5d40ffa90 msgr2=0x7ff5d4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.649+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 shutdown_connections 2024-09-05T11:56:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:45.650+0000 7ff5dc200640 1 -- 172.21.15.59:0/2292217939 wait complete. 2024-09-05T11:56:45.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2292217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:46.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2292217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2292217939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:46.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:56:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:47.029 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:46 smithi059 ceph-mon[30025]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:46 smithi189 ceph-mon[34772]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:46 smithi089 ceph-mon[34499]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 -- 172.21.15.59:0/1802659394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 msgr2=0x7fe8941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1802659394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe8941021c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe888009900 tx=0x7fe88802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 -- 172.21.15.59:0/1802659394 shutdown_connections 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1802659394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 0x7fe89410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1802659394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 0x7fe8941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1802659394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe8941021c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 -- 172.21.15.59:0/1802659394 >> 172.21.15.59:0/1802659394 conn(0x7fe8940fd8c0 msgr2=0x7fe8940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 -- 172.21.15.59:0/1802659394 shutdown_connections 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.277+0000 7fe89bc00640 1 -- 172.21.15.59:0/1802659394 wait complete. 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.278+0000 7fe89bc00640 1 Processor -- start 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 -- start start 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 0x7fe89419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 0x7fe89419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe894111fc0 con 0x7fe894101de0 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe894111e40 con 0x7fe894102790 2024-09-05T11:56:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe89bc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe894112140 con 0x7fe894106f10 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe899200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe899200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56756/0 (socket says 172.21.15.59:56756) 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe899200640 1 -- 172.21.15.59:0/1358925156 learned_addr learned my addr 172.21.15.59:0/1358925156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe893e00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 0x7fe89419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.279+0000 7fe899c00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 0x7fe89419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 -- 172.21.15.59:0/1358925156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 msgr2=0x7fe89419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 0x7fe89419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 -- 172.21.15.59:0/1358925156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 msgr2=0x7fe89419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 0x7fe89419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 -- 172.21.15.59:0/1358925156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89419f690 con 0x7fe894101de0 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe899200640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe888004770 tx=0x7fe888003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88803f070 con 0x7fe894101de0 2024-09-05T11:56:47.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888031ca0 con 0x7fe894101de0 2024-09-05T11:56:47.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.280+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888035030 con 0x7fe894101de0 2024-09-05T11:56:47.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.281+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89419f920 con 0x7fe894101de0 2024-09-05T11:56:47.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.281+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89419fdb0 con 0x7fe894101de0 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.282+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8941021c0 con 0x7fe894101de0 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.284+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe888042070 con 0x7fe894101de0 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.285+0000 7fe891600640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 0x7fe87c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.285+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe8880efb80 con 0x7fe894101de0 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.285+0000 7fe893e00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 0x7fe87c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.286+0000 7fe893e00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 0x7fe87c0a5150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8800095f0 tx=0x7fe880005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:47.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.290+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8880bbf80 con 0x7fe894101de0 2024-09-05T11:56:47.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.476+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8941a7450 con 0x7fe894101de0 2024-09-05T11:56:47.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.477+0000 7fe891600640 1 -- 172.21.15.59:0/1358925156 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7fe8880c0280 con 0x7fe894101de0 2024-09-05T11:56:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.479+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 msgr2=0x7fe87c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.479+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 0x7fe87c0a5150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8800095f0 tx=0x7fe880005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.479+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 msgr2=0x7fe89419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.479+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe888004770 tx=0x7fe888003c10 comp rx=0 tx=0).stop 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 shutdown_connections 2024-09-05T11:56:47.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe87c0a2c90 0x7fe87c0a5150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe894106f10 0x7fe89419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe894102790 0x7fe89419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 --2- 172.21.15.59:0/1358925156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894101de0 0x7fe89419a6e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 >> 172.21.15.59:0/1358925156 conn(0x7fe8940fd8c0 msgr2=0x7fe8940fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 shutdown_connections 2024-09-05T11:56:47.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:47.480+0000 7fe89bc00640 1 -- 172.21.15.59:0/1358925156 wait complete. 2024-09-05T11:56:48.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:47 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1358925156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:47 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1358925156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:48.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1358925156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:48.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T11:54:32.492433+0000', 'modified': '2024-09-05T11:55:14.725014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/2748476304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 2748476304}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 2748476304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4050620246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4050620246}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4050620246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24545]} 2024-09-05T11:56:48.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T11:56:48.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T11:56:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:48 smithi189 ceph-mon[34772]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:49.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:48 smithi189 ceph-mon[34772]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T11:56:49.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:48 smithi059 ceph-mon[30025]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:49.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:48 smithi059 ceph-mon[30025]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T11:56:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:48 smithi089 ceph-mon[34499]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:56:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:48 smithi089 ceph-mon[34499]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 -- 172.21.15.59:0/3848565858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61541047e0 msgr2=0x7f6154108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 --2- 172.21.15.59:0/3848565858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61541047e0 0x7f6154108830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6140009900 tx=0x7f614002a410 comp rx=0 tx=0).stop 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 -- 172.21.15.59:0/3848565858 shutdown_connections 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 --2- 172.21.15.59:0/3848565858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6154108f60 0x7f6154110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 --2- 172.21.15.59:0/3848565858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61541047e0 0x7f6154108830 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 --2- 172.21.15.59:0/3848565858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f6154104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 -- 172.21.15.59:0/3848565858 >> 172.21.15.59:0/3848565858 conn(0x7f61540ff8d0 msgr2=0x7f6154101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 -- 172.21.15.59:0/3848565858 shutdown_connections 2024-09-05T11:56:49.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.225+0000 7f615ae00640 1 -- 172.21.15.59:0/3848565858 wait complete. 2024-09-05T11:56:49.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.226+0000 7f615ae00640 1 Processor -- start 2024-09-05T11:56:49.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 -- start start 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f615419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61541047e0 0x7f615419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6154113fc0 con 0x7f6154108f60 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6154113e40 con 0x7f61541047e0 2024-09-05T11:56:49.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6154114140 con 0x7f6154103e30 2024-09-05T11:56:49.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f6159a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f615419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56768/0 (socket says 172.21.15.59:56768) 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.227+0000 7f615a400640 1 -- 172.21.15.59:0/3937748951 learned_addr learned my addr 172.21.15.59:0/3937748951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f6159000640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61541047e0 0x7f615419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 -- 172.21.15.59:0/3937748951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 msgr2=0x7f615419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f615419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 -- 172.21.15.59:0/3937748951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61541047e0 msgr2=0x7f615419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61541047e0 0x7f615419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 -- 172.21.15.59:0/3937748951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615419f410 con 0x7f6154108f60 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f6159a00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f615419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:49.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.228+0000 7f615a400640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f614c00eaa0 tx=0x7f614c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.229+0000 7f6152000640 1 -- 172.21.15.59:0/3937748951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c00cdb0 con 0x7f6154108f60 2024-09-05T11:56:49.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.229+0000 7f6152000640 1 -- 172.21.15.59:0/3937748951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614c005020 con 0x7f6154108f60 2024-09-05T11:56:49.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.229+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615419f700 con 0x7f6154108f60 2024-09-05T11:56:49.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.229+0000 7f6152000640 1 -- 172.21.15.59:0/3937748951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c0116b0 con 0x7f6154108f60 2024-09-05T11:56:49.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.229+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61541a6fe0 con 0x7f6154108f60 2024-09-05T11:56:49.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.232+0000 7f6152000640 1 -- 172.21.15.59:0/3937748951 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f614c005560 con 0x7f6154108f60 2024-09-05T11:56:49.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.232+0000 7f615f06c740 1 Processor -- start 2024-09-05T11:56:49.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.232+0000 7f615f06c740 1 -- start start 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 0x5645210468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x564521051180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x564521053ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564520fac0b0 con 0x5645210516c0 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564520f53ab0 con 0x564521046e00 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f615f06c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564520fa8070 con 0x5645210464e0 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 0x5645210468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 0x5645210468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:47412/0 (socket says 172.21.15.59:47412) 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159000640 1 -- 172.21.15.59:0/3889098402 learned_addr learned my addr 172.21.15.59:0/3889098402 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:49.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x564521051180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159000640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x564521053ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 -- 172.21.15.59:0/3889098402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 msgr2=0x5645210468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 0x5645210468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 -- 172.21.15.59:0/3889098402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 msgr2=0x564521053ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x564521053ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159a00640 1 -- 172.21.15.59:0/3889098402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564520f604a0 con 0x564521046e00 2024-09-05T11:56:49.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.233+0000 7f6159000640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x564521053ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6159a00640 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x564521051180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f61480103d0 tx=0x7f6148010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997dfef8d7e787f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6146a00640 1 -- 172.21.15.59:0/3889098402 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148004220 con 0x564521046e00 2024-09-05T11:56:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6146a00640 1 -- 172.21.15.59:0/3889098402 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61480043c0 con 0x564521046e00 2024-09-05T11:56:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6146a00640 1 -- 172.21.15.59:0/3889098402 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148011ac0 con 0x564521046e00 2024-09-05T11:56:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6152000640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 0x7f613c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6152000640 1 -- 172.21.15.59:0/3937748951 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f614c0cf580 con 0x7f6154108f60 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.234+0000 7f6159000640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 0x7f613c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 -- 172.21.15.59:0/3889098402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 msgr2=0x564521051180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x564521051180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f61480103d0 tx=0x7f6148010890 comp rx=0 tx=0).stop 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 -- 172.21.15.59:0/3889098402 shutdown_connections 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x564521053ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x564521051180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 --2- 172.21.15.59:0/3889098402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210464e0 0x5645210468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 -- 172.21.15.59:0/3889098402 >> 172.21.15.59:0/3889098402 conn(0x564521042a70 msgr2=0x564521042e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 -- 172.21.15.59:0/3889098402 shutdown_connections 2024-09-05T11:56:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f615f06c740 1 -- 172.21.15.59:0/3889098402 wait complete. 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.235+0000 7f6159000640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 0x7f613c0a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564521043df0 tx=0x7f61400058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 -- start start 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 Processor -- start 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564521054330 con 0x5645210464e0 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564520f34920 con 0x564521046e00 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5645210544b0 con 0x5645210516c0 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f615f06c740 10 client.0 _ll_register_callbacks cb 0x564520f588a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:56:49.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.236+0000 7f6159a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56782/0 (socket says 172.21.15.59:56782) 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 -- 172.21.15.59:0/51829509 learned_addr learned my addr 172.21.15.59:0/51829509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159000640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f615a400640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 -- 172.21.15.59:0/51829509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 msgr2=0x56452104f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 -- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 msgr2=0x56452104eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 -- 172.21.15.59:0/51829509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564520d1d2b0 con 0x5645210464e0 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159000640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f6159a00640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6148005890 tx=0x7f6148011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0badbc75490297 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.237+0000 7f613b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614802a060 con 0x5645210464e0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6148005040 con 0x5645210464e0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61480043c0 con 0x5645210464e0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6148031050 con 0x5645210464e0 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f615f06c740 10 client.15054 fetch_fsmap learned FSMap version 21 2024-09-05T11:56:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56452105ffb0 con 0x5645210464e0 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61480043c0 con 0x5645210464e0 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f614801b430 con 0x5645210464e0 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.238+0000 7f615f06c740 10 client.15054 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f615f06c740 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f615f06c740 20 client.15054 populate_metadata read hostname 'smithi059' 2024-09-05T11:56:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564521062180 0x5645210635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f615f06c740 4 client.15054 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564520fb9620 con 0x564521062180 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.239+0000 7f6159000640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564521062180 0x5645210635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.240+0000 7f6159000640 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564521062180 0x5645210635f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926924a920297593 server_cookie=e9a8e2fe76fd9ae1 in_seq=0 out_seq=0 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.240+0000 7f613b400640 10 client.15054 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.240+0000 7f613b400640 1 -- 172.21.15.59:0/51829509 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564520fb9620 con 0x564521062180 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.240+0000 7f613b400640 10 client.15054 handle_command_reply: tid=0 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.240+0000 7f615f06c740 1 client.15054 shutdown 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 msgr2=0x56452104e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6148005890 tx=0x7f6148011ad0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 shutdown_connections 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564521062180 0x5645210635f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 --2- 172.21.15.59:0/51829509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x56452104e580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 >> 172.21.15.59:0/51829509 conn(0x564521042a70 msgr2=0x564521048750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f613b400640 0 client.15054 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 shutdown_connections 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 1 -- 172.21.15.59:0/51829509 wait complete. 2024-09-05T11:56:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.241+0000 7f615f06c740 20 client.15054 trim_cache size 0 max 16384 2024-09-05T11:56:49.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 Processor -- start 2024-09-05T11:56:49.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 -- start start 2024-09-05T11:56:49.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564521061930 con 0x5645210464e0 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564521062000 con 0x564521046e00 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f615f06c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5645210626d0 con 0x5645210516c0 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.260+0000 7f6159a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56796/0 (socket says 172.21.15.59:56796) 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 -- 172.21.15.59:0/52154188 learned_addr learned my addr 172.21.15.59:0/52154188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159000640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f615a400640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 -- 172.21.15.59:0/52154188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 msgr2=0x56452104f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 -- 172.21.15.59:0/52154188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 msgr2=0x56452104ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 -- 172.21.15.59:0/52154188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564521043bf0 con 0x5645210464e0 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159000640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.261+0000 7f6159a00640 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f614801b950 tx=0x7f6148001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aba811269f68663 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f6146000640 1 -- 172.21.15.59:0/52154188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614802a070 con 0x5645210464e0 2024-09-05T11:56:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f6146000640 1 -- 172.21.15.59:0/52154188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614801be50 con 0x5645210464e0 2024-09-05T11:56:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f6146000640 1 -- 172.21.15.59:0/52154188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148004660 con 0x5645210464e0 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 -- 172.21.15.59:0/52154188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 msgr2=0x564521048f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f614801b950 tx=0x7f6148001e50 comp rx=0 tx=0).stop 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 -- 172.21.15.59:0/52154188 shutdown_connections 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645210516c0 0x56452104f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564521046e00 0x56452104ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 --2- 172.21.15.59:0/52154188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210464e0 0x564521048f40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.262+0000 7f615f06c740 1 -- 172.21.15.59:0/52154188 >> 172.21.15.59:0/52154188 conn(0x564521042a70 msgr2=0x56452105cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.263+0000 7f615f06c740 1 -- 172.21.15.59:0/52154188 shutdown_connections 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.263+0000 7f615f06c740 1 -- 172.21.15.59:0/52154188 wait complete. 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.263+0000 7f615f06c740 1 -- start start 2024-09-05T11:56:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.263+0000 7f615f06c740 1 Processor -- start 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 0x564521066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 0x564521066c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564521063470 con 0x5645210516c0 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564521062da0 con 0x5645210464e0 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564521063b40 con 0x564521046e00 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615f06c740 10 client.0 _ll_register_callbacks cb 0x7f614800bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 0x564521066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56802/0 (socket says 172.21.15.59:56802) 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 0x564521066c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615a400640 1 -- 172.21.15.59:0/2731502601 learned_addr learned my addr 172.21.15.59:0/2731502601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 -- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 msgr2=0x564521066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 0x564521066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 -- 172.21.15.59:0/2731502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 msgr2=0x56452106b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159000640 1 -- 172.21.15.59:0/2731502601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452106bc50 con 0x564521046e00 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f6159a00640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 0x564521066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.264+0000 7f615a400640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f6159000640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 0x564521066c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6140000c00 tx=0x7f614002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857292f0b451df72 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f613b400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:56:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140035040 con 0x564521046e00 2024-09-05T11:56:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614801be50 con 0x564521046e00 2024-09-05T11:56:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f61400043f0 con 0x564521046e00 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6140063050 con 0x564521046e00 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f615f06c740 10 client.25021 fetch_fsmap learned FSMap version 21 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.265+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5645210cd160 con 0x564521046e00 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614801be50 con 0x564521046e00 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6140047040 con 0x564521046e00 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 10 client.25021 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 10 client.25021 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 20 client.25021 populate_metadata read hostname 'smithi059' 2024-09-05T11:56:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5645210d1d60 0x5645210d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 4 client.25021 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "status"}) -- 0x564520fb3b30 con 0x5645210d1d60 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.266+0000 7f6159a00640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5645210d1d60 0x5645210d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f6159a00640 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5645210d1d60 0x5645210d0a70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f06390d8737a57 server_cookie=8119139fa64c9a92 in_seq=0 out_seq=0 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f613b400640 10 client.25021 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f613b400640 1 -- 172.21.15.59:0/2731502601 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564520fb3b30 con 0x5645210d1d60 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f613b400640 10 client.25021 handle_command_reply: tid=0 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 client.25021 shutdown 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 msgr2=0x564521066c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 0x564521066c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6140000c00 tx=0x7f614002a410 comp rx=0 tx=0).stop 2024-09-05T11:56:49.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 shutdown_connections 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5645210d1d60 0x5645210d0a70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645210516c0 0x56452106b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564521046e00 0x564521066c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f613b400640 0 client.25021 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 --2- 172.21.15.59:0/2731502601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5645210464e0 0x564521066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.267+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 >> 172.21.15.59:0/2731502601 conn(0x564521042a70 msgr2=0x5645210413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 shutdown_connections 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615f06c740 1 -- 172.21.15.59:0/2731502601 wait complete. 2024-09-05T11:56:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615f06c740 20 client.25021 trim_cache size 0 max 16384 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "53d560d4-6b7c-11ef-bcd6-c7b262605968", 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24545, 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 136.72140010499999, 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 21, 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 93, 2024-09-05T11:56:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 89, 2024-09-05T11:56:49.276 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 149.41597545600001 2024-09-05T11:56:49.276 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:56:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 msgr2=0x7f613c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 0x7f613c0a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564521043df0 tx=0x7f61400058d0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 msgr2=0x7f615419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.268+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f614c00eaa0 tx=0x7f614c00ef60 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 shutdown_connections 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f613c0a2c00 0x7f613c0a50c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6154108f60 0x7f615419ed30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61541047e0 0x7f615419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 --2- 172.21.15.59:0/3937748951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6154103e30 0x7f615419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 >> 172.21.15.59:0/3937748951 conn(0x7f61540ff8d0 msgr2=0x7f6154100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 shutdown_connections 2024-09-05T11:56:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:49.269+0000 7f615ae00640 1 -- 172.21.15.59:0/3937748951 wait complete. 2024-09-05T11:56:49.991 INFO:teuthology.run_tasks:Running task sequential... 2024-09-05T11:56:50.001 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-05T11:56:50.001 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-05T11:56:50.001 INFO:teuthology.task.print:Enabling mgr modules 2024-09-05T11:56:50.001 INFO:teuthology.run_tasks:Running task kclient... 2024-09-05T11:56:50.017 INFO:tasks.kclient:Mounting kernel clients... 2024-09-05T11:56:50.017 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-05T11:56:50.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:50.017 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-09-05T11:56:50.053 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-05T11:56:50.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:50.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T11:56:50.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:50.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-09-05T11:56:50.472 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-05T11:56:50.472 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T11:56:50.472 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T11:56:50.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T11:56:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:50.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:50 smithi059 ceph-mon[30025]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T11:56:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:50 smithi189 ceph-mon[34772]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T11:56:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:50 smithi089 ceph-mon[34499]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 -- 172.21.15.59:0/3546154434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd204101df0 msgr2=0x7fd2041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 --2- 172.21.15.59:0/3546154434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd204101df0 0x7fd2041021d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009900 tx=0x7fd1f402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 -- 172.21.15.59:0/3546154434 shutdown_connections 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 --2- 172.21.15.59:0/3546154434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd204106f20 0x7fd20410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 --2- 172.21.15.59:0/3546154434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2041027a0 0x7fd2041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 --2- 172.21.15.59:0/3546154434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd204101df0 0x7fd2041021d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 -- 172.21.15.59:0/3546154434 >> 172.21.15.59:0/3546154434 conn(0x7fd2040fd8b0 msgr2=0x7fd2040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 -- 172.21.15.59:0/3546154434 shutdown_connections 2024-09-05T11:56:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.388+0000 7fd20ae00640 1 -- 172.21.15.59:0/3546154434 wait complete. 2024-09-05T11:56:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.389+0000 7fd20ae00640 1 Processor -- start 2024-09-05T11:56:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.389+0000 7fd20ae00640 1 -- start start 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.389+0000 7fd20ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd204101df0 0x7fd20419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd20ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd20ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd204106f20 0x7fd20419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd20ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd204111fd0 con 0x7fd2041027a0 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd20ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd204111e50 con 0x7fd204101df0 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd20ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112150 con 0x7fd204106f20 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51414/0 (socket says 172.21.15.59:51414) 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 -- 172.21.15.59:0/2679170558 learned_addr learned my addr 172.21.15.59:0/2679170558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd203e00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd204101df0 0x7fd20419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:51.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 -- 172.21.15.59:0/2679170558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd204106f20 msgr2=0x7fd20419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd204106f20 0x7fd20419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 -- 172.21.15.59:0/2679170558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd204101df0 msgr2=0x7fd20419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd204101df0 0x7fd20419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.390+0000 7fd1fbe00640 1 -- 172.21.15.59:0/2679170558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20419f5f0 con 0x7fd2041027a0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd1fbe00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00e9f0 tx=0x7fd1ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00cd80 con 0x7fd2041027a0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20419f8e0 con 0x7fd2041027a0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec010910 con 0x7fd2041027a0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec018990 con 0x7fd2041027a0 2024-09-05T11:56:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.391+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a71c0 con 0x7fd2041027a0 2024-09-05T11:56:51.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.393+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2041021d0 con 0x7fd2041027a0 2024-09-05T11:56:51.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.394+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1ec010ac0 con 0x7fd2041027a0 2024-09-05T11:56:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.395+0000 7fd201600640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 0x7fd1e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.396+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd1ec0cf890 con 0x7fd2041027a0 2024-09-05T11:56:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.396+0000 7fd203e00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 0x7fd1e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:51.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.397+0000 7fd203e00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 0x7fd1e80a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:51.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.400+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1ec09c380 con 0x7fd2041027a0 2024-09-05T11:56:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.622+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd2041a7520 con 0x7fd2041027a0 2024-09-05T11:56:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.623+0000 7fd201600640 1 -- 172.21.15.59:0/2679170558 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7fd1ec09bc80 con 0x7fd2041027a0 2024-09-05T11:56:51.625 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 msgr2=0x7fd1e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 0x7fd1e80a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4005b50 comp rx=0 tx=0).stop 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 msgr2=0x7fd20419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00e9f0 tx=0x7fd1ec00eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 shutdown_connections 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1e80a2b80 0x7fd1e80a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd204106f20 0x7fd20419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2041027a0 0x7fd20419ab80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 --2- 172.21.15.59:0/2679170558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd204101df0 0x7fd20419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.625+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 >> 172.21.15.59:0/2679170558 conn(0x7fd2040fd8b0 msgr2=0x7fd2040feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.626+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 shutdown_connections 2024-09-05T11:56:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:51.626+0000 7fd20ae00640 1 -- 172.21.15.59:0/2679170558 wait complete. 2024-09-05T11:56:51.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:51 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2679170558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:56:52.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:51 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2679170558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:56:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:51 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2679170558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T11:56:52.536 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T11:56:52.537 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-05T11:56:52.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:52.538 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-09-05T11:56:52.574 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T11:56:52.574 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T11:56:52.574 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T11:56:52.574 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42086sec preferred_lft 42086sec 2024-09-05T11:56:52.575 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-09-05T11:56:52.576 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-09-05T11:56:52.576 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T11:56:52.576 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-09-05T11:56:52.576 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-09-05T11:56:52.576 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-09-05T11:56:52.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T11:56:52.577 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:56:52.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:52.762 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:52 smithi059 ceph-mon[30025]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:52.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:52.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:52.805 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T11:56:52.860 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-09-05T11:56:52.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:52.862 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-09-05T11:56:52.898 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-09-05T11:56:52.898 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T11:56:52.898 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-05T11:56:52.899 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-05T11:56:52.899 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-05T11:56:52.899 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:56:53.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:53.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:52 smithi189 ceph-mon[34772]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:53.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:53.097 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-09-05T11:56:53.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:53.133 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-09-05T11:56:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:52 smithi089 ceph-mon[34499]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:53.208 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:56:53.208 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:56:53.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T11:56:53.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-05T11:56:53.209 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:56:53.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:53.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:53.386 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-05T11:56:53.387 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:56:53.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:53.609 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T11:56:53.609 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-09-05T11:56:53.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-09-05T11:56:53.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T11:56:53.610 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-09-05T11:56:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:53.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:53.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T11:56:53.777 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-05T11:56:53.813 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-05T11:56:53.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T11:56:53.813 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-05T11:56:53.888 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-05T11:56:53.888 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-05T11:56:53.888 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7feb4e0e-1bee-4d90-aa3c-22db3410b75a 2024-09-05T11:56:53.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:53.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7feb4e0e-1bee-4d90-aa3c-22db3410b75a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-05T11:56:54.016 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-05T11:56:54.016 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-05T11:56:54.016 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-09-05T11:56:54.652 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-05T11:56:54.653 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-05T11:56:54.653 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-09-05T11:56:54.653 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.89:6789,172.21.15.189:6789" 2024-09-05T11:56:54.653 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.89:6789,172.21.15.189:6789:/volumes/qa/sv_1/7feb4e0e-1bee-4d90-aa3c-22db3410b75a 2024-09-05T11:56:54.653 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=53d560d4-6b7c-11ef-bcd6-c7b262605968" will pass to kernel 2024-09-05T11:56:54.655 INFO:teuthology.orchestra.run.smithi059.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-05T11:56:54.655 INFO:teuthology.orchestra.run.smithi059.stdout: You just mounted a file system that supports labels which does not 2024-09-05T11:56:54.655 INFO:teuthology.orchestra.run.smithi059.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-05T11:56:54.655 INFO:teuthology.orchestra.run.smithi059.stdout: applications will generate AVC messages and not be allowed access to 2024-09-05T11:56:54.656 INFO:teuthology.orchestra.run.smithi059.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-05T11:56:54.659 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-05T11:56:54.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T11:56:54.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-05T11:56:54.717 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T11:56:54.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T11:56:54.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:54.812 INFO:teuthology.orchestra.run.smithi059.stdout:client25027 2024-09-05T11:56:54.812 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:56:54.812 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T11:56:54.848 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T11:56:54.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:54 smithi059 ceph-mon[30025]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T11:56:54.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:54.979 INFO:teuthology.orchestra.run.smithi059.stdout:client25027 2024-09-05T11:56:54.980 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T11:56:55.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T11:56:55.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:55.070 INFO:teuthology.orchestra.run.smithi059.stdout:client25027 2024-09-05T11:56:55.071 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968.client25027/status') 2024-09-05T11:56:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:54 smithi189 ceph-mon[34772]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:55.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T11:56:55.171 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.25027 (3)172.21.15.59:0/916775706 2024-09-05T11:56:55.171 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2024-09-05T11:56:55.172 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2024-09-05T11:56:55.172 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2024-09-05T11:56:55.172 INFO:teuthology.orchestra.run.smithi059.stderr:70 bytes copied, 6.0123e-05 s, 1.2 MB/s 2024-09-05T11:56:55.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T11:56:55.183 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-05T11:56:55.196 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-05T11:56:55.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T11:56:55.208 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:54 smithi089 ceph-mon[34499]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:55.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 -- 172.21.15.59:0/1977745500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68103d30 msgr2=0x7f2d68104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 --2- 172.21.15.59:0/1977745500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68103d30 0x7f2d68104130 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009900 tx=0x7f2d5802a410 comp rx=0 tx=0).stop 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 -- 172.21.15.59:0/1977745500 shutdown_connections 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 --2- 172.21.15.59:0/1977745500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d6810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 --2- 172.21.15.59:0/1977745500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68104f20 0x7f2d681053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 --2- 172.21.15.59:0/1977745500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68103d30 0x7f2d68104130 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 -- 172.21.15.59:0/1977745500 >> 172.21.15.59:0/1977745500 conn(0x7f2d680ff4b0 msgr2=0x7f2d681018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 -- 172.21.15.59:0/1977745500 shutdown_connections 2024-09-05T11:56:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.369+0000 7f2d6d200640 1 -- 172.21.15.59:0/1977745500 wait complete. 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 Processor -- start 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 -- start start 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68103d30 0x7f2d6819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d681a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810fa00 con 0x7f2d68104f20 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810f880 con 0x7f2d68103d30 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d6d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6810fb80 con 0x7f2d681058e0 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d5fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d5fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51418/0 (socket says 172.21.15.59:51418) 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.370+0000 7f2d5fe00640 1 -- 172.21.15.59:0/746982354 learned_addr learned my addr 172.21.15.59:0/746982354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d66800640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d681a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d65e00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68103d30 0x7f2d6819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 -- 172.21.15.59:0/746982354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 msgr2=0x7f2d681a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d681a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 -- 172.21.15.59:0/746982354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68103d30 msgr2=0x7f2d6819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68103d30 0x7f2d6819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 -- 172.21.15.59:0/746982354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681a1c70 con 0x7f2d68104f20 2024-09-05T11:56:55.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d66800640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d681a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5fe00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000cd80 con 0x7f2d68104f20 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50010910 con 0x7f2d68104f20 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681a1f60 con 0x7f2d68104f20 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.371+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500188f0 con 0x7f2d68104f20 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.372+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a2420 con 0x7f2d68104f20 2024-09-05T11:56:55.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.373+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6810b0d0 con 0x7f2d68104f20 2024-09-05T11:56:55.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.375+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d50010ac0 con 0x7f2d68104f20 2024-09-05T11:56:55.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.376+0000 7f2d5ea00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 0x7f2d4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.377+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2d500cf910 con 0x7f2d68104f20 2024-09-05T11:56:55.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.377+0000 7f2d65e00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 0x7f2d4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.377+0000 7f2d65e00640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 0x7f2d4c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d5802aa00 tx=0x7f2d580315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:55.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.380+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d5009c400 con 0x7f2d68104f20 2024-09-05T11:56:55.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.571+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d68062e90 con 0x7f2d68104f20 2024-09-05T11:56:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.573+0000 7f2d5ea00640 1 -- 172.21.15.59:0/746982354 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7f2d5009bd00 con 0x7f2d68104f20 2024-09-05T11:56:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:55.576 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 msgr2=0x7f2d4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 0x7f2d4c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d5802aa00 tx=0x7f2d580315b0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 msgr2=0x7f2d6819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 shutdown_connections 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2d4c0a2b90 0x7f2d4c0a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d681058e0 0x7f2d681a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d68104f20 0x7f2d6819a590 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 --2- 172.21.15.59:0/746982354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d68103d30 0x7f2d6819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 >> 172.21.15.59:0/746982354 conn(0x7f2d680ff4b0 msgr2=0x7f2d681082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 shutdown_connections 2024-09-05T11:56:55.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.575+0000 7f2d6d200640 1 -- 172.21.15.59:0/746982354 wait complete. 2024-09-05T11:56:55.594 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T11:49:37.375375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T11:54:28.033815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T11:54:30.500703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T11:54:37.110249+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T11:56:55.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T11:56:55.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 -- 172.21.15.59:0/1172325827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38076230 msgr2=0x7f7d38074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 --2- 172.21.15.59:0/1172325827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38076230 0x7f7d38074680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a410 comp rx=0 tx=0).stop 2024-09-05T11:56:55.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 -- 172.21.15.59:0/1172325827 shutdown_connections 2024-09-05T11:56:55.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 --2- 172.21.15.59:0/1172325827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38109980 0x7f7d3810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 --2- 172.21.15.59:0/1172325827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38074c50 0x7f7d380750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 --2- 172.21.15.59:0/1172325827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38076230 0x7f7d38074680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 -- 172.21.15.59:0/1172325827 >> 172.21.15.59:0/1172325827 conn(0x7f7d380fd2b0 msgr2=0x7f7d380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:55.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 -- 172.21.15.59:0/1172325827 shutdown_connections 2024-09-05T11:56:55.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.757+0000 7f7d3e000640 1 -- 172.21.15.59:0/1172325827 wait complete. 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.758+0000 7f7d3e000640 1 Processor -- start 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.758+0000 7f7d3e000640 1 -- start start 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38076230 0x7f7d3819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 0x7f7d381a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810f9d0 con 0x7f7d38109980 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810f850 con 0x7f7d38074c50 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d3e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810fb50 con 0x7f7d38076230 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 0x7f7d381a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49170/0 (socket says 172.21.15.59:49170) 2024-09-05T11:56:55.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37400640 1 -- 172.21.15.59:0/283880482 learned_addr learned my addr 172.21.15.59:0/283880482 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d36a00640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38076230 0x7f7d3819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 -- 172.21.15.59:0/283880482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38076230 msgr2=0x7f7d3819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38076230 0x7f7d3819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 -- 172.21.15.59:0/283880482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 msgr2=0x7f7d3819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37e00640 1 -- 172.21.15.59:0/283880482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d381a1c40 con 0x7f7d38109980 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.759+0000 7f7d37400640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d37e00640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 0x7f7d381a15d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009fd0 tx=0x7f7d3000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000ce20 con 0x7f7d38109980 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d30005020 con 0x7f7d38109980 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d381a1f30 con 0x7f7d38109980 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30010720 con 0x7f7d38109980 2024-09-05T11:56:55.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.760+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381a23f0 con 0x7f7d38109980 2024-09-05T11:56:55.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.761+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d38108700 con 0x7f7d38109980 2024-09-05T11:56:55.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.763+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d30005540 con 0x7f7d38109980 2024-09-05T11:56:55.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.764+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:55.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.764+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7d300cfac0 con 0x7f7d38109980 2024-09-05T11:56:55.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.765+0000 7f7d37400640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:55.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.765+0000 7f7d37400640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d280096c0 tx=0x7f7d280315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:55.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.769+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d3009c5b0 con 0x7f7d38109980 2024-09-05T11:56:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.952+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d38062e90 con 0x7f7d38109980 2024-09-05T11:56:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.952+0000 7f7d2fe00640 1 -- 172.21.15.59:0/283880482 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f7d3009beb0 con 0x7f7d38109980 2024-09-05T11:56:55.955 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-09-05T11:56:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 msgr2=0x7f7d1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d280096c0 tx=0x7f7d280315b0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 msgr2=0x7f7d381a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 0x7f7d381a15d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009fd0 tx=0x7f7d3000ef10 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 shutdown_connections 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d38109980 0x7f7d381a15d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d38076230 0x7f7d3819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 --2- 172.21.15.59:0/283880482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38074c50 0x7f7d3819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 >> 172.21.15.59:0/283880482 conn(0x7f7d380fd2b0 msgr2=0x7f7d381058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 shutdown_connections 2024-09-05T11:56:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:55.955+0000 7f7d3e000640 1 -- 172.21.15.59:0/283880482 wait complete. 2024-09-05T11:56:55.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:55 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/746982354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:55.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T11:56:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:55 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/746982354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:56.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1952219807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c1058d0 msgr2=0x7fe19c10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1952219807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c1058d0 0x7fe19c10c160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe194009ca0 tx=0x7fe19402a8b0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1952219807 shutdown_connections 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1952219807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c1058d0 0x7fe19c10c160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1952219807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c104f10 0x7fe19c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1952219807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c103d20 0x7fe19c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1952219807 >> 172.21.15.59:0/1952219807 conn(0x7fe19c0ff4a0 msgr2=0x7fe19c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1952219807 shutdown_connections 2024-09-05T11:56:56.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.133+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1952219807 wait complete. 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 Processor -- start 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 -- start start 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c103d20 0x7fe19c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 0x7fe19c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f980 con 0x7fe19c104f10 2024-09-05T11:56:56.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f800 con 0x7fe19c103d20 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10fb00 con 0x7fe19c1058d0 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe193e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe193e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51460/0 (socket says 172.21.15.59:51460) 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe193e00640 1 -- 172.21.15.59:0/1567939339 learned_addr learned my addr 172.21.15.59:0/1567939339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe1a0e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 0x7fe19c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.134+0000 7fe19be00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c103d20 0x7fe19c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 -- 172.21.15.59:0/1567939339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 msgr2=0x7fe19c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 0x7fe19c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 -- 172.21.15.59:0/1567939339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c103d20 msgr2=0x7fe19c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c103d20 0x7fe19c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 -- 172.21.15.59:0/1567939339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c1a1bd0 con 0x7fe19c104f10 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe1a0e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 0x7fe19c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe193e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e9f0 tx=0x7fe18400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18400cd80 con 0x7fe19c104f10 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe184010910 con 0x7fe19c104f10 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1a1ec0 con 0x7fe19c104f10 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.135+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840188f0 con 0x7fe19c104f10 2024-09-05T11:56:56.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.136+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a2380 con 0x7fe19c104f10 2024-09-05T11:56:56.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.137+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c10b0c0 con 0x7fe19c104f10 2024-09-05T11:56:56.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.138+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe184010ac0 con 0x7fe19c104f10 2024-09-05T11:56:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.140+0000 7fe199600640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 0x7fe1800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.140+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe1840cf8b0 con 0x7fe19c104f10 2024-09-05T11:56:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.140+0000 7fe19be00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 0x7fe1800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.141+0000 7fe19be00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 0x7fe1800a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe19c104d70 tx=0x7fe18c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.144+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe18409c400 con 0x7fe19c104f10 2024-09-05T11:56:56.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:55 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/746982354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:56:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.329+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe19c062e90 con 0x7fe19c104f10 2024-09-05T11:56:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.330+0000 7fe199600640 1 -- 172.21.15.59:0/1567939339 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe18409bd00 con 0x7fe19c104f10 2024-09-05T11:56:56.332 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-09-05T11:56:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 msgr2=0x7fe1800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 0x7fe1800a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe19c104d70 tx=0x7fe18c005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 msgr2=0x7fe19c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e9f0 tx=0x7fe18400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 shutdown_connections 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe1800a2b90 0x7fe1800a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c1058d0 0x7fe19c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c104f10 0x7fe19c19a4f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 --2- 172.21.15.59:0/1567939339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe19c103d20 0x7fe19c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 >> 172.21.15.59:0/1567939339 conn(0x7fe19c0ff4a0 msgr2=0x7fe19c108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 shutdown_connections 2024-09-05T11:56:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.332+0000 7fe1a2e00640 1 -- 172.21.15.59:0/1567939339 wait complete. 2024-09-05T11:56:56.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 -- 172.21.15.59:0/3941687571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980103c10 msgr2=0x7f6980104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 --2- 172.21.15.59:0/3941687571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980103c10 0x7f6980104010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6970009860 tx=0x7f697002a340 comp rx=0 tx=0).stop 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 -- 172.21.15.59:0/3941687571 shutdown_connections 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 --2- 172.21.15.59:0/3941687571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 0x7f698010c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 --2- 172.21.15.59:0/3941687571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980104e00 0x7f6980105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 --2- 172.21.15.59:0/3941687571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980103c10 0x7f6980104010 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 -- 172.21.15.59:0/3941687571 >> 172.21.15.59:0/3941687571 conn(0x7f69800ff390 msgr2=0x7f69801017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:56.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 -- 172.21.15.59:0/3941687571 shutdown_connections 2024-09-05T11:56:56.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.509+0000 7f697f200640 1 -- 172.21.15.59:0/3941687571 wait complete. 2024-09-05T11:56:56.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 Processor -- start 2024-09-05T11:56:56.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 -- start start 2024-09-05T11:56:56.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980104e00 0x7f698019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 0x7f69801a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f698010f890 con 0x7f6980103c10 2024-09-05T11:56:56.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f698010f710 con 0x7f69801057c0 2024-09-05T11:56:56.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f698010fa10 con 0x7f6980104e00 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51476/0 (socket says 172.21.15.59:51476) 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.510+0000 7f697de00640 1 -- 172.21.15.59:0/1596725138 learned_addr learned my addr 172.21.15.59:0/1596725138 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697e800640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 0x7f69801a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697d400640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980104e00 0x7f698019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 -- 172.21.15.59:0/1596725138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980104e00 msgr2=0x7f698019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980104e00 0x7f698019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 -- 172.21.15.59:0/1596725138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 msgr2=0x7f69801a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 0x7f69801a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 -- 172.21.15.59:0/1596725138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a1b40 con 0x7f6980103c10 2024-09-05T11:56:56.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697de00640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f69700305b0 tx=0x7f6970033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697003f020 con 0x7f6980103c10 2024-09-05T11:56:56.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970030850 con 0x7f6980103c10 2024-09-05T11:56:56.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a1dd0 con 0x7f6980103c10 2024-09-05T11:56:56.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970037040 con 0x7f6980103c10 2024-09-05T11:56:56.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.511+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a2200 con 0x7f6980103c10 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.513+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698010afa0 con 0x7f6980103c10 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.514+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6970042020 con 0x7f6980103c10 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.515+0000 7f6976a00640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.516+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69700ef630 con 0x7f6980103c10 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.516+0000 7f697d400640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.516+0000 7f697d400640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 0x7f69640a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f698019b430 tx=0x7f696c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.519+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6970049030 con 0x7f6980103c10 2024-09-05T11:56:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.704+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6980062df0 con 0x7f6980103c10 2024-09-05T11:56:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.705+0000 7f6976a00640 1 -- 172.21.15.59:0/1596725138 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f69700bc120 con 0x7f6980103c10 2024-09-05T11:56:56.707 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 msgr2=0x7f69640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 0x7f69640a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f698019b430 tx=0x7f696c009260 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 msgr2=0x7f6980199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f69700305b0 tx=0x7f6970033d90 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 shutdown_connections 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69640a2b90 0x7f69640a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69801057c0 0x7f69801a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6980104e00 0x7f698019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 --2- 172.21.15.59:0/1596725138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6980103c10 0x7f6980199f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 >> 172.21.15.59:0/1596725138 conn(0x7f69800ff390 msgr2=0x7f6980108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 shutdown_connections 2024-09-05T11:56:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.707+0000 7f697f200640 1 -- 172.21.15.59:0/1596725138 wait complete. 2024-09-05T11:56:56.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 -- 172.21.15.59:0/3148376682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8103d20 msgr2=0x7fcde8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 --2- 172.21.15.59:0/3148376682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8103d20 0x7fcde8104120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009900 tx=0x7fcdd802a410 comp rx=0 tx=0).stop 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 -- 172.21.15.59:0/3148376682 shutdown_connections 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 --2- 172.21.15.59:0/3148376682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 --2- 172.21.15.59:0/3148376682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8104f10 0x7fcde8105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 --2- 172.21.15.59:0/3148376682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8103d20 0x7fcde8104120 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 -- 172.21.15.59:0/3148376682 >> 172.21.15.59:0/3148376682 conn(0x7fcde80ff4a0 msgr2=0x7fcde81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 -- 172.21.15.59:0/3148376682 shutdown_connections 2024-09-05T11:56:56.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.886+0000 7fcdee800640 1 -- 172.21.15.59:0/3148376682 wait complete. 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.887+0000 7fcdee800640 1 Processor -- start 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.887+0000 7fcdee800640 1 -- start start 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 0x7fcde819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810f9d0 con 0x7fcde8104f10 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810f850 con 0x7fcde81058d0 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcdee800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810fb50 con 0x7fcde8103d20 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:60514/0 (socket says 172.21.15.59:60514) 2024-09-05T11:56:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7400640 1 -- 172.21.15.59:0/4090317301 learned_addr learned my addr 172.21.15.59:0/4090317301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde6a00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 0x7fcde819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7e00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7e00640 1 -- 172.21.15.59:0/4090317301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 msgr2=0x7fcde819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.888+0000 7fcde7e00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde7e00640 1 -- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 msgr2=0x7fcde819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde7e00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 0x7fcde819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde7e00640 1 -- 172.21.15.59:0/4090317301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde81a1cd0 con 0x7fcde81058d0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde6a00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 0x7fcde819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde7400640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcde7e00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde81a1660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009fd0 tx=0x7fcde000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde000ce20 con 0x7fcde81058d0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde0005020 con 0x7fcde81058d0 2024-09-05T11:56:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde81a1fc0 con 0x7fcde81058d0 2024-09-05T11:56:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0010720 con 0x7fcde81058d0 2024-09-05T11:56:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.889+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde81a2430 con 0x7fcde81058d0 2024-09-05T11:56:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.890+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde810b110 con 0x7fcde81058d0 2024-09-05T11:56:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.892+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcde0005540 con 0x7fcde81058d0 2024-09-05T11:56:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.893+0000 7fcddfe00640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 0x7fcdcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.893+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcde00cfbc0 con 0x7fcde81058d0 2024-09-05T11:56:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.894+0000 7fcde7400640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 0x7fcdcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.894+0000 7fcde7400640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 0x7fcdcc0a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:56.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:56.897+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcde009c6b0 con 0x7fcde81058d0 2024-09-05T11:56:57.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.079+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcde8062e90 con 0x7fcde81058d0 2024-09-05T11:56:57.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.080+0000 7fcddfe00640 1 -- 172.21.15.59:0/4090317301 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fcde009bfb0 con 0x7fcde81058d0 2024-09-05T11:56:57.083 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-09-05T11:56:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 msgr2=0x7fcdcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 0x7fcdcc0a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80315b0 comp rx=0 tx=0).stop 2024-09-05T11:56:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 msgr2=0x7fcde81a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde81a1660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009fd0 tx=0x7fcde000ef10 comp rx=0 tx=0).stop 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 shutdown_connections 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.082+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcdcc0a2b90 0x7fcdcc0a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcde81058d0 0x7fcde81a1660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde8104f10 0x7fcde819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 --2- 172.21.15.59:0/4090317301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcde8103d20 0x7fcde819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 >> 172.21.15.59:0/4090317301 conn(0x7fcde80ff4a0 msgr2=0x7fcde81082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 shutdown_connections 2024-09-05T11:56:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:57.083+0000 7fcdee800640 1 -- 172.21.15.59:0/4090317301 wait complete. 2024-09-05T11:56:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:56 smithi189 ceph-mon[34772]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:56 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/283880482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:56 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1567939339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:56 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1596725138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:56 smithi059 ceph-mon[30025]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:57.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:56 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/283880482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:56 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1567939339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:56 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1596725138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.102 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-05T11:56:57.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:56:57.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:56 smithi089 ceph-mon[34499]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-05T11:56:57.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:56 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/283880482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:56 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1567939339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:56 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1596725138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T11:56:57.483 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:56:57.922 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:57 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4090317301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T11:56:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 -- 172.21.15.59:0/1244147705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60112b90 msgr2=0x7f0b60114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 --2- 172.21.15.59:0/1244147705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60112b90 0x7f0b60114f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009ca0 tx=0x7f0b5802a950 comp rx=0 tx=0).stop 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 -- 172.21.15.59:0/1244147705 shutdown_connections 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 --2- 172.21.15.59:0/1244147705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60112b90 0x7f0b60114f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 --2- 172.21.15.59:0/1244147705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60074eb0 0x7f0b60112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 --2- 172.21.15.59:0/1244147705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074590 0x7f0b60074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 -- 172.21.15.59:0/1244147705 >> 172.21.15.59:0/1244147705 conn(0x7f0b600fd8c0 msgr2=0x7f0b600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 -- 172.21.15.59:0/1244147705 shutdown_connections 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.034+0000 7f0b65c00640 1 -- 172.21.15.59:0/1244147705 wait complete. 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.035+0000 7f0b65c00640 1 Processor -- start 2024-09-05T11:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.035+0000 7f0b65c00640 1 -- start start 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 0x7f0b6019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074eb0 0x7f0b6019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 0x7f0b601a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601185c0 con 0x7f0b60074590 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60118440 con 0x7f0b60074eb0 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b65c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60118740 con 0x7f0b60112b90 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 0x7f0b6019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 0x7f0b601a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 0x7f0b6019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5ea00640 1 -- 172.21.15.59:0/441305529 learned_addr learned my addr 172.21.15.59:0/441305529 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b57e00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074eb0 0x7f0b6019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 -- 172.21.15.59:0/441305529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074eb0 msgr2=0x7f0b6019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074eb0 0x7f0b6019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 -- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 msgr2=0x7f0b6019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 0x7f0b6019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.036+0000 7f0b5f400640 1 -- 172.21.15.59:0/441305529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b601a3b40 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b5f400640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 0x7f0b601a3410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b5802af40 tx=0x7f0b58003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58034040 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b601a3dd0 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b5802f940 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580045a0 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.037+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601ab6c0 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.038+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b60074970 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.040+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b58037020 con 0x7f0b60112b90 2024-09-05T11:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.041+0000 7f0b57400640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 0x7f0b440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.041+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0b580f0170 con 0x7f0b60112b90 2024-09-05T11:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.042+0000 7f0b5ea00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 0x7f0b440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.042+0000 7f0b5ea00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 0x7f0b440a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b500095f0 tx=0x7f0b50005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.045+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b580bcc60 con 0x7f0b60112b90 2024-09-05T11:56:58.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:57 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4090317301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T11:56:58.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:57 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4090317301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T11:56:58.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.272+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b60104fd0 con 0x7f0b60112b90 2024-09-05T11:56:58.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.273+0000 7f0b57400640 1 -- 172.21.15.59:0/441305529 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f0b580bc560 con 0x7f0b60112b90 2024-09-05T11:56:58.276 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:58.276 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:56:58.276 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 msgr2=0x7f0b440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 0x7f0b440a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b500095f0 tx=0x7f0b50005f90 comp rx=0 tx=0).stop 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 msgr2=0x7f0b601a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 0x7f0b601a3410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b5802af40 tx=0x7f0b58003b80 comp rx=0 tx=0).stop 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 shutdown_connections 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b440a2b80 0x7f0b440a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b60112b90 0x7f0b601a3410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b60074eb0 0x7f0b6019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 --2- 172.21.15.59:0/441305529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b60074590 0x7f0b6019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.275+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 >> 172.21.15.59:0/441305529 conn(0x7f0b600fd8c0 msgr2=0x7f0b600ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.276+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 shutdown_connections 2024-09-05T11:56:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:58.276+0000 7f0b65c00640 1 -- 172.21.15.59:0/441305529 wait complete. 2024-09-05T11:56:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:58 smithi059 ceph-mon[30025]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T11:56:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:56:58 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/441305529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:59.006 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-05T11:56:59.006 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-05T11:56:59.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T11:56:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:58 smithi189 ceph-mon[34772]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T11:56:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:56:58 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/441305529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 -- 172.21.15.59:0/417214924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc103d30 msgr2=0x7fdcbc104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 --2- 172.21.15.59:0/417214924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc103d30 0x7fdcbc104130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac02a410 comp rx=0 tx=0).stop 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 -- 172.21.15.59:0/417214924 shutdown_connections 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 --2- 172.21.15.59:0/417214924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 --2- 172.21.15.59:0/417214924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc104f20 0x7fdcbc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 --2- 172.21.15.59:0/417214924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc103d30 0x7fdcbc104130 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 -- 172.21.15.59:0/417214924 >> 172.21.15.59:0/417214924 conn(0x7fdcbc0ff4b0 msgr2=0x7fdcbc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 -- 172.21.15.59:0/417214924 shutdown_connections 2024-09-05T11:56:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.164+0000 7fdcc2400640 1 -- 172.21.15.59:0/417214924 wait complete. 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 Processor -- start 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 -- start start 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc103d30 0x7fdcbc19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 0x7fdcbc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc10f990 con 0x7fdcbc1058e0 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc10f810 con 0x7fdcbc104f20 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcc2400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc10fb10 con 0x7fdcbc103d30 2024-09-05T11:56:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.165+0000 7fdcbbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 0x7fdcbc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcbb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc103d30 0x7fdcbc19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcbbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51516/0 (socket says 172.21.15.59:51516) 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcbbe00640 1 -- 172.21.15.59:0/3783732532 learned_addr learned my addr 172.21.15.59:0/3783732532 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 -- 172.21.15.59:0/3783732532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc103d30 msgr2=0x7fdcbc19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc103d30 0x7fdcbc19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 -- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 msgr2=0x7fdcbc1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 -- 172.21.15.59:0/3783732532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc1a1c60 con 0x7fdcbc104f20 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcbbe00640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.166+0000 7fdcb3e00640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 0x7fdcbc19a580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e9f0 tx=0x7fdca400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.167+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca400cd80 con 0x7fdcbc104f20 2024-09-05T11:56:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.167+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca40035f0 con 0x7fdcbc104f20 2024-09-05T11:56:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.167+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc1a1f50 con 0x7fdcbc104f20 2024-09-05T11:56:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.167+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4010670 con 0x7fdcbc104f20 2024-09-05T11:56:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.167+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc1a2410 con 0x7fdcbc104f20 2024-09-05T11:56:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.168+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc10b120 con 0x7fdcbc104f20 2024-09-05T11:56:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.170+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdca40037a0 con 0x7fdcbc104f20 2024-09-05T11:56:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.171+0000 7fdcb8c00640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 0x7fdca00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.171+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdca40cf780 con 0x7fdcbc104f20 2024-09-05T11:56:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.171+0000 7fdcbb400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 0x7fdca00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.172+0000 7fdcbb400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 0x7fdca00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcac005ab0 tx=0x7fdcac000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.173+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdca409c300 con 0x7fdcbc104f20 2024-09-05T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:58 smithi089 ceph-mon[34499]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:56:58 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/441305529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:56:59.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.355+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcbc062630 con 0x7fdca00a2b90 2024-09-05T11:56:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.365+0000 7fdcb8c00640 1 -- 172.21.15.59:0/3783732532 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7fdcbc062630 con 0x7fdca00a2b90 2024-09-05T11:56:59.372 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:59.374 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.369+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 msgr2=0x7fdca00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.369+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 0x7fdca00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcac005ab0 tx=0x7fdcac000bc0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 msgr2=0x7fdcbc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 0x7fdcbc19a580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e9f0 tx=0x7fdca400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 shutdown_connections 2024-09-05T11:56:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdca00a2b90 0x7fdca00a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcbc1058e0 0x7fdcbc1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcbc104f20 0x7fdcbc19a580 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 --2- 172.21.15.59:0/3783732532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcbc103d30 0x7fdcbc19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 >> 172.21.15.59:0/3783732532 conn(0x7fdcbc0ff4b0 msgr2=0x7fdcbc1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 shutdown_connections 2024-09-05T11:56:59.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.370+0000 7fdcc2400640 1 -- 172.21.15.59:0/3783732532 wait complete. 2024-09-05T11:56:59.388 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-05T11:56:58.888984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":346,"num_read_kb":286,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376224,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565600,"statfs":{"total":1151940427776,"available":1151555174400,"internally_reserved":0,"allocated":14843904,"data_stored":8352941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5933,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003750"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578235+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:19.578235+0000","last_peered":"2024-09-05T11:56:19.578235+0000","last_clean":"2024-09-05T11:56:19.578235+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:19.578235+0000","last_undegraded":"2024-09-05T11:56:19.578235+0000","last_fullsized":"2024-09-05T11:56:19.578235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600094+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600094+0000","last_peered":"2024-09-05T11:56:19.600094+0000","last_clean":"2024-09-05T11:56:19.600094+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600094+0000","last_undegraded":"2024-09-05T11:56:19.600094+0000","last_fullsized":"2024-09-05T11:56:19.600094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577064+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577064+0000","last_peered":"2024-09-05T11:56:19.577064+0000","last_clean":"2024-09-05T11:56:19.577064+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577064+0000","last_undegraded":"2024-09-05T11:56:19.577064+0000","last_fullsized":"2024-09-05T11:56:19.577064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581360+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581360+0000","last_peered":"2024-09-05T11:56:19.581360+0000","last_clean":"2024-09-05T11:56:19.581360+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581360+0000","last_undegraded":"2024-09-05T11:56:19.581360+0000","last_fullsized":"2024-09-05T11:56:19.581360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874855+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874855+0000","last_peered":"2024-09-05T11:56:46.874855+0000","last_clean":"2024-09-05T11:56:46.874855+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874855+0000","last_undegraded":"2024-09-05T11:56:46.874855+0000","last_fullsized":"2024-09-05T11:56:46.874855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581211+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581211+0000","last_peered":"2024-09-05T11:56:19.581211+0000","last_clean":"2024-09-05T11:56:19.581211+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581211+0000","last_undegraded":"2024-09-05T11:56:19.581211+0000","last_fullsized":"2024-09-05T11:56:19.581211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871296+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.871296+0000","last_peered":"2024-09-05T11:56:46.871296+0000","last_clean":"2024-09-05T11:56:46.871296+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.871296+0000","last_undegraded":"2024-09-05T11:56:46.871296+0000","last_fullsized":"2024-09-05T11:56:46.871296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574659+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574659+0000","last_peered":"2024-09-05T11:56:19.574659+0000","last_clean":"2024-09-05T11:56:19.574659+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574659+0000","last_undegraded":"2024-09-05T11:56:19.574659+0000","last_fullsized":"2024-09-05T11:56:19.574659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578142+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578142+0000","last_peered":"2024-09-05T11:56:19.578142+0000","last_clean":"2024-09-05T11:56:19.578142+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578142+0000","last_undegraded":"2024-09-05T11:56:19.578142+0000","last_fullsized":"2024-09-05T11:56:19.578142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601515+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601515+0000","last_peered":"2024-09-05T11:56:19.601515+0000","last_clean":"2024-09-05T11:56:19.601515+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601515+0000","last_undegraded":"2024-09-05T11:56:19.601515+0000","last_fullsized":"2024-09-05T11:56:19.601515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575209+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575209+0000","last_peered":"2024-09-05T11:56:19.575209+0000","last_clean":"2024-09-05T11:56:19.575209+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575209+0000","last_undegraded":"2024-09-05T11:56:19.575209+0000","last_fullsized":"2024-09-05T11:56:19.575209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":293,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:54.648792+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:54.648792+0000","last_peered":"2024-09-05T11:56:54.648792+0000","last_clean":"2024-09-05T11:56:54.648792+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:54.648792+0000","last_undegraded":"2024-09-05T11:56:54.648792+0000","last_fullsized":"2024-09-05T11:56:54.648792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":171,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578591+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578591+0000","last_peered":"2024-09-05T11:56:19.578591+0000","last_clean":"2024-09-05T11:56:19.578591+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578591+0000","last_undegraded":"2024-09-05T11:56:19.578591+0000","last_fullsized":"2024-09-05T11:56:19.578591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579886+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579886+0000","last_peered":"2024-09-05T11:56:19.579886+0000","last_clean":"2024-09-05T11:56:19.579886+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579886+0000","last_undegraded":"2024-09-05T11:56:19.579886+0000","last_fullsized":"2024-09-05T11:56:19.579886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869797+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.869797+0000","last_peered":"2024-09-05T11:56:46.869797+0000","last_clean":"2024-09-05T11:56:46.869797+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.869797+0000","last_undegraded":"2024-09-05T11:56:46.869797+0000","last_fullsized":"2024-09-05T11:56:46.869797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602278+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602278+0000","last_peered":"2024-09-05T11:56:19.602278+0000","last_clean":"2024-09-05T11:56:19.602278+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602278+0000","last_undegraded":"2024-09-05T11:56:19.602278+0000","last_fullsized":"2024-09-05T11:56:19.602278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868664+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868664+0000","last_peered":"2024-09-05T11:56:46.868664+0000","last_clean":"2024-09-05T11:56:46.868664+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868664+0000","last_undegraded":"2024-09-05T11:56:46.868664+0000","last_fullsized":"2024-09-05T11:56:46.868664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580692+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.580692+0000","last_peered":"2024-09-05T11:56:19.580692+0000","last_clean":"2024-09-05T11:56:19.580692+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.580692+0000","last_undegraded":"2024-09-05T11:56:19.580692+0000","last_fullsized":"2024-09-05T11:56:19.580692+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.860067+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.860067+0000","last_peered":"2024-09-05T11:56:46.860067+0000","last_clean":"2024-09-05T11:56:46.860067+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.860067+0000","last_undegraded":"2024-09-05T11:56:46.860067+0000","last_fullsized":"2024-09-05T11:56:46.860067+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871492+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871492+0000","last_peered":"2024-09-05T11:56:46.871492+0000","last_clean":"2024-09-05T11:56:46.871492+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871492+0000","last_undegraded":"2024-09-05T11:56:46.871492+0000","last_fullsized":"2024-09-05T11:56:46.871492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872755+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.872755+0000","last_peered":"2024-09-05T11:56:46.872755+0000","last_clean":"2024-09-05T11:56:46.872755+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.872755+0000","last_undegraded":"2024-09-05T11:56:46.872755+0000","last_fullsized":"2024-09-05T11:56:46.872755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870375+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870375+0000","last_peered":"2024-09-05T11:56:46.870375+0000","last_clean":"2024-09-05T11:56:46.870375+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870375+0000","last_undegraded":"2024-09-05T11:56:46.870375+0000","last_fullsized":"2024-09-05T11:56:46.870375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873576+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873576+0000","last_peered":"2024-09-05T11:56:46.873576+0000","last_clean":"2024-09-05T11:56:46.873576+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873576+0000","last_undegraded":"2024-09-05T11:56:46.873576+0000","last_fullsized":"2024-09-05T11:56:46.873576+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870916+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.870916+0000","last_peered":"2024-09-05T11:56:46.870916+0000","last_clean":"2024-09-05T11:56:46.870916+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.870916+0000","last_undegraded":"2024-09-05T11:56:46.870916+0000","last_fullsized":"2024-09-05T11:56:46.870916+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868242+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868242+0000","last_peered":"2024-09-05T11:56:46.868242+0000","last_clean":"2024-09-05T11:56:46.868242+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868242+0000","last_undegraded":"2024-09-05T11:56:46.868242+0000","last_fullsized":"2024-09-05T11:56:46.868242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":220,"num_read_kb":177,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1188587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:56:59.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 -- 172.21.15.59:0/3682520585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568104ee0 msgr2=0x7fd5681072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 --2- 172.21.15.59:0/3682520585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568104ee0 0x7fd5681072d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd554009860 tx=0x7fd55402a340 comp rx=0 tx=0).stop 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 -- 172.21.15.59:0/3682520585 shutdown_connections 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 --2- 172.21.15.59:0/3682520585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd568107810 0x7fd568109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 --2- 172.21.15.59:0/3682520585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568104ee0 0x7fd5681072d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 --2- 172.21.15.59:0/3682520585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5681025b0 0x7fd5681049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 -- 172.21.15.59:0/3682520585 >> 172.21.15.59:0/3682520585 conn(0x7fd5680fc1b0 msgr2=0x7fd5680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:59.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 -- 172.21.15.59:0/3682520585 shutdown_connections 2024-09-05T11:56:59.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.586+0000 7fd56f800640 1 -- 172.21.15.59:0/3682520585 wait complete. 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.587+0000 7fd56f800640 1 Processor -- start 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.587+0000 7fd56f800640 1 -- start start 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.587+0000 7fd56f800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.587+0000 7fd56f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd568104ee0 0x7fd568100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56f800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568107810 0x7fd568100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810d7f0 con 0x7fd568104ee0 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56f800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810d670 con 0x7fd5681025b0 2024-09-05T11:56:59.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56f800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd56810d970 con 0x7fd568107810 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49260/0 (socket says 172.21.15.59:49260) 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 -- 172.21.15.59:0/2354385021 learned_addr learned my addr 172.21.15.59:0/2354385021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd567e00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd568104ee0 0x7fd568100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56d800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568107810 0x7fd568100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 -- 172.21.15.59:0/2354385021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568107810 msgr2=0x7fd568100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568107810 0x7fd568100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 -- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd568104ee0 msgr2=0x7fd568100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd568104ee0 0x7fd568100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 -- 172.21.15.59:0/2354385021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd568101330 con 0x7fd5681025b0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.588+0000 7fd56ce00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00e9c0 tx=0x7fd55c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.589+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00cde0 con 0x7fd5681025b0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.589+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c0035f0 con 0x7fd5681025b0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.589+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681a1ff0 con 0x7fd5681025b0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.589+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c010690 con 0x7fd5681025b0 2024-09-05T11:56:59.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.589+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5681a2450 con 0x7fd5681025b0 2024-09-05T11:56:59.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.590+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5680fdca0 con 0x7fd5681025b0 2024-09-05T11:56:59.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.592+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd55c0037a0 con 0x7fd5681025b0 2024-09-05T11:56:59.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.593+0000 7fd565600640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:56:59.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.593+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd55c0cf8c0 con 0x7fd5681025b0 2024-09-05T11:56:59.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.593+0000 7fd567e00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:56:59.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.594+0000 7fd567e00640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 0x7fd5500a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd55402a930 tx=0x7fd554030570 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:56:59.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.595+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd55c09c440 con 0x7fd5681025b0 2024-09-05T11:56:59.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.780+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd568062630 con 0x7fd5500a2b90 2024-09-05T11:56:59.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.788+0000 7fd565600640 1 -- 172.21.15.59:0/2354385021 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7fd568062630 con 0x7fd5500a2b90 2024-09-05T11:56:59.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:56:59.799 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 msgr2=0x7fd5500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 0x7fd5500a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd55402a930 tx=0x7fd554030570 comp rx=0 tx=0).stop 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 msgr2=0x7fd568101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00e9c0 tx=0x7fd55c00ee80 comp rx=0 tx=0).stop 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 shutdown_connections 2024-09-05T11:56:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd568107810 0x7fd568100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd568104ee0 0x7fd568100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 --2- 172.21.15.59:0/2354385021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5681025b0 0x7fd568101f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 >> 172.21.15.59:0/2354385021 conn(0x7fd5680fc1b0 msgr2=0x7fd568102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 shutdown_connections 2024-09-05T11:56:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:56:59.793+0000 7fd56f800640 1 -- 172.21.15.59:0/2354385021 wait complete. 2024-09-05T11:56:59.812 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-05T11:56:58.888984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":346,"num_read_kb":286,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376224,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565600,"statfs":{"total":1151940427776,"available":1151555174400,"internally_reserved":0,"allocated":14843904,"data_stored":8352941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5933,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003750"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578235+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:19.578235+0000","last_peered":"2024-09-05T11:56:19.578235+0000","last_clean":"2024-09-05T11:56:19.578235+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:19.578235+0000","last_undegraded":"2024-09-05T11:56:19.578235+0000","last_fullsized":"2024-09-05T11:56:19.578235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600094+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600094+0000","last_peered":"2024-09-05T11:56:19.600094+0000","last_clean":"2024-09-05T11:56:19.600094+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600094+0000","last_undegraded":"2024-09-05T11:56:19.600094+0000","last_fullsized":"2024-09-05T11:56:19.600094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577064+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577064+0000","last_peered":"2024-09-05T11:56:19.577064+0000","last_clean":"2024-09-05T11:56:19.577064+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577064+0000","last_undegraded":"2024-09-05T11:56:19.577064+0000","last_fullsized":"2024-09-05T11:56:19.577064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581360+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581360+0000","last_peered":"2024-09-05T11:56:19.581360+0000","last_clean":"2024-09-05T11:56:19.581360+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581360+0000","last_undegraded":"2024-09-05T11:56:19.581360+0000","last_fullsized":"2024-09-05T11:56:19.581360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874855+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874855+0000","last_peered":"2024-09-05T11:56:46.874855+0000","last_clean":"2024-09-05T11:56:46.874855+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874855+0000","last_undegraded":"2024-09-05T11:56:46.874855+0000","last_fullsized":"2024-09-05T11:56:46.874855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581211+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581211+0000","last_peered":"2024-09-05T11:56:19.581211+0000","last_clean":"2024-09-05T11:56:19.581211+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581211+0000","last_undegraded":"2024-09-05T11:56:19.581211+0000","last_fullsized":"2024-09-05T11:56:19.581211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871296+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.871296+0000","last_peered":"2024-09-05T11:56:46.871296+0000","last_clean":"2024-09-05T11:56:46.871296+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.871296+0000","last_undegraded":"2024-09-05T11:56:46.871296+0000","last_fullsized":"2024-09-05T11:56:46.871296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574659+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574659+0000","last_peered":"2024-09-05T11:56:19.574659+0000","last_clean":"2024-09-05T11:56:19.574659+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574659+0000","last_undegraded":"2024-09-05T11:56:19.574659+0000","last_fullsized":"2024-09-05T11:56:19.574659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578142+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578142+0000","last_peered":"2024-09-05T11:56:19.578142+0000","last_clean":"2024-09-05T11:56:19.578142+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578142+0000","last_undegraded":"2024-09-05T11:56:19.578142+0000","last_fullsized":"2024-09-05T11:56:19.578142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601515+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601515+0000","last_peered":"2024-09-05T11:56:19.601515+0000","last_clean":"2024-09-05T11:56:19.601515+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601515+0000","last_undegraded":"2024-09-05T11:56:19.601515+0000","last_fullsized":"2024-09-05T11:56:19.601515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575209+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575209+0000","last_peered":"2024-09-05T11:56:19.575209+0000","last_clean":"2024-09-05T11:56:19.575209+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575209+0000","last_undegraded":"2024-09-05T11:56:19.575209+0000","last_fullsized":"2024-09-05T11:56:19.575209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":293,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:54.648792+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:54.648792+0000","last_peered":"2024-09-05T11:56:54.648792+0000","last_clean":"2024-09-05T11:56:54.648792+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:54.648792+0000","last_undegraded":"2024-09-05T11:56:54.648792+0000","last_fullsized":"2024-09-05T11:56:54.648792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":171,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578591+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578591+0000","last_peered":"2024-09-05T11:56:19.578591+0000","last_clean":"2024-09-05T11:56:19.578591+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578591+0000","last_undegraded":"2024-09-05T11:56:19.578591+0000","last_fullsized":"2024-09-05T11:56:19.578591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579886+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579886+0000","last_peered":"2024-09-05T11:56:19.579886+0000","last_clean":"2024-09-05T11:56:19.579886+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579886+0000","last_undegraded":"2024-09-05T11:56:19.579886+0000","last_fullsized":"2024-09-05T11:56:19.579886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869797+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.869797+0000","last_peered":"2024-09-05T11:56:46.869797+0000","last_clean":"2024-09-05T11:56:46.869797+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.869797+0000","last_undegraded":"2024-09-05T11:56:46.869797+0000","last_fullsized":"2024-09-05T11:56:46.869797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602278+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602278+0000","last_peered":"2024-09-05T11:56:19.602278+0000","last_clean":"2024-09-05T11:56:19.602278+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602278+0000","last_undegraded":"2024-09-05T11:56:19.602278+0000","last_fullsized":"2024-09-05T11:56:19.602278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868664+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868664+0000","last_peered":"2024-09-05T11:56:46.868664+0000","last_clean":"2024-09-05T11:56:46.868664+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868664+0000","last_undegraded":"2024-09-05T11:56:46.868664+0000","last_fullsized":"2024-09-05T11:56:46.868664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580692+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.580692+0000","last_peered":"2024-09-05T11:56:19.580692+0000","last_clean":"2024-09-05T11:56:19.580692+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.580692+0000","last_undegraded":"2024-09-05T11:56:19.580692+0000","last_fullsized":"2024-09-05T11:56:19.580692+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.860067+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.860067+0000","last_peered":"2024-09-05T11:56:46.860067+0000","last_clean":"2024-09-05T11:56:46.860067+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.860067+0000","last_undegraded":"2024-09-05T11:56:46.860067+0000","last_fullsized":"2024-09-05T11:56:46.860067+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871492+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871492+0000","last_peered":"2024-09-05T11:56:46.871492+0000","last_clean":"2024-09-05T11:56:46.871492+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871492+0000","last_undegraded":"2024-09-05T11:56:46.871492+0000","last_fullsized":"2024-09-05T11:56:46.871492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872755+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.872755+0000","last_peered":"2024-09-05T11:56:46.872755+0000","last_clean":"2024-09-05T11:56:46.872755+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.872755+0000","last_undegraded":"2024-09-05T11:56:46.872755+0000","last_fullsized":"2024-09-05T11:56:46.872755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870375+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870375+0000","last_peered":"2024-09-05T11:56:46.870375+0000","last_clean":"2024-09-05T11:56:46.870375+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870375+0000","last_undegraded":"2024-09-05T11:56:46.870375+0000","last_fullsized":"2024-09-05T11:56:46.870375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873576+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873576+0000","last_peered":"2024-09-05T11:56:46.873576+0000","last_clean":"2024-09-05T11:56:46.873576+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873576+0000","last_undegraded":"2024-09-05T11:56:46.873576+0000","last_fullsized":"2024-09-05T11:56:46.873576+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870916+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.870916+0000","last_peered":"2024-09-05T11:56:46.870916+0000","last_clean":"2024-09-05T11:56:46.870916+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.870916+0000","last_undegraded":"2024-09-05T11:56:46.870916+0000","last_fullsized":"2024-09-05T11:56:46.870916+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868242+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868242+0000","last_peered":"2024-09-05T11:56:46.868242+0000","last_clean":"2024-09-05T11:56:46.868242+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868242+0000","last_undegraded":"2024-09-05T11:56:46.868242+0000","last_fullsized":"2024-09-05T11:56:46.868242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":220,"num_read_kb":177,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1188587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:56:59.817 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-05T11:56:59.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T11:57:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.008+0000 7f2cb6400640 1 -- 172.21.15.59:0/954378986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0104d50 msgr2=0x7f2cb01051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 --2- 172.21.15.59:0/954378986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0104d50 0x7f2cb01051d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02a3f0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 -- 172.21.15.59:0/954378986 shutdown_connections 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 --2- 172.21.15.59:0/954378986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0105710 0x7f2cb010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 --2- 172.21.15.59:0/954378986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0104d50 0x7f2cb01051d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 --2- 172.21.15.59:0/954378986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 -- 172.21.15.59:0/954378986 >> 172.21.15.59:0/954378986 conn(0x7f2cb00ff2c0 msgr2=0x7f2cb0101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 -- 172.21.15.59:0/954378986 shutdown_connections 2024-09-05T11:57:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.009+0000 7f2cb6400640 1 -- 172.21.15.59:0/954378986 wait complete. 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 Processor -- start 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 -- start start 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0104d50 0x7f2cb019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0105710 0x7f2cb01a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010f810 con 0x7f2cb0103b60 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010f690 con 0x7f2cb0104d50 2024-09-05T11:57:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb6400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010f990 con 0x7f2cb0105710 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51548/0 (socket says 172.21.15.59:51548) 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb5000640 1 -- 172.21.15.59:0/837153381 learned_addr learned my addr 172.21.15.59:0/837153381 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cb5a00640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0105710 0x7f2cb01a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.010+0000 7f2cafe00640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0104d50 0x7f2cb019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 -- 172.21.15.59:0/837153381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0105710 msgr2=0x7f2cb01a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0105710 0x7f2cb01a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 -- 172.21.15.59:0/837153381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0104d50 msgr2=0x7f2cb019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0104d50 0x7f2cb019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 -- 172.21.15.59:0/837153381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01a1a90 con 0x7f2cb0103b60 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb5000640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e9c0 tx=0x7f2ca400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400cde0 con 0x7f2cb0103b60 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4010910 con 0x7f2cb0103b60 2024-09-05T11:57:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb01a1d80 con 0x7f2cb0103b60 2024-09-05T11:57:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4018a00 con 0x7f2cb0103b60 2024-09-05T11:57:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.011+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb01a21e0 con 0x7f2cb0103b60 2024-09-05T11:57:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.013+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb010af50 con 0x7f2cb0103b60 2024-09-05T11:57:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.014+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ca401f020 con 0x7f2cb0103b60 2024-09-05T11:57:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.016+0000 7f2cad600640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 0x7f2c980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.016+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2ca40cf670 con 0x7f2cb0103b60 2024-09-05T11:57:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.016+0000 7f2cafe00640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 0x7f2c980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.016+0000 7f2cafe00640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 0x7f2c980a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c031440 tx=0x7f2c9c0315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.020+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ca409c160 con 0x7f2cb0103b60 2024-09-05T11:57:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.202+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cb0062630 con 0x7f2c980a2b90 2024-09-05T11:57:00.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.210+0000 7f2cad600640 1 -- 172.21.15.59:0/837153381 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7f2cb0062630 con 0x7f2c980a2b90 2024-09-05T11:57:00.217 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:00.219 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:57:00.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 msgr2=0x7f2c980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:00.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 0x7f2c980a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c031440 tx=0x7f2c9c0315b0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 msgr2=0x7f2cb0199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e9c0 tx=0x7f2ca400ee80 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 shutdown_connections 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c980a2b90 0x7f2c980a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cb0105710 0x7f2cb01a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb0104d50 0x7f2cb019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 --2- 172.21.15.59:0/837153381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb0103b60 0x7f2cb0199e70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 >> 172.21.15.59:0/837153381 conn(0x7f2cb00ff2c0 msgr2=0x7f2cb0108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 shutdown_connections 2024-09-05T11:57:00.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:00.215+0000 7f2cb6400640 1 -- 172.21.15.59:0/837153381 wait complete. 2024-09-05T11:57:00.233 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-05T11:56:58.888984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":346,"num_read_kb":286,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376224,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565600,"statfs":{"total":1151940427776,"available":1151555174400,"internally_reserved":0,"allocated":14843904,"data_stored":8352941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5933,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003750"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578235+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:19.578235+0000","last_peered":"2024-09-05T11:56:19.578235+0000","last_clean":"2024-09-05T11:56:19.578235+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:19.578235+0000","last_undegraded":"2024-09-05T11:56:19.578235+0000","last_fullsized":"2024-09-05T11:56:19.578235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600094+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600094+0000","last_peered":"2024-09-05T11:56:19.600094+0000","last_clean":"2024-09-05T11:56:19.600094+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600094+0000","last_undegraded":"2024-09-05T11:56:19.600094+0000","last_fullsized":"2024-09-05T11:56:19.600094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577064+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577064+0000","last_peered":"2024-09-05T11:56:19.577064+0000","last_clean":"2024-09-05T11:56:19.577064+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577064+0000","last_undegraded":"2024-09-05T11:56:19.577064+0000","last_fullsized":"2024-09-05T11:56:19.577064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581360+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581360+0000","last_peered":"2024-09-05T11:56:19.581360+0000","last_clean":"2024-09-05T11:56:19.581360+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581360+0000","last_undegraded":"2024-09-05T11:56:19.581360+0000","last_fullsized":"2024-09-05T11:56:19.581360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874855+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874855+0000","last_peered":"2024-09-05T11:56:46.874855+0000","last_clean":"2024-09-05T11:56:46.874855+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874855+0000","last_undegraded":"2024-09-05T11:56:46.874855+0000","last_fullsized":"2024-09-05T11:56:46.874855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581211+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581211+0000","last_peered":"2024-09-05T11:56:19.581211+0000","last_clean":"2024-09-05T11:56:19.581211+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581211+0000","last_undegraded":"2024-09-05T11:56:19.581211+0000","last_fullsized":"2024-09-05T11:56:19.581211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871296+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.871296+0000","last_peered":"2024-09-05T11:56:46.871296+0000","last_clean":"2024-09-05T11:56:46.871296+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.871296+0000","last_undegraded":"2024-09-05T11:56:46.871296+0000","last_fullsized":"2024-09-05T11:56:46.871296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574659+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574659+0000","last_peered":"2024-09-05T11:56:19.574659+0000","last_clean":"2024-09-05T11:56:19.574659+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574659+0000","last_undegraded":"2024-09-05T11:56:19.574659+0000","last_fullsized":"2024-09-05T11:56:19.574659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578142+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578142+0000","last_peered":"2024-09-05T11:56:19.578142+0000","last_clean":"2024-09-05T11:56:19.578142+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578142+0000","last_undegraded":"2024-09-05T11:56:19.578142+0000","last_fullsized":"2024-09-05T11:56:19.578142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601515+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601515+0000","last_peered":"2024-09-05T11:56:19.601515+0000","last_clean":"2024-09-05T11:56:19.601515+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601515+0000","last_undegraded":"2024-09-05T11:56:19.601515+0000","last_fullsized":"2024-09-05T11:56:19.601515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575209+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575209+0000","last_peered":"2024-09-05T11:56:19.575209+0000","last_clean":"2024-09-05T11:56:19.575209+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575209+0000","last_undegraded":"2024-09-05T11:56:19.575209+0000","last_fullsized":"2024-09-05T11:56:19.575209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":293,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:54.648792+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:54.648792+0000","last_peered":"2024-09-05T11:56:54.648792+0000","last_clean":"2024-09-05T11:56:54.648792+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:54.648792+0000","last_undegraded":"2024-09-05T11:56:54.648792+0000","last_fullsized":"2024-09-05T11:56:54.648792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":214,"num_read_kb":171,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578591+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578591+0000","last_peered":"2024-09-05T11:56:19.578591+0000","last_clean":"2024-09-05T11:56:19.578591+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578591+0000","last_undegraded":"2024-09-05T11:56:19.578591+0000","last_fullsized":"2024-09-05T11:56:19.578591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579886+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579886+0000","last_peered":"2024-09-05T11:56:19.579886+0000","last_clean":"2024-09-05T11:56:19.579886+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579886+0000","last_undegraded":"2024-09-05T11:56:19.579886+0000","last_fullsized":"2024-09-05T11:56:19.579886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869797+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.869797+0000","last_peered":"2024-09-05T11:56:46.869797+0000","last_clean":"2024-09-05T11:56:46.869797+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.869797+0000","last_undegraded":"2024-09-05T11:56:46.869797+0000","last_fullsized":"2024-09-05T11:56:46.869797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602278+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602278+0000","last_peered":"2024-09-05T11:56:19.602278+0000","last_clean":"2024-09-05T11:56:19.602278+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602278+0000","last_undegraded":"2024-09-05T11:56:19.602278+0000","last_fullsized":"2024-09-05T11:56:19.602278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868664+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868664+0000","last_peered":"2024-09-05T11:56:46.868664+0000","last_clean":"2024-09-05T11:56:46.868664+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868664+0000","last_undegraded":"2024-09-05T11:56:46.868664+0000","last_fullsized":"2024-09-05T11:56:46.868664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580692+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.580692+0000","last_peered":"2024-09-05T11:56:19.580692+0000","last_clean":"2024-09-05T11:56:19.580692+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.580692+0000","last_undegraded":"2024-09-05T11:56:19.580692+0000","last_fullsized":"2024-09-05T11:56:19.580692+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.860067+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.860067+0000","last_peered":"2024-09-05T11:56:46.860067+0000","last_clean":"2024-09-05T11:56:46.860067+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.860067+0000","last_undegraded":"2024-09-05T11:56:46.860067+0000","last_fullsized":"2024-09-05T11:56:46.860067+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871492+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871492+0000","last_peered":"2024-09-05T11:56:46.871492+0000","last_clean":"2024-09-05T11:56:46.871492+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871492+0000","last_undegraded":"2024-09-05T11:56:46.871492+0000","last_fullsized":"2024-09-05T11:56:46.871492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872755+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.872755+0000","last_peered":"2024-09-05T11:56:46.872755+0000","last_clean":"2024-09-05T11:56:46.872755+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.872755+0000","last_undegraded":"2024-09-05T11:56:46.872755+0000","last_fullsized":"2024-09-05T11:56:46.872755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870375+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870375+0000","last_peered":"2024-09-05T11:56:46.870375+0000","last_clean":"2024-09-05T11:56:46.870375+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870375+0000","last_undegraded":"2024-09-05T11:56:46.870375+0000","last_fullsized":"2024-09-05T11:56:46.870375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873576+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873576+0000","last_peered":"2024-09-05T11:56:46.873576+0000","last_clean":"2024-09-05T11:56:46.873576+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873576+0000","last_undegraded":"2024-09-05T11:56:46.873576+0000","last_fullsized":"2024-09-05T11:56:46.873576+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870916+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.870916+0000","last_peered":"2024-09-05T11:56:46.870916+0000","last_clean":"2024-09-05T11:56:46.870916+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.870916+0000","last_undegraded":"2024-09-05T11:56:46.870916+0000","last_fullsized":"2024-09-05T11:56:46.870916+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868242+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868242+0000","last_peered":"2024-09-05T11:56:46.868242+0000","last_clean":"2024-09-05T11:56:46.868242+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868242+0000","last_undegraded":"2024-09-05T11:56:46.868242+0000","last_fullsized":"2024-09-05T11:56:46.868242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":220,"num_read_kb":177,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1744896,"data_stored":1188587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:57:00.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:01.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:00 smithi059 ceph-mon[30025]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-09-05T11:57:01.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:00 smithi059 ceph-mon[30025]: from='client.25112 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:00 smithi059 ceph-mon[30025]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:00 smithi189 ceph-mon[34772]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-09-05T11:57:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:00 smithi189 ceph-mon[34772]: from='client.25112 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:00 smithi189 ceph-mon[34772]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 -- 172.21.15.59:0/594189690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 msgr2=0x7fca30112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 --2- 172.21.15.59:0/594189690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca30112650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009900 tx=0x7fca1c02a490 comp rx=0 tx=0).stop 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 -- 172.21.15.59:0/594189690 shutdown_connections 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 --2- 172.21.15.59:0/594189690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30112b90 0x7fca30114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 --2- 172.21.15.59:0/594189690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca30112650 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 --2- 172.21.15.59:0/594189690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30074590 0x7fca30074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 -- 172.21.15.59:0/594189690 >> 172.21.15.59:0/594189690 conn(0x7fca300fd8c0 msgr2=0x7fca300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 -- 172.21.15.59:0/594189690 shutdown_connections 2024-09-05T11:57:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.158+0000 7fca36e00640 1 -- 172.21.15.59:0/594189690 wait complete. 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 Processor -- start 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 -- start start 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca301a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30112b90 0x7fca3019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca30118560 con 0x7fca30074eb0 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca301183e0 con 0x7fca30074590 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.159+0000 7fca36e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca301186e0 con 0x7fca30112b90 2024-09-05T11:57:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca301a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50874/0 (socket says 172.21.15.59:50874) 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 -- 172.21.15.59:0/672322322 learned_addr learned my addr 172.21.15.59:0/672322322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca34e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30112b90 0x7fca3019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 -- 172.21.15.59:0/672322322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30112b90 msgr2=0x7fca3019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30112b90 0x7fca3019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 -- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 msgr2=0x7fca301a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca301a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 -- 172.21.15.59:0/672322322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3019f040 con 0x7fca30074590 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.160+0000 7fca2fe00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9f0 tx=0x7fca2000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.161+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000cd80 con 0x7fca30074590 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.161+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3019f330 con 0x7fca30074590 2024-09-05T11:57:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.161+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20005500 con 0x7fca30074590 2024-09-05T11:57:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.161+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20010690 con 0x7fca30074590 2024-09-05T11:57:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.161+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301ab620 con 0x7fca30074590 2024-09-05T11:57:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.162+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca30074970 con 0x7fca30074590 2024-09-05T11:57:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.164+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca200046c0 con 0x7fca30074590 2024-09-05T11:57:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.165+0000 7fca2cc00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 0x7fca140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.165+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca200cf890 con 0x7fca30074590 2024-09-05T11:57:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.165+0000 7fca2f400640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 0x7fca140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.166+0000 7fca2f400640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 0x7fca140a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009230 tx=0x7fca1c0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:01.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.169+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca2009c380 con 0x7fca30074590 2024-09-05T11:57:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:00 smithi089 ceph-mon[34499]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-09-05T11:57:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:00 smithi089 ceph-mon[34499]: from='client.25112 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:00 smithi089 ceph-mon[34499]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:01.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.390+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca3019fe40 con 0x7fca30074590 2024-09-05T11:57:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.391+0000 7fca2cc00640 1 -- 172.21.15.59:0/672322322 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fca2009bc80 con 0x7fca30074590 2024-09-05T11:57:01.394 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:01.394 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 msgr2=0x7fca140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 0x7fca140a5040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009230 tx=0x7fca1c0058d0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 msgr2=0x7fca301a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9f0 tx=0x7fca2000edf0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 shutdown_connections 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca140a2b80 0x7fca140a5040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca30112b90 0x7fca3019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca30074eb0 0x7fca301a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 --2- 172.21.15.59:0/672322322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca30074590 0x7fca301a4750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 >> 172.21.15.59:0/672322322 conn(0x7fca300fd8c0 msgr2=0x7fca300ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 shutdown_connections 2024-09-05T11:57:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:01.394+0000 7fca36e00640 1 -- 172.21.15.59:0/672322322 wait complete. 2024-09-05T11:57:02.086 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:01 smithi089 ceph-mon[34499]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:02.086 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:01 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/672322322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:02.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:01 smithi189 ceph-mon[34772]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:02.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:01 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/672322322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:02.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:01 smithi059 ceph-mon[30025]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:02.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:01 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/672322322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:02.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:57:02.511 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:02.956 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:02 smithi059 ceph-mon[30025]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-05T11:57:03.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 -- 172.21.15.59:0/2070168579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84102790 msgr2=0x7f9f841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 --2- 172.21.15.59:0/2070168579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84102790 0x7f9f841067e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009900 tx=0x7f9f7002a410 comp rx=0 tx=0).stop 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 -- 172.21.15.59:0/2070168579 shutdown_connections 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 --2- 172.21.15.59:0/2070168579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 --2- 172.21.15.59:0/2070168579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84102790 0x7f9f841067e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 --2- 172.21.15.59:0/2070168579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84101de0 0x7f9f841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 -- 172.21.15.59:0/2070168579 >> 172.21.15.59:0/2070168579 conn(0x7f9f840fd8c0 msgr2=0x7f9f840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 -- 172.21.15.59:0/2070168579 shutdown_connections 2024-09-05T11:57:03.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.067+0000 7f9f8a000640 1 -- 172.21.15.59:0/2070168579 wait complete. 2024-09-05T11:57:03.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 Processor -- start 2024-09-05T11:57:03.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 -- start start 2024-09-05T11:57:03.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:03.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84102790 0x7f9f8419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84111f70 con 0x7f9f84101de0 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84111df0 con 0x7f9f84106f10 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.068+0000 7f9f8a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841120f0 con 0x7f9f84102790 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59668/0 (socket says 172.21.15.59:59668) 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83400640 1 -- 172.21.15.59:0/1191541160 learned_addr learned my addr 172.21.15.59:0/1191541160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f82a00640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84102790 0x7f9f8419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 -- 172.21.15.59:0/1191541160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84102790 msgr2=0x7f9f8419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84102790 0x7f9f8419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 -- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 msgr2=0x7f9f8419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:03.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 -- 172.21.15.59:0/1191541160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8419f620 con 0x7f9f84106f10 2024-09-05T11:57:03.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83400640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:03.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.069+0000 7f9f83e00640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8419ef40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00da10 tx=0x7f9f7c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.070+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00bd70 con 0x7f9f84106f10 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.070+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f7c003e50 con 0x7f9f84106f10 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.070+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8419f910 con 0x7f9f84106f10 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.070+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0106a0 con 0x7f9f84106f10 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.070+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841a71f0 con 0x7f9f84106f10 2024-09-05T11:57:03.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.071+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f841021c0 con 0x7f9f84106f10 2024-09-05T11:57:03.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.073+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f7c003970 con 0x7f9f84106f10 2024-09-05T11:57:03.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.074+0000 7f9f7be00640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 0x7f9f680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:03.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.074+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9f7c0cf7a0 con 0x7f9f84106f10 2024-09-05T11:57:03.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.074+0000 7f9f83400640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 0x7f9f680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:03.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.075+0000 7f9f83400640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 0x7f9f680a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f740104e0 tx=0x7f9f7400b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:03.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.079+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f7c09c290 con 0x7f9f84106f10 2024-09-05T11:57:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:02 smithi189 ceph-mon[34772]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-05T11:57:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:02 smithi089 ceph-mon[34499]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-05T11:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.262+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f841a7450 con 0x7f9f84106f10 2024-09-05T11:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.265+0000 7f9f7be00640 1 -- 172.21.15.59:0/1191541160 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7f9f7c09bb90 con 0x7f9f84106f10 2024-09-05T11:57:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:03.267 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 msgr2=0x7f9f680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 0x7f9f680a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f740104e0 tx=0x7f9f7400b410 comp rx=0 tx=0).stop 2024-09-05T11:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 msgr2=0x7f9f8419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8419ef40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00da10 tx=0x7f9f7c00ded0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 shutdown_connections 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9f680a2c00 0x7f9f680a50c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84106f10 0x7f9f8419ef40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f84102790 0x7f9f8419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 --2- 172.21.15.59:0/1191541160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f84101de0 0x7f9f8419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.267+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 >> 172.21.15.59:0/1191541160 conn(0x7f9f840fd8c0 msgr2=0x7f9f840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.268+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 shutdown_connections 2024-09-05T11:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:03.268+0000 7f9f8a000640 1 -- 172.21.15.59:0/1191541160 wait complete. 2024-09-05T11:57:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:03 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1191541160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:04.029 DEBUG:tasks.fwd_scrub:Yielding 2024-09-05T11:57:04.029 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-05T11:57:04.044 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T11:57:04.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:57:04.045 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-05T11:57:04.045 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:57:04.045 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:57:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:03 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1191541160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:04.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:03 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1191541160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:04.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:04 smithi059 ceph-mon[30025]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:04 smithi189 ceph-mon[34772]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:05.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:04 smithi089 ceph-mon[34499]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 -- 172.21.15.59:0/3143348477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 msgr2=0x7efc94104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 --2- 172.21.15.59:0/3143348477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc94104210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efc84009900 tx=0x7efc8402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 -- 172.21.15.59:0/3143348477 shutdown_connections 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 --2- 172.21.15.59:0/3143348477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc94108f60 0x7efc94110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 --2- 172.21.15.59:0/3143348477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc941047e0 0x7efc94108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 --2- 172.21.15.59:0/3143348477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc94104210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 -- 172.21.15.59:0/3143348477 >> 172.21.15.59:0/3143348477 conn(0x7efc940ff910 msgr2=0x7efc94101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 -- 172.21.15.59:0/3143348477 shutdown_connections 2024-09-05T11:57:05.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.188+0000 7efc99c00640 1 -- 172.21.15.59:0/3143348477 wait complete. 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.189+0000 7efc99c00640 1 Processor -- start 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.189+0000 7efc99c00640 1 -- start start 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc941047e0 0x7efc9419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 0x7efc9419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc94113fc0 con 0x7efc94103e30 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc94113e40 con 0x7efc94108f60 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc94114140 con 0x7efc941047e0 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59686/0 (socket says 172.21.15.59:59686) 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc93e00640 1 -- 172.21.15.59:0/1481783939 learned_addr learned my addr 172.21.15.59:0/1481783939 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc93400640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc941047e0 0x7efc9419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.190+0000 7efc99200640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 0x7efc9419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 -- 172.21.15.59:0/1481783939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc941047e0 msgr2=0x7efc9419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc941047e0 0x7efc9419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 -- 172.21.15.59:0/1481783939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 msgr2=0x7efc9419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 0x7efc9419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 -- 172.21.15.59:0/1481783939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9419f430 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc99200640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 0x7efc9419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc93e00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc84006010 tx=0x7efc84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8403f030 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9419f6c0 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc84004440 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84035070 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.191+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc941a7000 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.193+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94105310 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.194+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc84042030 con 0x7efc94103e30 2024-09-05T11:57:05.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.196+0000 7efc90c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 0x7efc780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.196+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efc840ef450 con 0x7efc94103e30 2024-09-05T11:57:05.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.196+0000 7efc93400640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 0x7efc780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.197+0000 7efc93400640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 0x7efc780a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc940ffd30 tx=0x7efc80005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.200+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc840bb8f0 con 0x7efc94103e30 2024-09-05T11:57:05.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.383+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc9410c1b0 con 0x7efc780a2b80 2024-09-05T11:57:05.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.393+0000 7efc90c00640 1 -- 172.21.15.59:0/1481783939 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340405 (secure 0 0 0) 0x7efc8001b3f0 con 0x7efc780a2b80 2024-09-05T11:57:05.395 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:05.401 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:57:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.397+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 msgr2=0x7efc780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.397+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 0x7efc780a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc940ffd30 tx=0x7efc80005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.397+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 msgr2=0x7efc9419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.397+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc84006010 tx=0x7efc84004040 comp rx=0 tx=0).stop 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 shutdown_connections 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efc780a2b80 0x7efc780a5040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc94108f60 0x7efc9419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc941047e0 0x7efc9419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 --2- 172.21.15.59:0/1481783939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc94103e30 0x7efc9419a480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 >> 172.21.15.59:0/1481783939 conn(0x7efc940ff910 msgr2=0x7efc94100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 shutdown_connections 2024-09-05T11:57:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.398+0000 7efc99c00640 1 -- 172.21.15.59:0/1481783939 wait complete. 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.514+0000 7f00aee00640 1 -- 172.21.15.59:0/4123190405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 msgr2=0x7f00a81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.514+0000 7f00aee00640 1 --2- 172.21.15.59:0/4123190405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a81089f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0094009900 tx=0x7f009402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 -- 172.21.15.59:0/4123190405 shutdown_connections 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 --2- 172.21.15.59:0/4123190405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8109120 0x7f00a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 --2- 172.21.15.59:0/4123190405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a81089f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 --2- 172.21.15.59:0/4123190405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8103ff0 0x7f00a81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 -- 172.21.15.59:0/4123190405 >> 172.21.15.59:0/4123190405 conn(0x7f00a80ffa90 msgr2=0x7f00a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 -- 172.21.15.59:0/4123190405 shutdown_connections 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.515+0000 7f00aee00640 1 -- 172.21.15.59:0/4123190405 wait complete. 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 Processor -- start 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 -- start start 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 0x7f00a819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8109120 0x7f00a819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8114170 con 0x7f00a81049a0 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8113ff0 con 0x7f00a8103ff0 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00aee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81142f0 con 0x7f00a8109120 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.516+0000 7f00a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 0x7f00a819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59708/0 (socket says 172.21.15.59:59708) 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7400640 1 -- 172.21.15.59:0/803654261 learned_addr learned my addr 172.21.15.59:0/803654261 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00ace00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8109120 0x7f00a819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 -- 172.21.15.59:0/803654261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8109120 msgr2=0x7f00a819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8109120 0x7f00a819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 -- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 msgr2=0x7f00a819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 -- 172.21.15.59:0/803654261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a819f600 con 0x7f00a8103ff0 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a7e00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 0x7f00a819a650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f009800d6d0 tx=0x7f009800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.517+0000 7f00a4c00640 1 -- 172.21.15.59:0/803654261 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098017070 con 0x7f00a8103ff0 2024-09-05T11:57:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.518+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a819f8f0 con 0x7f00a8103ff0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.518+0000 7f00a4c00640 1 -- 172.21.15.59:0/803654261 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0098009820 con 0x7f00a8103ff0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.518+0000 7f00a4c00640 1 -- 172.21.15.59:0/803654261 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098004cd0 con 0x7f00a8103ff0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.518+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81a71d0 con 0x7f00a8103ff0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 Processor -- start 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 -- start start 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00a4c00640 1 -- 172.21.15.59:0/803654261 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00980099d0 con 0x7f00a8103ff0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920dee9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 0x55920def4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 0x55920def7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55920de50290 con 0x55920deea3e0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55920ddf7b40 con 0x55920def4ca0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00b18c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55920de4c250 con 0x55920dee9ac0 2024-09-05T11:57:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.521+0000 7f00a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 0x55920def4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 0x55920def7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 0x55920def4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59716/0 (socket says 172.21.15.59:59716) 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7e00640 1 -- 172.21.15.59:0/4011724720 learned_addr learned my addr 172.21.15.59:0/4011724720 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00ace00640 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920dee9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 -- 172.21.15.59:0/4011724720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 msgr2=0x55920dee9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920dee9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 -- 172.21.15.59:0/4011724720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 msgr2=0x55920def4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 0x55920def4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 -- 172.21.15.59:0/4011724720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55920dbc16a0 con 0x55920def4ca0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.522+0000 7f00a7400640 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 0x55920def7090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f00940040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454bdfd5db7a41e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f009d600640 1 -- 172.21.15.59:0/4011724720 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094040070 con 0x55920def4ca0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f009d600640 1 -- 172.21.15.59:0/4011724720 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094005500 con 0x55920def4ca0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f009d600640 1 -- 172.21.15.59:0/4011724720 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094035040 con 0x55920def4ca0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00a4c00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 0x7f008c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00a4c00640 1 -- 172.21.15.59:0/803654261 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f00980cef50 con 0x7f00a8103ff0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00ace00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 0x7f008c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00b18c6740 1 -- 172.21.15.59:0/4011724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 msgr2=0x55920def7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00b18c6740 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 0x55920def7090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f00940040c0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.523+0000 7f00b18c6740 1 -- 172.21.15.59:0/4011724720 shutdown_connections 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920def4ca0 0x55920def7090 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f00940040c0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920deea3e0 0x55920def4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 --2- 172.21.15.59:0/4011724720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920dee9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 -- 172.21.15.59:0/4011724720 >> 172.21.15.59:0/4011724720 conn(0x55920dee6050 msgr2=0x55920dee6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 -- 172.21.15.59:0/4011724720 shutdown_connections 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 -- 172.21.15.59:0/4011724720 wait complete. 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00ace00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 0x7f008c0a50c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005fd0 tx=0x7f00a0009600 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 -- start start 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.524+0000 7f00b18c6740 1 Processor -- start 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920deea3e0 0x55920def1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920df00d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55920def78b0 con 0x55920def24f0 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55920de0afa0 con 0x55920dee9ac0 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55920def7a30 con 0x55920deea3e0 2024-09-05T11:57:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00b18c6740 10 client.0 _ll_register_callbacks cb 0x55920db5eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50940/0 (socket says 172.21.15.59:50940) 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00a7e00640 1 -- 172.21.15.59:0/2843573739 learned_addr learned my addr 172.21.15.59:0/2843573739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00ace00640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920df00d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.525+0000 7f00a7400640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920deea3e0 0x55920def1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 -- 172.21.15.59:0/2843573739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920deea3e0 msgr2=0x55920def1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920deea3e0 0x55920def1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 -- 172.21.15.59:0/2843573739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 msgr2=0x55920df00d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920df00d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 -- 172.21.15.59:0/2843573739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55920def4ad0 con 0x55920dee9ac0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00a7e00640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0098009610 tx=0x7f0098005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adeb47f62bd86eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f0092000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098020050 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0094005500 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0098097a60 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.526+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f009809af30 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 10 client.25154 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55920df03da0 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094005500 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f00980d2020 con 0x55920dee9ac0 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 10 client.25154 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 10 client.25154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 20 client.25154 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df051b0 0x55920df075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 4 client.25154 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:05.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.527+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55920de5d800 con 0x55920df051b0 2024-09-05T11:57:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.528+0000 7f00a7400640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df051b0 0x55920df075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.528+0000 7f00a7400640 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df051b0 0x55920df075a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1e5f9949b3f2ea server_cookie=f7ca4386a1096a55 in_seq=0 out_seq=0 2024-09-05T11:57:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.528+0000 7f0092000640 10 client.25154 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:05.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f0092000640 1 -- 172.21.15.59:0/2843573739 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55920de5d800 con 0x55920df051b0 2024-09-05T11:57:05.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f0092000640 10 client.25154 handle_command_reply: tid=0 2024-09-05T11:57:05.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 client.25154 shutdown 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 msgr2=0x55920def1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0098009610 tx=0x7f0098005d30 comp rx=0 tx=0).stop 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 shutdown_connections 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df051b0 0x55920df075a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.529+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920df00d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920deea3e0 0x55920def1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 1 --2- 172.21.15.59:0/2843573739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920dee9ac0 0x55920def1a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 >> 172.21.15.59:0/2843573739 conn(0x55920dee6050 msgr2=0x55920deebdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f0092000640 0 client.25154 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 shutdown_connections 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 1 -- 172.21.15.59:0/2843573739 wait complete. 2024-09-05T11:57:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.530+0000 7f00b18c6740 20 client.25154 trim_cache size 0 max 16384 2024-09-05T11:57:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 Processor -- start 2024-09-05T11:57:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 -- start start 2024-09-05T11:57:05.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920def1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920deece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81af0d0 con 0x55920def24f0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81b1950 con 0x55920deea3e0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00b18c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81af740 con 0x55920dee9ac0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920deece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50956/0 (socket says 172.21.15.59:50956) 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00a7400640 1 -- 172.21.15.59:0/3101789097 learned_addr learned my addr 172.21.15.59:0/3101789097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.550+0000 7f00a7e00640 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920def1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 -- 172.21.15.59:0/3101789097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 msgr2=0x55920def1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920def1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 -- 172.21.15.59:0/3101789097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 msgr2=0x55920deece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920deece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 -- 172.21.15.59:0/3101789097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55920deeffc0 con 0x55920deea3e0 2024-09-05T11:57:05.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f00a7400640 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0094031550 tx=0x7f0094005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888d234731dbdf22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f009cc00640 1 -- 172.21.15.59:0/3101789097 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094054070 con 0x55920deea3e0 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f009cc00640 1 -- 172.21.15.59:0/3101789097 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094004d80 con 0x55920deea3e0 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.551+0000 7f009cc00640 1 -- 172.21.15.59:0/3101789097 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094040070 con 0x55920deea3e0 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 -- 172.21.15.59:0/3101789097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 msgr2=0x55920df00b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0094031550 tx=0x7f0094005c00 comp rx=0 tx=0).stop 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 -- 172.21.15.59:0/3101789097 shutdown_connections 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920deece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920df00b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 --2- 172.21.15.59:0/3101789097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920def1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 -- 172.21.15.59:0/3101789097 >> 172.21.15.59:0/3101789097 conn(0x55920dee6050 msgr2=0x55920deec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 -- 172.21.15.59:0/3101789097 shutdown_connections 2024-09-05T11:57:05.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.552+0000 7f00b18c6740 1 -- 172.21.15.59:0/3101789097 wait complete. 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 -- start start 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 Processor -- start 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920defccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920def9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81afe10 con 0x55920def24f0 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81b1280 con 0x55920deea3e0 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81b04e0 con 0x55920dee9ac0 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.553+0000 7f00b18c6740 10 client.0 _ll_register_callbacks cb 0x7f00980057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50964/0 (socket says 172.21.15.59:50964) 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 -- 172.21.15.59:0/1616862354 learned_addr learned my addr 172.21.15.59:0/1616862354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920def9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7e00640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920defccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 -- 172.21.15.59:0/1616862354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 msgr2=0x55920defccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920defccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 -- 172.21.15.59:0/1616862354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 msgr2=0x55920def9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920def9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 -- 172.21.15.59:0/1616862354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55920defd8b0 con 0x55920deea3e0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f00a7400640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f00940092a0 tx=0x7f0094005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f632ec953b7f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.554+0000 7f0092000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009405d060 con 0x55920deea3e0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f00940046c0 con 0x55920deea3e0 2024-09-05T11:57:05.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0094004d80 con 0x55920deea3e0 2024-09-05T11:57:05.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0094064050 con 0x55920deea3e0 2024-09-05T11:57:05.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f00b18c6740 10 client.25160 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:05.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55920df70450 con 0x55920deea3e0 2024-09-05T11:57:05.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094004d80 con 0x55920deea3e0 2024-09-05T11:57:05.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.555+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0094031750 con 0x55920deea3e0 2024-09-05T11:57:05.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 10 client.25160 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:05.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 10 client.25160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:05.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 20 client.25160 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:05.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df706d0 0x55920df72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 4 client.25160 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55920de57d10 con 0x55920df706d0 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00a7e00640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df706d0 0x55920df72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f00a7e00640 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df706d0 0x55920df72ac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3d3da57643562f server_cookie=eee7ef623e8e1f85 in_seq=0 out_seq=0 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.556+0000 7f0092000640 10 client.25160 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.557+0000 7f0092000640 1 -- 172.21.15.59:0/1616862354 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55920de57d10 con 0x55920df706d0 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.557+0000 7f0092000640 10 client.25160 handle_command_reply: tid=0 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.557+0000 7f00b18c6740 1 client.25160 shutdown 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 msgr2=0x55920defd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f00940092a0 tx=0x7f0094005b90 comp rx=0 tx=0).stop 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 shutdown_connections 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55920df706d0 0x55920df72ac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55920def24f0 0x55920def9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55920deea3e0 0x55920defd1f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 --2- 172.21.15.59:0/1616862354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55920dee9ac0 0x55920defccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f0092000640 0 client.25160 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 >> 172.21.15.59:0/1616862354 conn(0x55920dee6050 msgr2=0x55920deecf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 shutdown_connections 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 1 -- 172.21.15.59:0/1616862354 wait complete. 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.558+0000 7f00b18c6740 20 client.25160 trim_cache size 0 max 16384 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:57:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "22c2d830-c704-4889-a14e-f86c35288f9a", 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 msgr2=0x7f008c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 0x7f008c0a50c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005fd0 tx=0x7f00a0009600 comp rx=0 tx=0).stop 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 msgr2=0x7f00a819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 0x7f00a819a650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f009800d6d0 tx=0x7f009800db90 comp rx=0 tx=0).stop 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 shutdown_connections 2024-09-05T11:57:05.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f008c0a2c00 0x7f008c0a50c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00a8109120 0x7f00a819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a81049a0 0x7f00a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 --2- 172.21.15.59:0/803654261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00a8103ff0 0x7f00a819a650 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 >> 172.21.15.59:0/803654261 conn(0x7f00a80ffa90 msgr2=0x7f00a8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 shutdown_connections 2024-09-05T11:57:05.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:05.559+0000 7f00aee00640 1 -- 172.21.15.59:0/803654261 wait complete. 2024-09-05T11:57:06.480 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-05T11:57:04.890676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":357,"num_read_kb":297,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376232,"kb_used_data":14504,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565592,"statfs":{"total":1151940427776,"available":1151555166208,"internally_reserved":0,"allocated":14852096,"data_stored":8358314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003454"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578235+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:19.578235+0000","last_peered":"2024-09-05T11:56:19.578235+0000","last_clean":"2024-09-05T11:56:19.578235+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:19.578235+0000","last_undegraded":"2024-09-05T11:56:19.578235+0000","last_fullsized":"2024-09-05T11:56:19.578235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600094+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600094+0000","last_peered":"2024-09-05T11:56:19.600094+0000","last_clean":"2024-09-05T11:56:19.600094+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600094+0000","last_undegraded":"2024-09-05T11:56:19.600094+0000","last_fullsized":"2024-09-05T11:56:19.600094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577064+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577064+0000","last_peered":"2024-09-05T11:56:19.577064+0000","last_clean":"2024-09-05T11:56:19.577064+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577064+0000","last_undegraded":"2024-09-05T11:56:19.577064+0000","last_fullsized":"2024-09-05T11:56:19.577064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581425+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581425+0000","last_peered":"2024-09-05T11:56:19.581425+0000","last_clean":"2024-09-05T11:56:19.581425+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581425+0000","last_undegraded":"2024-09-05T11:56:19.581425+0000","last_fullsized":"2024-09-05T11:56:19.581425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874976+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874976+0000","last_peered":"2024-09-05T11:56:46.874976+0000","last_clean":"2024-09-05T11:56:46.874976+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874976+0000","last_undegraded":"2024-09-05T11:56:46.874976+0000","last_fullsized":"2024-09-05T11:56:46.874976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581278+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581278+0000","last_peered":"2024-09-05T11:56:19.581278+0000","last_clean":"2024-09-05T11:56:19.581278+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581278+0000","last_undegraded":"2024-09-05T11:56:19.581278+0000","last_fullsized":"2024-09-05T11:56:19.581278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872911+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.872911+0000","last_peered":"2024-09-05T11:56:46.872911+0000","last_clean":"2024-09-05T11:56:46.872911+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.872911+0000","last_undegraded":"2024-09-05T11:56:46.872911+0000","last_fullsized":"2024-09-05T11:56:46.872911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574712+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574712+0000","last_peered":"2024-09-05T11:56:19.574712+0000","last_clean":"2024-09-05T11:56:19.574712+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574712+0000","last_undegraded":"2024-09-05T11:56:19.574712+0000","last_fullsized":"2024-09-05T11:56:19.574712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578218+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578218+0000","last_peered":"2024-09-05T11:56:19.578218+0000","last_clean":"2024-09-05T11:56:19.578218+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578218+0000","last_undegraded":"2024-09-05T11:56:19.578218+0000","last_fullsized":"2024-09-05T11:56:19.578218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601591+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601591+0000","last_peered":"2024-09-05T11:56:19.601591+0000","last_clean":"2024-09-05T11:56:19.601591+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601591+0000","last_undegraded":"2024-09-05T11:56:19.601591+0000","last_fullsized":"2024-09-05T11:56:19.601591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575209+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575209+0000","last_peered":"2024-09-05T11:56:19.575209+0000","last_clean":"2024-09-05T11:56:19.575209+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575209+0000","last_undegraded":"2024-09-05T11:56:19.575209+0000","last_fullsized":"2024-09-05T11:56:19.575209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":304,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:59.637290+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:56:59.637290+0000","last_peered":"2024-09-05T11:56:59.637290+0000","last_clean":"2024-09-05T11:56:59.637290+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:56:59.637290+0000","last_undegraded":"2024-09-05T11:56:59.637290+0000","last_fullsized":"2024-09-05T11:56:59.637290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":225,"num_read_kb":182,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578665+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578665+0000","last_peered":"2024-09-05T11:56:19.578665+0000","last_clean":"2024-09-05T11:56:19.578665+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578665+0000","last_undegraded":"2024-09-05T11:56:19.578665+0000","last_fullsized":"2024-09-05T11:56:19.578665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579981+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579981+0000","last_peered":"2024-09-05T11:56:19.579981+0000","last_clean":"2024-09-05T11:56:19.579981+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579981+0000","last_undegraded":"2024-09-05T11:56:19.579981+0000","last_fullsized":"2024-09-05T11:56:19.579981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870033+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.870033+0000","last_peered":"2024-09-05T11:56:46.870033+0000","last_clean":"2024-09-05T11:56:46.870033+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.870033+0000","last_undegraded":"2024-09-05T11:56:46.870033+0000","last_fullsized":"2024-09-05T11:56:46.870033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602775+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602775+0000","last_peered":"2024-09-05T11:56:19.602775+0000","last_clean":"2024-09-05T11:56:19.602775+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602775+0000","last_undegraded":"2024-09-05T11:56:19.602775+0000","last_fullsized":"2024-09-05T11:56:19.602775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868826+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868826+0000","last_peered":"2024-09-05T11:56:46.868826+0000","last_clean":"2024-09-05T11:56:46.868826+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868826+0000","last_undegraded":"2024-09-05T11:56:46.868826+0000","last_fullsized":"2024-09-05T11:56:46.868826+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581507+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.581507+0000","last_peered":"2024-09-05T11:56:19.581507+0000","last_clean":"2024-09-05T11:56:19.581507+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.581507+0000","last_undegraded":"2024-09-05T11:56:19.581507+0000","last_fullsized":"2024-09-05T11:56:19.581507+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.861127+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.861127+0000","last_peered":"2024-09-05T11:56:46.861127+0000","last_clean":"2024-09-05T11:56:46.861127+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.861127+0000","last_undegraded":"2024-09-05T11:56:46.861127+0000","last_fullsized":"2024-09-05T11:56:46.861127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871715+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871715+0000","last_peered":"2024-09-05T11:56:46.871715+0000","last_clean":"2024-09-05T11:56:46.871715+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871715+0000","last_undegraded":"2024-09-05T11:56:46.871715+0000","last_fullsized":"2024-09-05T11:56:46.871715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874263+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.874263+0000","last_peered":"2024-09-05T11:56:46.874263+0000","last_clean":"2024-09-05T11:56:46.874263+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.874263+0000","last_undegraded":"2024-09-05T11:56:46.874263+0000","last_fullsized":"2024-09-05T11:56:46.874263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870515+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870515+0000","last_peered":"2024-09-05T11:56:46.870515+0000","last_clean":"2024-09-05T11:56:46.870515+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870515+0000","last_undegraded":"2024-09-05T11:56:46.870515+0000","last_fullsized":"2024-09-05T11:56:46.870515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873576+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873576+0000","last_peered":"2024-09-05T11:56:46.873576+0000","last_clean":"2024-09-05T11:56:46.873576+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873576+0000","last_undegraded":"2024-09-05T11:56:46.873576+0000","last_fullsized":"2024-09-05T11:56:46.873576+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870916+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.870916+0000","last_peered":"2024-09-05T11:56:46.870916+0000","last_clean":"2024-09-05T11:56:46.870916+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.870916+0000","last_undegraded":"2024-09-05T11:56:46.870916+0000","last_fullsized":"2024-09-05T11:56:46.870916+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868370+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868370+0000","last_peered":"2024-09-05T11:56:46.868370+0000","last_clean":"2024-09-05T11:56:46.868370+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868370+0000","last_undegraded":"2024-09-05T11:56:46.868370+0000","last_fullsized":"2024-09-05T11:56:46.868370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":231,"num_read_kb":188,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37056,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708096,"statfs":{"total":95995035648,"available":95957090304,"internally_reserved":0,"allocated":1753088,"data_stored":1193960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135933,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:57:06.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:57:06.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:57:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:57:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:06 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:57:07.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:06 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:06 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:07.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 -- 172.21.15.59:0/1366246018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d141049c0 msgr2=0x7f3d14108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:07.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 --2- 172.21.15.59:0/1366246018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d141049c0 0x7f3d14108a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009860 tx=0x7f3d0002a340 comp rx=0 tx=0).stop 2024-09-05T11:57:07.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 -- 172.21.15.59:0/1366246018 shutdown_connections 2024-09-05T11:57:07.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 --2- 172.21.15.59:0/1366246018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14109140 0x7f3d14110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 --2- 172.21.15.59:0/1366246018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d141049c0 0x7f3d14108a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 --2- 172.21.15.59:0/1366246018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d14104010 0x7f3d141043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 -- 172.21.15.59:0/1366246018 >> 172.21.15.59:0/1366246018 conn(0x7f3d140ffa50 msgr2=0x7f3d14101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:07.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 -- 172.21.15.59:0/1366246018 shutdown_connections 2024-09-05T11:57:07.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.542+0000 7f3d19000640 1 -- 172.21.15.59:0/1366246018 wait complete. 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.543+0000 7f3d19000640 1 Processor -- start 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.543+0000 7f3d19000640 1 -- start start 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d141049c0 0x7f3d1419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d14109140 0x7f3d1419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14114140 con 0x7f3d14104010 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14113fc0 con 0x7f3d141049c0 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d19000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141142c0 con 0x7f3d14109140 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:07.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d11e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59746/0 (socket says 172.21.15.59:59746) 2024-09-05T11:57:07.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d11e00640 1 -- 172.21.15.59:0/4022163529 learned_addr learned my addr 172.21.15.59:0/4022163529 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:07.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d11400640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d141049c0 0x7f3d1419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:07.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.544+0000 7f3d12800640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d14109140 0x7f3d1419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 -- 172.21.15.59:0/4022163529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d14109140 msgr2=0x7f3d1419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d14109140 0x7f3d1419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 -- 172.21.15.59:0/4022163529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d141049c0 msgr2=0x7f3d1419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d141049c0 0x7f3d1419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 -- 172.21.15.59:0/4022163529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1419f660 con 0x7f3d14104010 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d11e00640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9f0 tx=0x7f3d0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:07.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400cd80 con 0x7f3d14104010 2024-09-05T11:57:07.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.545+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04010910 con 0x7f3d14104010 2024-09-05T11:57:07.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.546+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1419f950 con 0x7f3d14104010 2024-09-05T11:57:07.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.546+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04018990 con 0x7f3d14104010 2024-09-05T11:57:07.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.546+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d141a7230 con 0x7f3d14104010 2024-09-05T11:57:07.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.547+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d141054f0 con 0x7f3d14104010 2024-09-05T11:57:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.551+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d04010ac0 con 0x7f3d14104010 2024-09-05T11:57:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.553+0000 7f3d0aa00640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 0x7f3cf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.554+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3d040cfc00 con 0x7f3d14104010 2024-09-05T11:57:07.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.554+0000 7f3d11400640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 0x7f3cf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:07.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.554+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d040fe230 con 0x7f3d14104010 2024-09-05T11:57:07.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.555+0000 7f3d11400640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 0x7f3cf80a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d0002a930 tx=0x7f3d000056d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:07.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.743+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d14062630 con 0x7f3cf80a2b80 2024-09-05T11:57:07.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.752+0000 7f3d0aa00640 1 -- 172.21.15.59:0/4022163529 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340405 (secure 0 0 0) 0x7f3d14062630 con 0x7f3cf80a2b80 2024-09-05T11:57:07.759 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 msgr2=0x7f3cf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 0x7f3cf80a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d0002a930 tx=0x7f3d000056d0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 msgr2=0x7f3d1419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400e9f0 tx=0x7f3d0400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 shutdown_connections 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf80a2b80 0x7f3cf80a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d14109140 0x7f3d1419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d141049c0 0x7f3d1419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 --2- 172.21.15.59:0/4022163529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d14104010 0x7f3d1419a6b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 >> 172.21.15.59:0/4022163529 conn(0x7f3d140ffa50 msgr2=0x7f3d14100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 shutdown_connections 2024-09-05T11:57:07.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:07.757+0000 7f3d19000640 1 -- 172.21.15.59:0/4022163529 wait complete. 2024-09-05T11:57:08.627 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-05T11:57:06.891203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":367,"num_read_kb":302,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376232,"kb_used_data":14504,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565592,"statfs":{"total":1151940427776,"available":1151555166208,"internally_reserved":0,"allocated":14852096,"data_stored":8358314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5684,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":21,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003402"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578235+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:19.578235+0000","last_peered":"2024-09-05T11:56:19.578235+0000","last_clean":"2024-09-05T11:56:19.578235+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:19.578235+0000","last_undegraded":"2024-09-05T11:56:19.578235+0000","last_fullsized":"2024-09-05T11:56:19.578235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600152+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600152+0000","last_peered":"2024-09-05T11:56:19.600152+0000","last_clean":"2024-09-05T11:56:19.600152+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600152+0000","last_undegraded":"2024-09-05T11:56:19.600152+0000","last_fullsized":"2024-09-05T11:56:19.600152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577131+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577131+0000","last_peered":"2024-09-05T11:56:19.577131+0000","last_clean":"2024-09-05T11:56:19.577131+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577131+0000","last_undegraded":"2024-09-05T11:56:19.577131+0000","last_fullsized":"2024-09-05T11:56:19.577131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581425+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581425+0000","last_peered":"2024-09-05T11:56:19.581425+0000","last_clean":"2024-09-05T11:56:19.581425+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581425+0000","last_undegraded":"2024-09-05T11:56:19.581425+0000","last_fullsized":"2024-09-05T11:56:19.581425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874976+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874976+0000","last_peered":"2024-09-05T11:56:46.874976+0000","last_clean":"2024-09-05T11:56:46.874976+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874976+0000","last_undegraded":"2024-09-05T11:56:46.874976+0000","last_fullsized":"2024-09-05T11:56:46.874976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581278+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581278+0000","last_peered":"2024-09-05T11:56:19.581278+0000","last_clean":"2024-09-05T11:56:19.581278+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581278+0000","last_undegraded":"2024-09-05T11:56:19.581278+0000","last_fullsized":"2024-09-05T11:56:19.581278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872911+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.872911+0000","last_peered":"2024-09-05T11:56:46.872911+0000","last_clean":"2024-09-05T11:56:46.872911+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.872911+0000","last_undegraded":"2024-09-05T11:56:46.872911+0000","last_fullsized":"2024-09-05T11:56:46.872911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574712+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574712+0000","last_peered":"2024-09-05T11:56:19.574712+0000","last_clean":"2024-09-05T11:56:19.574712+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574712+0000","last_undegraded":"2024-09-05T11:56:19.574712+0000","last_fullsized":"2024-09-05T11:56:19.574712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578218+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578218+0000","last_peered":"2024-09-05T11:56:19.578218+0000","last_clean":"2024-09-05T11:56:19.578218+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578218+0000","last_undegraded":"2024-09-05T11:56:19.578218+0000","last_fullsized":"2024-09-05T11:56:19.578218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601591+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601591+0000","last_peered":"2024-09-05T11:56:19.601591+0000","last_clean":"2024-09-05T11:56:19.601591+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601591+0000","last_undegraded":"2024-09-05T11:56:19.601591+0000","last_fullsized":"2024-09-05T11:56:19.601591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575257+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575257+0000","last_peered":"2024-09-05T11:56:19.575257+0000","last_clean":"2024-09-05T11:56:19.575257+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575257+0000","last_undegraded":"2024-09-05T11:56:19.575257+0000","last_fullsized":"2024-09-05T11:56:19.575257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":314,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:57:04.645035+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:57:04.645035+0000","last_peered":"2024-09-05T11:57:04.645035+0000","last_clean":"2024-09-05T11:57:04.645035+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:57:04.645035+0000","last_undegraded":"2024-09-05T11:57:04.645035+0000","last_fullsized":"2024-09-05T11:57:04.645035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":235,"num_read_kb":187,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578665+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578665+0000","last_peered":"2024-09-05T11:56:19.578665+0000","last_clean":"2024-09-05T11:56:19.578665+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578665+0000","last_undegraded":"2024-09-05T11:56:19.578665+0000","last_fullsized":"2024-09-05T11:56:19.578665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579981+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579981+0000","last_peered":"2024-09-05T11:56:19.579981+0000","last_clean":"2024-09-05T11:56:19.579981+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579981+0000","last_undegraded":"2024-09-05T11:56:19.579981+0000","last_fullsized":"2024-09-05T11:56:19.579981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870033+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.870033+0000","last_peered":"2024-09-05T11:56:46.870033+0000","last_clean":"2024-09-05T11:56:46.870033+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.870033+0000","last_undegraded":"2024-09-05T11:56:46.870033+0000","last_fullsized":"2024-09-05T11:56:46.870033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602775+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602775+0000","last_peered":"2024-09-05T11:56:19.602775+0000","last_clean":"2024-09-05T11:56:19.602775+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602775+0000","last_undegraded":"2024-09-05T11:56:19.602775+0000","last_fullsized":"2024-09-05T11:56:19.602775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868826+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868826+0000","last_peered":"2024-09-05T11:56:46.868826+0000","last_clean":"2024-09-05T11:56:46.868826+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868826+0000","last_undegraded":"2024-09-05T11:56:46.868826+0000","last_fullsized":"2024-09-05T11:56:46.868826+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581507+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.581507+0000","last_peered":"2024-09-05T11:56:19.581507+0000","last_clean":"2024-09-05T11:56:19.581507+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.581507+0000","last_undegraded":"2024-09-05T11:56:19.581507+0000","last_fullsized":"2024-09-05T11:56:19.581507+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.861127+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.861127+0000","last_peered":"2024-09-05T11:56:46.861127+0000","last_clean":"2024-09-05T11:56:46.861127+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.861127+0000","last_undegraded":"2024-09-05T11:56:46.861127+0000","last_fullsized":"2024-09-05T11:56:46.861127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871715+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871715+0000","last_peered":"2024-09-05T11:56:46.871715+0000","last_clean":"2024-09-05T11:56:46.871715+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871715+0000","last_undegraded":"2024-09-05T11:56:46.871715+0000","last_fullsized":"2024-09-05T11:56:46.871715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874263+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.874263+0000","last_peered":"2024-09-05T11:56:46.874263+0000","last_clean":"2024-09-05T11:56:46.874263+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.874263+0000","last_undegraded":"2024-09-05T11:56:46.874263+0000","last_fullsized":"2024-09-05T11:56:46.874263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870515+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870515+0000","last_peered":"2024-09-05T11:56:46.870515+0000","last_clean":"2024-09-05T11:56:46.870515+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870515+0000","last_undegraded":"2024-09-05T11:56:46.870515+0000","last_fullsized":"2024-09-05T11:56:46.870515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873689+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873689+0000","last_peered":"2024-09-05T11:56:46.873689+0000","last_clean":"2024-09-05T11:56:46.873689+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873689+0000","last_undegraded":"2024-09-05T11:56:46.873689+0000","last_fullsized":"2024-09-05T11:56:46.873689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875619+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.875619+0000","last_peered":"2024-09-05T11:56:46.875619+0000","last_clean":"2024-09-05T11:56:46.875619+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.875619+0000","last_undegraded":"2024-09-05T11:56:46.875619+0000","last_fullsized":"2024-09-05T11:56:46.875619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868370+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868370+0000","last_peered":"2024-09-05T11:56:46.868370+0000","last_clean":"2024-09-05T11:56:46.868370+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868370+0000","last_undegraded":"2024-09-05T11:56:46.868370+0000","last_fullsized":"2024-09-05T11:56:46.868370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":241,"num_read_kb":193,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37056,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708096,"statfs":{"total":95995035648,"available":95957090304,"internally_reserved":0,"allocated":1753088,"data_stored":1193960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135934,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953553,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:57:08.638 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T11:57:08.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T11:57:08.843 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:08 smithi059 ceph-mon[30025]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:08.843 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:08 smithi059 ceph-mon[30025]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 -- 172.21.15.59:0/3920773005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 msgr2=0x7f0a20110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3920773005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a20110cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009ca0 tx=0x7f0a1802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 -- 172.21.15.59:0/3920773005 shutdown_connections 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3920773005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a20110cc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3920773005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a201049c0 0x7f0a20108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3920773005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a20104010 0x7f0a201043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 -- 172.21.15.59:0/3920773005 >> 172.21.15.59:0/3920773005 conn(0x7f0a200ffa50 msgr2=0x7f0a20101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 -- 172.21.15.59:0/3920773005 shutdown_connections 2024-09-05T11:57:08.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.863+0000 7f0a25a00640 1 -- 172.21.15.59:0/3920773005 wait complete. 2024-09-05T11:57:08.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.864+0000 7f0a25a00640 1 Processor -- start 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.864+0000 7f0a25a00640 1 -- start start 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a20104010 0x7f0a2019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 0x7f0a2019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201141a0 con 0x7f0a20109140 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a20114020 con 0x7f0a20104010 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a25a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a20114320 con 0x7f0a201049c0 2024-09-05T11:57:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a20104010 0x7f0a2019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a1e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 0x7f0a2019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59776/0 (socket says 172.21.15.59:59776) 2024-09-05T11:57:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.865+0000 7f0a1f400640 1 -- 172.21.15.59:0/3547151314 learned_addr learned my addr 172.21.15.59:0/3547151314 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 -- 172.21.15.59:0/3547151314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a20104010 msgr2=0x7f0a2019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a20104010 0x7f0a2019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 -- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 msgr2=0x7f0a2019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 -- 172.21.15.59:0/3547151314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2019f5b0 con 0x7f0a201049c0 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1f400640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a1e000640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 0x7f0a2019ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00e9f0 tx=0x7f0a0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a17400640 1 -- 172.21.15.59:0/3547151314 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00ccc0 con 0x7f0a201049c0 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a17400640 1 -- 172.21.15.59:0/3547151314 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c010910 con 0x7f0a201049c0 2024-09-05T11:57:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.866+0000 7f0a17400640 1 -- 172.21.15.59:0/3547151314 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c018960 con 0x7f0a201049c0 2024-09-05T11:57:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.867+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2019f8a0 con 0x7f0a201049c0 2024-09-05T11:57:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.867+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201a7180 con 0x7f0a201049c0 2024-09-05T11:57:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 Processor -- start 2024-09-05T11:57:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a17400640 1 -- 172.21.15.59:0/3547151314 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a0c010ac0 con 0x7f0a201049c0 2024-09-05T11:57:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 -- start start 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 0x5565fa35ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa369830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa36c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5565fa2c53e0 con 0x5565fa35eb90 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565fa26cd80 con 0x5565fa369d70 2024-09-05T11:57:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.870+0000 7f0a283bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565fa2c13a0 con 0x5565fa35f4b0 2024-09-05T11:57:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 0x5565fa35ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa369830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 0x5565fa35ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59786/0 (socket says 172.21.15.59:59786) 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1f400640 1 -- 172.21.15.59:0/1948358536 learned_addr learned my addr 172.21.15.59:0/1948358536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1e000640 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa36c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 -- 172.21.15.59:0/1948358536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 msgr2=0x5565fa36c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa36c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 -- 172.21.15.59:0/1948358536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 msgr2=0x5565fa35ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 0x5565fa35ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.871+0000 7f0a1ea00640 1 -- 172.21.15.59:0/1948358536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565fa28bae0 con 0x5565fa35f4b0 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa369830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009c20 tx=0x7f0a10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4708e83917699af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a17400640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 0x7f0a040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a0be00640 1 -- 172.21.15.59:0/1948358536 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1001e070 con 0x5565fa35f4b0 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a0be00640 1 -- 172.21.15.59:0/1948358536 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10009dd0 con 0x5565fa35f4b0 2024-09-05T11:57:08.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a0be00640 1 -- 172.21.15.59:0/1948358536 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10018870 con 0x5565fa35f4b0 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.872+0000 7f0a17400640 1 -- 172.21.15.59:0/3547151314 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0a0c0cf760 con 0x7f0a201049c0 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a1f400640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 0x7f0a040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 -- 172.21.15.59:0/1948358536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 msgr2=0x5565fa369830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa369830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009c20 tx=0x7f0a10011810 comp rx=0 tx=0).stop 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a1f400640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 0x7f0a040a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a1802f3f0 tx=0x7f0a18005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 -- 172.21.15.59:0/1948358536 shutdown_connections 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa36c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa369830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1948358536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35eb90 0x5565fa35ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 -- 172.21.15.59:0/1948358536 >> 172.21.15.59:0/1948358536 conn(0x5565fa35b120 msgr2=0x5565fa35b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.873+0000 7f0a283bf740 1 -- 172.21.15.59:0/1948358536 shutdown_connections 2024-09-05T11:57:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.874+0000 7f0a283bf740 1 -- 172.21.15.59:0/1948358536 wait complete. 2024-09-05T11:57:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.874+0000 7f0a283bf740 1 -- start start 2024-09-05T11:57:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.874+0000 7f0a283bf740 1 Processor -- start 2024-09-05T11:57:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 0x5565fa366ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa367720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5565fa36c9e0 con 0x5565fa35f4b0 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565fa2800f0 con 0x5565fa369d70 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565fa36cb60 con 0x5565fa35eb90 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a283bf740 10 client.0 _ll_register_callbacks cb 0x5565fa2719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a1e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a1e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59788/0 (socket says 172.21.15.59:59788) 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a1e000640 1 -- 172.21.15.59:0/1109250524 learned_addr learned my addr 172.21.15.59:0/1109250524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a1f400640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa367720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.875+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 0x5565fa366ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 -- 172.21.15.59:0/1109250524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 msgr2=0x5565fa366ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 0x5565fa366ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 -- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 msgr2=0x5565fa367720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa367720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 -- 172.21.15.59:0/1109250524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565fa0363c0 con 0x5565fa35f4b0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 0x5565fa366ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a1e000640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a4290 tx=0x7f0a0c097f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc9f0ecd2aac54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a08c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.876+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c09bc00 con 0x5565fa35f4b0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a10009dd0 con 0x5565fa35f4b0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0a0c0cfb50 con 0x5565fa35f4b0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c09ff00 con 0x5565fa35f4b0 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 10 client.15174 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:08.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565fa3786e0 con 0x5565fa35f4b0 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10009dd0 con 0x5565fa35f4b0 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0a0c09b1e0 con 0x5565fa35f4b0 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 10 client.15174 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 10 client.15174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.877+0000 7f0a283bf740 20 client.15174 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa379ba0 0x5565fa37bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a283bf740 4 client.15174 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565fa2d2950 con 0x5565fa379ba0 2024-09-05T11:57:08.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa379ba0 0x5565fa37bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa379ba0 0x5565fa37bff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a00950ab070c70a server_cookie=fef4be77f03a4e58 in_seq=0 out_seq=0 2024-09-05T11:57:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.878+0000 7f0a08c00640 10 client.15174 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.879+0000 7f0a08c00640 1 -- 172.21.15.59:0/1109250524 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565fa2d2950 con 0x5565fa379ba0 2024-09-05T11:57:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.879+0000 7f0a08c00640 10 client.15174 handle_command_reply: tid=0 2024-09-05T11:57:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.879+0000 7f0a283bf740 1 client.15174 shutdown 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.879+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 msgr2=0x5565fa3671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.879+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0a4290 tx=0x7f0a0c097f60 comp rx=0 tx=0).stop 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 shutdown_connections 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa369d70 0x5565fa367720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa379ba0 0x5565fa37bff0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa35f4b0 0x5565fa3671e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 --2- 172.21.15.59:0/1109250524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35eb90 0x5565fa366ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a08c00640 0 client.15174 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 >> 172.21.15.59:0/1109250524 conn(0x5565fa35b120 msgr2=0x5565fa360e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 shutdown_connections 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 1 -- 172.21.15.59:0/1109250524 wait complete. 2024-09-05T11:57:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.880+0000 7f0a283bf740 20 client.15174 trim_cache size 0 max 16384 2024-09-05T11:57:08.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 Processor -- start 2024-09-05T11:57:08.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 -- start start 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa3818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa381e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201af230 con 0x5565fa369d70 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201b1ab0 con 0x5565fa35eb90 2024-09-05T11:57:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.899+0000 7f0a283bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201af8a0 con 0x5565fa35f4b0 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa381e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51024/0 (socket says 172.21.15.59:51024) 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa381e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59800/0 (socket says 172.21.15.59:59800) 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 -- 172.21.15.59:0/3922215936 learned_addr learned my addr 172.21.15.59:0/3922215936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1e000640 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa3818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 -- 172.21.15.59:0/3922215936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 msgr2=0x5565fa3818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa3818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 -- 172.21.15.59:0/3922215936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 msgr2=0x5565fa381e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa381e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.900+0000 7f0a1ea00640 1 -- 172.21.15.59:0/3922215936 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565fa35af50 con 0x5565fa35eb90 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.901+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009c20 tx=0x7f0a10018450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952e648b5cded764 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.901+0000 7f0a0b400640 1 -- 172.21.15.59:0/3922215936 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1001e040 con 0x5565fa35eb90 2024-09-05T11:57:08.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.901+0000 7f0a0b400640 1 -- 172.21.15.59:0/3922215936 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10005600 con 0x5565fa35eb90 2024-09-05T11:57:08.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.901+0000 7f0a0b400640 1 -- 172.21.15.59:0/3922215936 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10018840 con 0x5565fa35eb90 2024-09-05T11:57:08.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 -- 172.21.15.59:0/3922215936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 msgr2=0x5565fa368c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009c20 tx=0x7f0a10018450 comp rx=0 tx=0).stop 2024-09-05T11:57:08.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 -- 172.21.15.59:0/3922215936 shutdown_connections 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa381e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa3818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 --2- 172.21.15.59:0/3922215936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa368c10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 -- 172.21.15.59:0/3922215936 >> 172.21.15.59:0/3922215936 conn(0x5565fa35b120 msgr2=0x5565fa368500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 -- 172.21.15.59:0/3922215936 shutdown_connections 2024-09-05T11:57:08.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.902+0000 7f0a283bf740 1 -- 172.21.15.59:0/3922215936 wait complete. 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 -- start start 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 Processor -- start 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa36d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa36d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa3e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201aff70 con 0x5565fa369d70 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201b13e0 con 0x5565fa35eb90 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201b0640 con 0x5565fa35f4b0 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.903+0000 7f0a283bf740 10 client.0 _ll_register_callbacks cb 0x7f0a0c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa3e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa36d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa36d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa3e2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59812/0 (socket says 172.21.15.59:59812) 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1f400640 1 -- 172.21.15.59:0/4187783530 learned_addr learned my addr 172.21.15.59:0/4187783530 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 -- 172.21.15.59:0/4187783530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 msgr2=0x5565fa36d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa36d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 -- 172.21.15.59:0/4187783530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 msgr2=0x5565fa3e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa3e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 -- 172.21.15.59:0/4187783530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565fa3e2ec0 con 0x5565fa35eb90 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa36d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009bf0 tx=0x7f0a10005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c1e911c254b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.904+0000 7f0a08c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1001e040 con 0x5565fa35eb90 2024-09-05T11:57:08.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a10005600 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0a100035f0 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0a10009dd0 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a283bf740 10 client.25184 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565fa3e4190 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.905+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a10005600 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0a100189c0 con 0x5565fa35eb90 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 10 client.25184 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 20 client.25184 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa3e45b0 0x5565fa3e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:08.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 4 client.25184 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:57:08.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565fa2cce60 con 0x5565fa3e45b0 2024-09-05T11:57:08.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.906+0000 7f0a1e000640 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa3e45b0 0x5565fa3e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a1e000640 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa3e45b0 0x5565fa3e69a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d96a981c64af5fe server_cookie=c9cdb36c184980b5 in_seq=0 out_seq=0 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a08c00640 10 client.25184 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a08c00640 1 -- 172.21.15.59:0/4187783530 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5565fa2cce60 con 0x5565fa3e45b0 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a08c00640 10 client.25184 handle_command_reply: tid=0 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 client.25184 shutdown 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 msgr2=0x5565fa36d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa36d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009bf0 tx=0x7f0a10005360 comp rx=0 tx=0).stop 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 shutdown_connections 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565fa3e45b0 0x5565fa3e69a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565fa369d70 0x5565fa3e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565fa35f4b0 0x5565fa36d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 --2- 172.21.15.59:0/4187783530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565fa35eb90 0x5565fa36d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a08c00640 0 client.25184 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 >> 172.21.15.59:0/4187783530 conn(0x5565fa35b120 msgr2=0x5565fa367480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 shutdown_connections 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.907+0000 7f0a283bf740 1 -- 172.21.15.59:0/4187783530 wait complete. 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a283bf740 20 client.25184 trim_cache size 0 max 16384 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T11:57:08.914 INFO:teuthology.orchestra.run.smithi059.stdout: "22c2d830-c704-4889-a14e-f86c35288f9a": { 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "22c2d830-c704-4889-a14e-f86c35288f9a", 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:08.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 msgr2=0x7f0a040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 0x7f0a040a50c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a1802f3f0 tx=0x7f0a18005d80 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 msgr2=0x7f0a2019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 0x7f0a2019ab40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00e9f0 tx=0x7f0a0c00eeb0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.908+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 shutdown_connections 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0a040a2c00 0x7f0a040a50c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a20109140 0x7f0a2019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a201049c0 0x7f0a2019ab40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 --2- 172.21.15.59:0/3547151314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a20104010 0x7f0a2019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 >> 172.21.15.59:0/3547151314 conn(0x7f0a200ffa50 msgr2=0x7f0a20100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 shutdown_connections 2024-09-05T11:57:08.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:08.909+0000 7f0a25a00640 1 -- 172.21.15.59:0/3547151314 wait complete. 2024-09-05T11:57:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:08 smithi189 ceph-mon[34772]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:08 smithi189 ceph-mon[34772]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:08 smithi089 ceph-mon[34499]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:08 smithi089 ceph-mon[34499]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:09.734 INFO:tasks.cephfs.filesystem:scrub status for tag:22c2d830-c704-4889-a14e-f86c35288f9a - {'path': '/', 'tag': '22c2d830-c704-4889-a14e-f86c35288f9a', 'options': 'recursive,force'} 2024-09-05T11:57:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 -- 172.21.15.59:0/2030334670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 msgr2=0x7f103c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 --2- 172.21.15.59:0/2030334670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c108a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1028009900 tx=0x7f102802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 -- 172.21.15.59:0/2030334670 shutdown_connections 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 --2- 172.21.15.59:0/2030334670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c109140 0x7f103c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 --2- 172.21.15.59:0/2030334670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c108a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 --2- 172.21.15.59:0/2030334670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c104010 0x7f103c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 -- 172.21.15.59:0/2030334670 >> 172.21.15.59:0/2030334670 conn(0x7f103c0ffa50 msgr2=0x7f103c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 -- 172.21.15.59:0/2030334670 shutdown_connections 2024-09-05T11:57:10.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.422+0000 7f1041e00640 1 -- 172.21.15.59:0/2030334670 wait complete. 2024-09-05T11:57:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.423+0000 7f1041e00640 1 Processor -- start 2024-09-05T11:57:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 -- start start 2024-09-05T11:57:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c109140 0x7f103c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f103c114160 con 0x7f103c104010 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f103c113fe0 con 0x7f103c109140 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f1041e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1142e0 con 0x7f103c1049c0 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f103aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f103aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59824/0 (socket says 172.21.15.59:59824) 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f103aa00640 1 -- 172.21.15.59:0/4199328103 learned_addr learned my addr 172.21.15.59:0/4199328103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f103b400640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c109140 0x7f103c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.424+0000 7f103a000640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 -- 172.21.15.59:0/4199328103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 msgr2=0x7f103c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 -- 172.21.15.59:0/4199328103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c109140 msgr2=0x7f103c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c109140 0x7f103c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 -- 172.21.15.59:0/4199328103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19f640 con 0x7f103c104010 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f103aa00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c00ccc0 con 0x7f103c104010 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c00ce60 con 0x7f103c104010 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c19f930 con 0x7f103c104010 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c010690 con 0x7f103c104010 2024-09-05T11:57:10.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.425+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a7210 con 0x7f103c104010 2024-09-05T11:57:10.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.427+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c1043f0 con 0x7f103c104010 2024-09-05T11:57:10.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.429+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f102c0046c0 con 0x7f103c104010 2024-09-05T11:57:10.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.430+0000 7f1040a00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 0x7f10200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:10.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.430+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f102c0cfa30 con 0x7f103c104010 2024-09-05T11:57:10.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.430+0000 7f103a000640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 0x7f10200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:10.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.431+0000 7f103a000640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 0x7f10200a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f10280058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:10.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.433+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f102c0541b0 con 0x7f103c104010 2024-09-05T11:57:10.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.614+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 --> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f103c10c3f0 con 0x7f10200a2b80 2024-09-05T11:57:10.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.626+0000 7f1040a00640 1 -- 172.21.15.59:0/4199328103 <== mgr.14146 v2:172.21.15.59:6800/23374249 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340402 (secure 0 0 0) 0x7f1028031630 con 0x7f10200a2b80 2024-09-05T11:57:10.632 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:10.634 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 msgr2=0x7f10200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 0x7f10200a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f10280058d0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 msgr2=0x7f103c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e9f0 tx=0x7f102c00edf0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 shutdown_connections 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f10200a2b80 0x7f10200a5040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f103c109140 0x7f103c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f103c1049c0 0x7f103c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 --2- 172.21.15.59:0/4199328103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f103c104010 0x7f103c19a690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 >> 172.21.15.59:0/4199328103 conn(0x7f103c0ffa50 msgr2=0x7f103c100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 shutdown_connections 2024-09-05T11:57:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:10.631+0000 7f1041e00640 1 -- 172.21.15.59:0/4199328103 wait complete. 2024-09-05T11:57:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:10 smithi059 ceph-mon[30025]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:10 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:57:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:10 smithi189 ceph-mon[34772]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:10 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:57:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:10 smithi089 ceph-mon[34499]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:10 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:57:11.355 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-05T11:57:08.891814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662464,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":368,"num_read_kb":303,"num_write":281,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":720,"ondisk_log_size":720,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376232,"kb_used_data":14504,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565592,"statfs":{"total":1151940427776,"available":1151555166208,"internally_reserved":0,"allocated":14852096,"data_stored":8358314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003516"},"pg_stats":[{"pgid":"2.1f","version":"82'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:33.653029+0000","last_change":"2024-09-05T11:54:29.103734+0000","last_active":"2024-09-05T11:56:33.653029+0000","last_peered":"2024-09-05T11:56:33.653029+0000","last_clean":"2024-09-05T11:56:33.653029+0000","last_became_active":"2024-09-05T11:54:29.102026+0000","last_became_peered":"2024-09-05T11:54:29.102026+0000","last_unstale":"2024-09-05T11:56:33.653029+0000","last_undegraded":"2024-09-05T11:56:33.653029+0000","last_fullsized":"2024-09-05T11:56:33.653029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:39:22.078498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870720+0000","last_change":"2024-09-05T11:54:31.528465+0000","last_active":"2024-09-05T11:56:46.870720+0000","last_peered":"2024-09-05T11:56:46.870720+0000","last_clean":"2024-09-05T11:56:46.870720+0000","last_became_active":"2024-09-05T11:54:31.528081+0000","last_became_peered":"2024-09-05T11:54:31.528081+0000","last_unstale":"2024-09-05T11:56:46.870720+0000","last_undegraded":"2024-09-05T11:56:46.870720+0000","last_fullsized":"2024-09-05T11:56:46.870720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:40.901089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600152+0000","last_change":"2024-09-05T11:55:18.960937+0000","last_active":"2024-09-05T11:56:19.600152+0000","last_peered":"2024-09-05T11:56:19.600152+0000","last_clean":"2024-09-05T11:56:19.600152+0000","last_became_active":"2024-09-05T11:55:18.960423+0000","last_became_peered":"2024-09-05T11:55:18.960423+0000","last_unstale":"2024-09-05T11:56:19.600152+0000","last_undegraded":"2024-09-05T11:56:19.600152+0000","last_fullsized":"2024-09-05T11:56:19.600152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:46.960968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580728+0000","last_change":"2024-09-05T11:54:29.543106+0000","last_active":"2024-09-05T11:56:19.580728+0000","last_peered":"2024-09-05T11:56:19.580728+0000","last_clean":"2024-09-05T11:56:19.580728+0000","last_became_active":"2024-09-05T11:54:29.542377+0000","last_became_peered":"2024-09-05T11:54:29.542377+0000","last_unstale":"2024-09-05T11:56:19.580728+0000","last_undegraded":"2024-09-05T11:56:19.580728+0000","last_fullsized":"2024-09-05T11:56:19.580728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.636567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870490+0000","last_change":"2024-09-05T11:54:31.535484+0000","last_active":"2024-09-05T11:56:46.870490+0000","last_peered":"2024-09-05T11:56:46.870490+0000","last_clean":"2024-09-05T11:56:46.870490+0000","last_became_active":"2024-09-05T11:54:31.535088+0000","last_became_peered":"2024-09-05T11:54:31.535088+0000","last_unstale":"2024-09-05T11:56:46.870490+0000","last_undegraded":"2024-09-05T11:56:46.870490+0000","last_fullsized":"2024-09-05T11:56:46.870490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:19.588723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577131+0000","last_change":"2024-09-05T11:55:18.960868+0000","last_active":"2024-09-05T11:56:19.577131+0000","last_peered":"2024-09-05T11:56:19.577131+0000","last_clean":"2024-09-05T11:56:19.577131+0000","last_became_active":"2024-09-05T11:55:18.960503+0000","last_became_peered":"2024-09-05T11:55:18.960503+0000","last_unstale":"2024-09-05T11:56:19.577131+0000","last_undegraded":"2024-09-05T11:56:19.577131+0000","last_fullsized":"2024-09-05T11:56:19.577131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:34:14.197943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581425+0000","last_change":"2024-09-05T11:54:29.101138+0000","last_active":"2024-09-05T11:56:19.581425+0000","last_peered":"2024-09-05T11:56:19.581425+0000","last_clean":"2024-09-05T11:56:19.581425+0000","last_became_active":"2024-09-05T11:54:29.100853+0000","last_became_peered":"2024-09-05T11:54:29.100853+0000","last_unstale":"2024-09-05T11:56:19.581425+0000","last_undegraded":"2024-09-05T11:56:19.581425+0000","last_fullsized":"2024-09-05T11:56:19.581425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:04.272441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874976+0000","last_change":"2024-09-05T11:54:31.525496+0000","last_active":"2024-09-05T11:56:46.874976+0000","last_peered":"2024-09-05T11:56:46.874976+0000","last_clean":"2024-09-05T11:56:46.874976+0000","last_became_active":"2024-09-05T11:54:31.525132+0000","last_became_peered":"2024-09-05T11:54:31.525132+0000","last_unstale":"2024-09-05T11:56:46.874976+0000","last_undegraded":"2024-09-05T11:56:46.874976+0000","last_fullsized":"2024-09-05T11:56:46.874976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:00:05.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578413+0000","last_change":"2024-09-05T11:54:38.608521+0000","last_active":"2024-09-05T11:56:19.578413+0000","last_peered":"2024-09-05T11:56:19.578413+0000","last_clean":"2024-09-05T11:56:19.578413+0000","last_became_active":"2024-09-05T11:54:38.608120+0000","last_became_peered":"2024-09-05T11:54:38.608120+0000","last_unstale":"2024-09-05T11:56:19.578413+0000","last_undegraded":"2024-09-05T11:56:19.578413+0000","last_fullsized":"2024-09-05T11:56:19.578413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:24.396369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581278+0000","last_change":"2024-09-05T11:54:29.099653+0000","last_active":"2024-09-05T11:56:19.581278+0000","last_peered":"2024-09-05T11:56:19.581278+0000","last_clean":"2024-09-05T11:56:19.581278+0000","last_became_active":"2024-09-05T11:54:29.099337+0000","last_became_peered":"2024-09-05T11:54:29.099337+0000","last_unstale":"2024-09-05T11:56:19.581278+0000","last_undegraded":"2024-09-05T11:56:19.581278+0000","last_fullsized":"2024-09-05T11:56:19.581278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:31.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872911+0000","last_change":"2024-09-05T11:54:31.528150+0000","last_active":"2024-09-05T11:56:46.872911+0000","last_peered":"2024-09-05T11:56:46.872911+0000","last_clean":"2024-09-05T11:56:46.872911+0000","last_became_active":"2024-09-05T11:54:31.527809+0000","last_became_peered":"2024-09-05T11:54:31.527809+0000","last_unstale":"2024-09-05T11:56:46.872911+0000","last_undegraded":"2024-09-05T11:56:46.872911+0000","last_fullsized":"2024-09-05T11:56:46.872911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:51:43.343253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.582402+0000","last_change":"2024-09-05T11:56:19.582402+0000","last_active":"2024-09-05T11:56:19.582402+0000","last_peered":"2024-09-05T11:56:19.582402+0000","last_clean":"2024-09-05T11:56:19.582402+0000","last_became_active":"2024-09-05T11:56:19.581995+0000","last_became_peered":"2024-09-05T11:56:19.581995+0000","last_unstale":"2024-09-05T11:56:19.582402+0000","last_undegraded":"2024-09-05T11:56:19.582402+0000","last_fullsized":"2024-09-05T11:56:19.582402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:31:10.671757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877187+0000","last_change":"2024-09-05T11:54:31.532167+0000","last_active":"2024-09-05T11:56:46.877187+0000","last_peered":"2024-09-05T11:56:46.877187+0000","last_clean":"2024-09-05T11:56:46.877187+0000","last_became_active":"2024-09-05T11:54:31.531851+0000","last_became_peered":"2024-09-05T11:54:31.531851+0000","last_unstale":"2024-09-05T11:56:46.877187+0000","last_undegraded":"2024-09-05T11:56:46.877187+0000","last_fullsized":"2024-09-05T11:56:46.877187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:01.497634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"82'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581044+0000","last_change":"2024-09-05T11:54:29.103582+0000","last_active":"2024-09-05T11:56:19.581044+0000","last_peered":"2024-09-05T11:56:19.581044+0000","last_clean":"2024-09-05T11:56:19.581044+0000","last_became_active":"2024-09-05T11:54:29.103278+0000","last_became_peered":"2024-09-05T11:54:29.103278+0000","last_unstale":"2024-09-05T11:56:19.581044+0000","last_undegraded":"2024-09-05T11:56:19.581044+0000","last_fullsized":"2024-09-05T11:56:19.581044+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:39:07.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577428+0000","last_change":"2024-09-05T11:54:38.857672+0000","last_active":"2024-09-05T11:56:19.577428+0000","last_peered":"2024-09-05T11:56:19.577428+0000","last_clean":"2024-09-05T11:56:19.577428+0000","last_became_active":"2024-09-05T11:54:38.856872+0000","last_became_peered":"2024-09-05T11:54:38.856872+0000","last_unstale":"2024-09-05T11:56:19.577428+0000","last_undegraded":"2024-09-05T11:56:19.577428+0000","last_fullsized":"2024-09-05T11:56:19.577428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:49:00.647616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877233+0000","last_change":"2024-09-05T11:54:31.540084+0000","last_active":"2024-09-05T11:56:46.877233+0000","last_peered":"2024-09-05T11:56:46.877233+0000","last_clean":"2024-09-05T11:56:46.877233+0000","last_became_active":"2024-09-05T11:54:31.539830+0000","last_became_peered":"2024-09-05T11:54:31.539830+0000","last_unstale":"2024-09-05T11:56:46.877233+0000","last_undegraded":"2024-09-05T11:56:46.877233+0000","last_fullsized":"2024-09-05T11:56:46.877233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:56:12.397535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600510+0000","last_change":"2024-09-05T11:54:29.105791+0000","last_active":"2024-09-05T11:56:19.600510+0000","last_peered":"2024-09-05T11:56:19.600510+0000","last_clean":"2024-09-05T11:56:19.600510+0000","last_became_active":"2024-09-05T11:54:29.105311+0000","last_became_peered":"2024-09-05T11:54:29.105311+0000","last_unstale":"2024-09-05T11:56:19.600510+0000","last_undegraded":"2024-09-05T11:56:19.600510+0000","last_fullsized":"2024-09-05T11:56:19.600510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:42.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577416+0000","last_change":"2024-09-05T11:55:18.752237+0000","last_active":"2024-09-05T11:56:19.577416+0000","last_peered":"2024-09-05T11:56:19.577416+0000","last_clean":"2024-09-05T11:56:19.577416+0000","last_became_active":"2024-09-05T11:55:18.751642+0000","last_became_peered":"2024-09-05T11:55:18.751642+0000","last_unstale":"2024-09-05T11:56:19.577416+0000","last_undegraded":"2024-09-05T11:56:19.577416+0000","last_fullsized":"2024-09-05T11:56:19.577416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:37:06.429980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871444+0000","last_change":"2024-09-05T11:54:31.539926+0000","last_active":"2024-09-05T11:56:46.871444+0000","last_peered":"2024-09-05T11:56:46.871444+0000","last_clean":"2024-09-05T11:56:46.871444+0000","last_became_active":"2024-09-05T11:54:31.539458+0000","last_became_peered":"2024-09-05T11:54:31.539458+0000","last_unstale":"2024-09-05T11:56:46.871444+0000","last_undegraded":"2024-09-05T11:56:46.871444+0000","last_fullsized":"2024-09-05T11:56:46.871444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:23.052160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580176+0000","last_change":"2024-09-05T11:54:29.103642+0000","last_active":"2024-09-05T11:56:19.580176+0000","last_peered":"2024-09-05T11:56:19.580176+0000","last_clean":"2024-09-05T11:56:19.580176+0000","last_became_active":"2024-09-05T11:54:29.103222+0000","last_became_peered":"2024-09-05T11:54:29.103222+0000","last_unstale":"2024-09-05T11:56:19.580176+0000","last_undegraded":"2024-09-05T11:56:19.580176+0000","last_fullsized":"2024-09-05T11:56:19.580176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:35:48.934727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578102+0000","last_change":"2024-09-05T11:54:38.617556+0000","last_active":"2024-09-05T11:56:19.578102+0000","last_peered":"2024-09-05T11:56:19.578102+0000","last_clean":"2024-09-05T11:56:19.578102+0000","last_became_active":"2024-09-05T11:54:38.612778+0000","last_became_peered":"2024-09-05T11:54:38.612778+0000","last_unstale":"2024-09-05T11:56:19.578102+0000","last_undegraded":"2024-09-05T11:56:19.578102+0000","last_fullsized":"2024-09-05T11:56:19.578102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:00:25.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.830493+0000","last_change":"2024-09-05T11:54:31.523637+0000","last_active":"2024-09-05T11:56:46.830493+0000","last_peered":"2024-09-05T11:56:46.830493+0000","last_clean":"2024-09-05T11:56:46.830493+0000","last_became_active":"2024-09-05T11:54:31.523278+0000","last_became_peered":"2024-09-05T11:54:31.523278+0000","last_unstale":"2024-09-05T11:56:46.830493+0000","last_undegraded":"2024-09-05T11:56:46.830493+0000","last_fullsized":"2024-09-05T11:56:46.830493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:56:50.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600663+0000","last_change":"2024-09-05T11:54:29.542271+0000","last_active":"2024-09-05T11:56:19.600663+0000","last_peered":"2024-09-05T11:56:19.600663+0000","last_clean":"2024-09-05T11:56:19.600663+0000","last_became_active":"2024-09-05T11:54:29.541926+0000","last_became_peered":"2024-09-05T11:54:29.541926+0000","last_unstale":"2024-09-05T11:56:19.600663+0000","last_undegraded":"2024-09-05T11:56:19.600663+0000","last_fullsized":"2024-09-05T11:56:19.600663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:14:25.873106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577340+0000","last_change":"2024-09-05T11:54:38.857664+0000","last_active":"2024-09-05T11:56:19.577340+0000","last_peered":"2024-09-05T11:56:19.577340+0000","last_clean":"2024-09-05T11:56:19.577340+0000","last_became_active":"2024-09-05T11:54:38.857084+0000","last_became_peered":"2024-09-05T11:54:38.857084+0000","last_unstale":"2024-09-05T11:56:19.577340+0000","last_undegraded":"2024-09-05T11:56:19.577340+0000","last_fullsized":"2024-09-05T11:56:19.577340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:52.785668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872497+0000","last_change":"2024-09-05T11:54:31.540139+0000","last_active":"2024-09-05T11:56:46.872497+0000","last_peered":"2024-09-05T11:56:46.872497+0000","last_clean":"2024-09-05T11:56:46.872497+0000","last_became_active":"2024-09-05T11:54:31.539200+0000","last_became_peered":"2024-09-05T11:54:31.539200+0000","last_unstale":"2024-09-05T11:56:46.872497+0000","last_undegraded":"2024-09-05T11:56:46.872497+0000","last_fullsized":"2024-09-05T11:56:46.872497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:08:25.317311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577977+0000","last_change":"2024-09-05T11:54:29.257362+0000","last_active":"2024-09-05T11:56:19.577977+0000","last_peered":"2024-09-05T11:56:19.577977+0000","last_clean":"2024-09-05T11:56:19.577977+0000","last_became_active":"2024-09-05T11:54:29.257066+0000","last_became_peered":"2024-09-05T11:54:29.257066+0000","last_unstale":"2024-09-05T11:56:19.577977+0000","last_undegraded":"2024-09-05T11:56:19.577977+0000","last_fullsized":"2024-09-05T11:56:19.577977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:06.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600212+0000","last_change":"2024-09-05T11:54:38.602169+0000","last_active":"2024-09-05T11:56:19.600212+0000","last_peered":"2024-09-05T11:56:19.600212+0000","last_clean":"2024-09-05T11:56:19.600212+0000","last_became_active":"2024-09-05T11:54:38.601741+0000","last_became_peered":"2024-09-05T11:54:38.601741+0000","last_unstale":"2024-09-05T11:56:19.600212+0000","last_undegraded":"2024-09-05T11:56:19.600212+0000","last_fullsized":"2024-09-05T11:56:19.600212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:59:57.553875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872511+0000","last_change":"2024-09-05T11:54:31.529520+0000","last_active":"2024-09-05T11:56:46.872511+0000","last_peered":"2024-09-05T11:56:46.872511+0000","last_clean":"2024-09-05T11:56:46.872511+0000","last_became_active":"2024-09-05T11:54:31.529150+0000","last_became_peered":"2024-09-05T11:54:31.529150+0000","last_unstale":"2024-09-05T11:56:46.872511+0000","last_undegraded":"2024-09-05T11:56:46.872511+0000","last_fullsized":"2024-09-05T11:56:46.872511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:36:13.980973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:47.573998+0000","last_change":"2024-09-05T11:56:19.605859+0000","last_active":"2024-09-05T11:56:47.573998+0000","last_peered":"2024-09-05T11:56:47.573998+0000","last_clean":"2024-09-05T11:56:47.573998+0000","last_became_active":"2024-09-05T11:56:19.603112+0000","last_became_peered":"2024-09-05T11:56:19.603112+0000","last_unstale":"2024-09-05T11:56:47.573998+0000","last_undegraded":"2024-09-05T11:56:47.573998+0000","last_fullsized":"2024-09-05T11:56:47.573998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:50.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579781+0000","last_change":"2024-09-05T11:54:38.856292+0000","last_active":"2024-09-05T11:56:19.579781+0000","last_peered":"2024-09-05T11:56:19.579781+0000","last_clean":"2024-09-05T11:56:19.579781+0000","last_became_active":"2024-09-05T11:54:38.855789+0000","last_became_peered":"2024-09-05T11:54:38.855789+0000","last_unstale":"2024-09-05T11:56:19.579781+0000","last_undegraded":"2024-09-05T11:56:19.579781+0000","last_fullsized":"2024-09-05T11:56:19.579781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:12:44.485120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869826+0000","last_change":"2024-09-05T11:54:31.523532+0000","last_active":"2024-09-05T11:56:46.869826+0000","last_peered":"2024-09-05T11:56:46.869826+0000","last_clean":"2024-09-05T11:56:46.869826+0000","last_became_active":"2024-09-05T11:54:31.521405+0000","last_became_peered":"2024-09-05T11:54:31.521405+0000","last_unstale":"2024-09-05T11:56:46.869826+0000","last_undegraded":"2024-09-05T11:56:46.869826+0000","last_fullsized":"2024-09-05T11:56:46.869826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:53:31.582675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581428+0000","last_change":"2024-09-05T11:54:29.543204+0000","last_active":"2024-09-05T11:56:19.581428+0000","last_peered":"2024-09-05T11:56:19.581428+0000","last_clean":"2024-09-05T11:56:19.581428+0000","last_became_active":"2024-09-05T11:54:29.542883+0000","last_became_peered":"2024-09-05T11:54:29.542883+0000","last_unstale":"2024-09-05T11:56:19.581428+0000","last_undegraded":"2024-09-05T11:56:19.581428+0000","last_fullsized":"2024-09-05T11:56:19.581428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:29:55.627971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602050+0000","last_change":"2024-09-05T11:54:38.605565+0000","last_active":"2024-09-05T11:56:19.602050+0000","last_peered":"2024-09-05T11:56:19.602050+0000","last_clean":"2024-09-05T11:56:19.602050+0000","last_became_active":"2024-09-05T11:54:38.605187+0000","last_became_peered":"2024-09-05T11:54:38.605187+0000","last_unstale":"2024-09-05T11:56:19.602050+0000","last_undegraded":"2024-09-05T11:56:19.602050+0000","last_fullsized":"2024-09-05T11:56:19.602050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:13:32.163859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.856862+0000","last_change":"2024-09-05T11:54:31.518787+0000","last_active":"2024-09-05T11:56:46.856862+0000","last_peered":"2024-09-05T11:56:46.856862+0000","last_clean":"2024-09-05T11:56:46.856862+0000","last_became_active":"2024-09-05T11:54:31.518292+0000","last_became_peered":"2024-09-05T11:54:31.518292+0000","last_unstale":"2024-09-05T11:56:46.856862+0000","last_undegraded":"2024-09-05T11:56:46.856862+0000","last_fullsized":"2024-09-05T11:56:46.856862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:52:35.790970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578250+0000","last_change":"2024-09-05T11:54:29.543423+0000","last_active":"2024-09-05T11:56:19.578250+0000","last_peered":"2024-09-05T11:56:19.578250+0000","last_clean":"2024-09-05T11:56:19.578250+0000","last_became_active":"2024-09-05T11:54:29.542426+0000","last_became_peered":"2024-09-05T11:54:29.542426+0000","last_unstale":"2024-09-05T11:56:19.578250+0000","last_undegraded":"2024-09-05T11:56:19.578250+0000","last_fullsized":"2024-09-05T11:56:19.578250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:22.528949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580578+0000","last_change":"2024-09-05T11:54:38.856642+0000","last_active":"2024-09-05T11:56:19.580578+0000","last_peered":"2024-09-05T11:56:19.580578+0000","last_clean":"2024-09-05T11:56:19.580578+0000","last_became_active":"2024-09-05T11:54:38.856116+0000","last_became_peered":"2024-09-05T11:54:38.856116+0000","last_unstale":"2024-09-05T11:56:19.580578+0000","last_undegraded":"2024-09-05T11:56:19.580578+0000","last_fullsized":"2024-09-05T11:56:19.580578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:19:30.010505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869882+0000","last_change":"2024-09-05T11:54:31.534965+0000","last_active":"2024-09-05T11:56:46.869882+0000","last_peered":"2024-09-05T11:56:46.869882+0000","last_clean":"2024-09-05T11:56:46.869882+0000","last_became_active":"2024-09-05T11:54:31.534635+0000","last_became_peered":"2024-09-05T11:54:31.534635+0000","last_unstale":"2024-09-05T11:56:46.869882+0000","last_undegraded":"2024-09-05T11:56:46.869882+0000","last_fullsized":"2024-09-05T11:56:46.869882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:43:52.678587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577938+0000","last_change":"2024-09-05T11:54:29.106824+0000","last_active":"2024-09-05T11:56:19.577938+0000","last_peered":"2024-09-05T11:56:19.577938+0000","last_clean":"2024-09-05T11:56:19.577938+0000","last_became_active":"2024-09-05T11:54:29.106344+0000","last_became_peered":"2024-09-05T11:54:29.106344+0000","last_unstale":"2024-09-05T11:56:19.577938+0000","last_undegraded":"2024-09-05T11:56:19.577938+0000","last_fullsized":"2024-09-05T11:56:19.577938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:47.430666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574514+0000","last_change":"2024-09-05T11:54:38.604727+0000","last_active":"2024-09-05T11:56:19.574514+0000","last_peered":"2024-09-05T11:56:19.574514+0000","last_clean":"2024-09-05T11:56:19.574514+0000","last_became_active":"2024-09-05T11:54:38.604284+0000","last_became_peered":"2024-09-05T11:54:38.604284+0000","last_unstale":"2024-09-05T11:56:19.574514+0000","last_undegraded":"2024-09-05T11:56:19.574514+0000","last_fullsized":"2024-09-05T11:56:19.574514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:04:10.836313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872916+0000","last_change":"2024-09-05T11:54:31.523249+0000","last_active":"2024-09-05T11:56:46.872916+0000","last_peered":"2024-09-05T11:56:46.872916+0000","last_clean":"2024-09-05T11:56:46.872916+0000","last_became_active":"2024-09-05T11:54:31.522948+0000","last_became_peered":"2024-09-05T11:54:31.522948+0000","last_unstale":"2024-09-05T11:56:46.872916+0000","last_undegraded":"2024-09-05T11:56:46.872916+0000","last_fullsized":"2024-09-05T11:56:46.872916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:13:20.660407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574053+0000","last_change":"2024-09-05T11:54:29.543852+0000","last_active":"2024-09-05T11:56:19.574053+0000","last_peered":"2024-09-05T11:56:19.574053+0000","last_clean":"2024-09-05T11:56:19.574053+0000","last_became_active":"2024-09-05T11:54:29.542285+0000","last_became_peered":"2024-09-05T11:54:29.542285+0000","last_unstale":"2024-09-05T11:56:19.574053+0000","last_undegraded":"2024-09-05T11:56:19.574053+0000","last_fullsized":"2024-09-05T11:56:19.574053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:50:44.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574535+0000","last_change":"2024-09-05T11:54:38.856796+0000","last_active":"2024-09-05T11:56:19.574535+0000","last_peered":"2024-09-05T11:56:19.574535+0000","last_clean":"2024-09-05T11:56:19.574535+0000","last_became_active":"2024-09-05T11:54:38.855831+0000","last_became_peered":"2024-09-05T11:54:38.855831+0000","last_unstale":"2024-09-05T11:56:19.574535+0000","last_undegraded":"2024-09-05T11:56:19.574535+0000","last_fullsized":"2024-09-05T11:56:19.574535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:36.345039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.863942+0000","last_change":"2024-09-05T11:54:31.528006+0000","last_active":"2024-09-05T11:56:46.863942+0000","last_peered":"2024-09-05T11:56:46.863942+0000","last_clean":"2024-09-05T11:56:46.863942+0000","last_became_active":"2024-09-05T11:54:31.527157+0000","last_became_peered":"2024-09-05T11:54:31.527157+0000","last_unstale":"2024-09-05T11:56:46.863942+0000","last_undegraded":"2024-09-05T11:56:46.863942+0000","last_fullsized":"2024-09-05T11:56:46.863942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:45.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577630+0000","last_change":"2024-09-05T11:54:29.100178+0000","last_active":"2024-09-05T11:56:19.577630+0000","last_peered":"2024-09-05T11:56:19.577630+0000","last_clean":"2024-09-05T11:56:19.577630+0000","last_became_active":"2024-09-05T11:54:29.099834+0000","last_became_peered":"2024-09-05T11:54:29.099834+0000","last_unstale":"2024-09-05T11:56:19.577630+0000","last_undegraded":"2024-09-05T11:56:19.577630+0000","last_fullsized":"2024-09-05T11:56:19.577630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:29.999928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575439+0000","last_change":"2024-09-05T11:54:38.857310+0000","last_active":"2024-09-05T11:56:19.575439+0000","last_peered":"2024-09-05T11:56:19.575439+0000","last_clean":"2024-09-05T11:56:19.575439+0000","last_became_active":"2024-09-05T11:54:38.856717+0000","last_became_peered":"2024-09-05T11:54:38.856717+0000","last_unstale":"2024-09-05T11:56:19.575439+0000","last_undegraded":"2024-09-05T11:56:19.575439+0000","last_fullsized":"2024-09-05T11:56:19.575439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:25:11.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869867+0000","last_change":"2024-09-05T11:54:31.537387+0000","last_active":"2024-09-05T11:56:46.869867+0000","last_peered":"2024-09-05T11:56:46.869867+0000","last_clean":"2024-09-05T11:56:46.869867+0000","last_became_active":"2024-09-05T11:54:31.536850+0000","last_became_peered":"2024-09-05T11:54:31.536850+0000","last_unstale":"2024-09-05T11:56:46.869867+0000","last_undegraded":"2024-09-05T11:56:46.869867+0000","last_fullsized":"2024-09-05T11:56:46.869867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:35:27.493882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578780+0000","last_change":"2024-09-05T11:54:29.104658+0000","last_active":"2024-09-05T11:56:19.578780+0000","last_peered":"2024-09-05T11:56:19.578780+0000","last_clean":"2024-09-05T11:56:19.578780+0000","last_became_active":"2024-09-05T11:54:29.104249+0000","last_became_peered":"2024-09-05T11:54:29.104249+0000","last_unstale":"2024-09-05T11:56:19.578780+0000","last_undegraded":"2024-09-05T11:56:19.578780+0000","last_fullsized":"2024-09-05T11:56:19.578780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:13:21.268014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578065+0000","last_change":"2024-09-05T11:53:12.383957+0000","last_active":"2024-09-05T11:56:19.578065+0000","last_peered":"2024-09-05T11:56:19.578065+0000","last_clean":"2024-09-05T11:56:19.578065+0000","last_became_active":"2024-09-05T11:53:12.383572+0000","last_became_peered":"2024-09-05T11:53:12.383572+0000","last_unstale":"2024-09-05T11:56:19.578065+0000","last_undegraded":"2024-09-05T11:56:19.578065+0000","last_fullsized":"2024-09-05T11:56:19.578065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:49:37.375919+0000","last_clean_scrub_stamp":"2024-09-05T11:49:37.375919+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:47:36.397002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578388+0000","last_change":"2024-09-05T11:54:38.606006+0000","last_active":"2024-09-05T11:56:19.578388+0000","last_peered":"2024-09-05T11:56:19.578388+0000","last_clean":"2024-09-05T11:56:19.578388+0000","last_became_active":"2024-09-05T11:54:38.605586+0000","last_became_peered":"2024-09-05T11:54:38.605586+0000","last_unstale":"2024-09-05T11:56:19.578388+0000","last_undegraded":"2024-09-05T11:56:19.578388+0000","last_fullsized":"2024-09-05T11:56:19.578388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:50.595723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877322+0000","last_change":"2024-09-05T11:54:31.528038+0000","last_active":"2024-09-05T11:56:46.877322+0000","last_peered":"2024-09-05T11:56:46.877322+0000","last_clean":"2024-09-05T11:56:46.877322+0000","last_became_active":"2024-09-05T11:54:31.527673+0000","last_became_peered":"2024-09-05T11:54:31.527673+0000","last_unstale":"2024-09-05T11:56:46.877322+0000","last_undegraded":"2024-09-05T11:56:46.877322+0000","last_fullsized":"2024-09-05T11:56:46.877322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:59.631374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600843+0000","last_change":"2024-09-05T11:54:29.542891+0000","last_active":"2024-09-05T11:56:19.600843+0000","last_peered":"2024-09-05T11:56:19.600843+0000","last_clean":"2024-09-05T11:56:19.600843+0000","last_became_active":"2024-09-05T11:54:29.542294+0000","last_became_peered":"2024-09-05T11:54:29.542294+0000","last_unstale":"2024-09-05T11:56:19.600843+0000","last_undegraded":"2024-09-05T11:56:19.600843+0000","last_fullsized":"2024-09-05T11:56:19.600843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:51:53.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600878+0000","last_change":"2024-09-05T11:54:38.597002+0000","last_active":"2024-09-05T11:56:19.600878+0000","last_peered":"2024-09-05T11:56:19.600878+0000","last_clean":"2024-09-05T11:56:19.600878+0000","last_became_active":"2024-09-05T11:54:38.596577+0000","last_became_peered":"2024-09-05T11:54:38.596577+0000","last_unstale":"2024-09-05T11:56:19.600878+0000","last_undegraded":"2024-09-05T11:56:19.600878+0000","last_fullsized":"2024-09-05T11:56:19.600878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:28:17.409687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873308+0000","last_change":"2024-09-05T11:54:31.530600+0000","last_active":"2024-09-05T11:56:46.873308+0000","last_peered":"2024-09-05T11:56:46.873308+0000","last_clean":"2024-09-05T11:56:46.873308+0000","last_became_active":"2024-09-05T11:54:31.530271+0000","last_became_peered":"2024-09-05T11:54:31.530271+0000","last_unstale":"2024-09-05T11:56:46.873308+0000","last_undegraded":"2024-09-05T11:56:46.873308+0000","last_fullsized":"2024-09-05T11:56:46.873308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:15:58.006479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575481+0000","last_change":"2024-09-05T11:54:29.100226+0000","last_active":"2024-09-05T11:56:19.575481+0000","last_peered":"2024-09-05T11:56:19.575481+0000","last_clean":"2024-09-05T11:56:19.575481+0000","last_became_active":"2024-09-05T11:54:29.099942+0000","last_became_peered":"2024-09-05T11:54:29.099942+0000","last_unstale":"2024-09-05T11:56:19.575481+0000","last_undegraded":"2024-09-05T11:56:19.575481+0000","last_fullsized":"2024-09-05T11:56:19.575481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:45.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600535+0000","last_change":"2024-09-05T11:54:38.605871+0000","last_active":"2024-09-05T11:56:19.600535+0000","last_peered":"2024-09-05T11:56:19.600535+0000","last_clean":"2024-09-05T11:56:19.600535+0000","last_became_active":"2024-09-05T11:54:38.604421+0000","last_became_peered":"2024-09-05T11:54:38.604421+0000","last_unstale":"2024-09-05T11:56:19.600535+0000","last_undegraded":"2024-09-05T11:56:19.600535+0000","last_fullsized":"2024-09-05T11:56:19.600535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:30.567657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874669+0000","last_change":"2024-09-05T11:54:31.529824+0000","last_active":"2024-09-05T11:56:46.874669+0000","last_peered":"2024-09-05T11:56:46.874669+0000","last_clean":"2024-09-05T11:56:46.874669+0000","last_became_active":"2024-09-05T11:54:31.529472+0000","last_became_peered":"2024-09-05T11:54:31.529472+0000","last_unstale":"2024-09-05T11:56:46.874669+0000","last_undegraded":"2024-09-05T11:56:46.874669+0000","last_fullsized":"2024-09-05T11:56:46.874669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:40:47.312116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574724+0000","last_change":"2024-09-05T11:54:29.101158+0000","last_active":"2024-09-05T11:56:19.574724+0000","last_peered":"2024-09-05T11:56:19.574724+0000","last_clean":"2024-09-05T11:56:19.574724+0000","last_became_active":"2024-09-05T11:54:29.100253+0000","last_became_peered":"2024-09-05T11:54:29.100253+0000","last_unstale":"2024-09-05T11:56:19.574724+0000","last_undegraded":"2024-09-05T11:56:19.574724+0000","last_fullsized":"2024-09-05T11:56:19.574724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:24.099978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574621+0000","last_change":"2024-09-05T11:54:38.601570+0000","last_active":"2024-09-05T11:56:19.574621+0000","last_peered":"2024-09-05T11:56:19.574621+0000","last_clean":"2024-09-05T11:56:19.574621+0000","last_became_active":"2024-09-05T11:54:38.601195+0000","last_became_peered":"2024-09-05T11:54:38.601195+0000","last_unstale":"2024-09-05T11:56:19.574621+0000","last_undegraded":"2024-09-05T11:56:19.574621+0000","last_fullsized":"2024-09-05T11:56:19.574621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:19:57.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867717+0000","last_change":"2024-09-05T11:54:31.522828+0000","last_active":"2024-09-05T11:56:46.867717+0000","last_peered":"2024-09-05T11:56:46.867717+0000","last_clean":"2024-09-05T11:56:46.867717+0000","last_became_active":"2024-09-05T11:54:31.522482+0000","last_became_peered":"2024-09-05T11:54:31.522482+0000","last_unstale":"2024-09-05T11:56:46.867717+0000","last_undegraded":"2024-09-05T11:56:46.867717+0000","last_fullsized":"2024-09-05T11:56:46.867717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:07.523136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581489+0000","last_change":"2024-09-05T11:54:29.095836+0000","last_active":"2024-09-05T11:56:19.581489+0000","last_peered":"2024-09-05T11:56:19.581489+0000","last_clean":"2024-09-05T11:56:19.581489+0000","last_became_active":"2024-09-05T11:54:29.095541+0000","last_became_peered":"2024-09-05T11:54:29.095541+0000","last_unstale":"2024-09-05T11:56:19.581489+0000","last_undegraded":"2024-09-05T11:56:19.581489+0000","last_fullsized":"2024-09-05T11:56:19.581489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:41.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574696+0000","last_change":"2024-09-05T11:54:38.612969+0000","last_active":"2024-09-05T11:56:19.574696+0000","last_peered":"2024-09-05T11:56:19.574696+0000","last_clean":"2024-09-05T11:56:19.574696+0000","last_became_active":"2024-09-05T11:54:38.612453+0000","last_became_peered":"2024-09-05T11:54:38.612453+0000","last_unstale":"2024-09-05T11:56:19.574696+0000","last_undegraded":"2024-09-05T11:56:19.574696+0000","last_fullsized":"2024-09-05T11:56:19.574696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:06.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875118+0000","last_change":"2024-09-05T11:54:31.527967+0000","last_active":"2024-09-05T11:56:46.875118+0000","last_peered":"2024-09-05T11:56:46.875118+0000","last_clean":"2024-09-05T11:56:46.875118+0000","last_became_active":"2024-09-05T11:54:31.527511+0000","last_became_peered":"2024-09-05T11:54:31.527511+0000","last_unstale":"2024-09-05T11:56:46.875118+0000","last_undegraded":"2024-09-05T11:56:46.875118+0000","last_fullsized":"2024-09-05T11:56:46.875118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:06.134841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"82'8","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.580543+0000","last_change":"2024-09-05T11:54:29.543362+0000","last_active":"2024-09-05T11:56:19.580543+0000","last_peered":"2024-09-05T11:56:19.580543+0000","last_clean":"2024-09-05T11:56:19.580543+0000","last_became_active":"2024-09-05T11:54:29.543040+0000","last_became_peered":"2024-09-05T11:54:29.543040+0000","last_unstale":"2024-09-05T11:56:19.580543+0000","last_undegraded":"2024-09-05T11:56:19.580543+0000","last_fullsized":"2024-09-05T11:56:19.580543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:59:06.648539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574712+0000","last_change":"2024-09-05T11:54:38.615319+0000","last_active":"2024-09-05T11:56:19.574712+0000","last_peered":"2024-09-05T11:56:19.574712+0000","last_clean":"2024-09-05T11:56:19.574712+0000","last_became_active":"2024-09-05T11:54:38.611350+0000","last_became_peered":"2024-09-05T11:54:38.611350+0000","last_unstale":"2024-09-05T11:56:19.574712+0000","last_undegraded":"2024-09-05T11:56:19.574712+0000","last_fullsized":"2024-09-05T11:56:19.574712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:33.033504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872968+0000","last_change":"2024-09-05T11:54:31.523185+0000","last_active":"2024-09-05T11:56:46.872968+0000","last_peered":"2024-09-05T11:56:46.872968+0000","last_clean":"2024-09-05T11:56:46.872968+0000","last_became_active":"2024-09-05T11:54:31.522883+0000","last_became_peered":"2024-09-05T11:54:31.522883+0000","last_unstale":"2024-09-05T11:56:46.872968+0000","last_undegraded":"2024-09-05T11:56:46.872968+0000","last_fullsized":"2024-09-05T11:56:46.872968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:11.274257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.603536+0000","last_change":"2024-09-05T11:56:19.603044+0000","last_active":"2024-09-05T11:56:19.603536+0000","last_peered":"2024-09-05T11:56:19.603536+0000","last_clean":"2024-09-05T11:56:19.603536+0000","last_became_active":"2024-09-05T11:56:19.602654+0000","last_became_peered":"2024-09-05T11:56:19.602654+0000","last_unstale":"2024-09-05T11:56:19.603536+0000","last_undegraded":"2024-09-05T11:56:19.603536+0000","last_fullsized":"2024-09-05T11:56:19.603536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:29:37.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578218+0000","last_change":"2024-09-05T11:54:38.613641+0000","last_active":"2024-09-05T11:56:19.578218+0000","last_peered":"2024-09-05T11:56:19.578218+0000","last_clean":"2024-09-05T11:56:19.578218+0000","last_became_active":"2024-09-05T11:54:38.612178+0000","last_became_peered":"2024-09-05T11:54:38.612178+0000","last_unstale":"2024-09-05T11:56:19.578218+0000","last_undegraded":"2024-09-05T11:56:19.578218+0000","last_fullsized":"2024-09-05T11:56:19.578218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:19.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877664+0000","last_change":"2024-09-05T11:54:31.532490+0000","last_active":"2024-09-05T11:56:46.877664+0000","last_peered":"2024-09-05T11:56:46.877664+0000","last_clean":"2024-09-05T11:56:46.877664+0000","last_became_active":"2024-09-05T11:54:31.532162+0000","last_became_peered":"2024-09-05T11:54:31.532162+0000","last_unstale":"2024-09-05T11:56:46.877664+0000","last_undegraded":"2024-09-05T11:56:46.877664+0000","last_fullsized":"2024-09-05T11:56:46.877664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:53.309606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575007+0000","last_change":"2024-09-05T11:54:29.105302+0000","last_active":"2024-09-05T11:56:19.575007+0000","last_peered":"2024-09-05T11:56:19.575007+0000","last_clean":"2024-09-05T11:56:19.575007+0000","last_became_active":"2024-09-05T11:54:29.104957+0000","last_became_peered":"2024-09-05T11:54:29.104957+0000","last_unstale":"2024-09-05T11:56:19.575007+0000","last_undegraded":"2024-09-05T11:56:19.575007+0000","last_fullsized":"2024-09-05T11:56:19.575007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:38:47.962219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601591+0000","last_change":"2024-09-05T11:54:38.616807+0000","last_active":"2024-09-05T11:56:19.601591+0000","last_peered":"2024-09-05T11:56:19.601591+0000","last_clean":"2024-09-05T11:56:19.601591+0000","last_became_active":"2024-09-05T11:54:38.616414+0000","last_became_peered":"2024-09-05T11:54:38.616414+0000","last_unstale":"2024-09-05T11:56:19.601591+0000","last_undegraded":"2024-09-05T11:56:19.601591+0000","last_fullsized":"2024-09-05T11:56:19.601591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:55:23.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.864768+0000","last_change":"2024-09-05T11:54:31.520545+0000","last_active":"2024-09-05T11:56:46.864768+0000","last_peered":"2024-09-05T11:56:46.864768+0000","last_clean":"2024-09-05T11:56:46.864768+0000","last_became_active":"2024-09-05T11:54:31.520178+0000","last_became_peered":"2024-09-05T11:54:31.520178+0000","last_unstale":"2024-09-05T11:56:46.864768+0000","last_undegraded":"2024-09-05T11:56:46.864768+0000","last_fullsized":"2024-09-05T11:56:46.864768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:09:39.292448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600986+0000","last_change":"2024-09-05T11:54:29.100691+0000","last_active":"2024-09-05T11:56:19.600986+0000","last_peered":"2024-09-05T11:56:19.600986+0000","last_clean":"2024-09-05T11:56:19.600986+0000","last_became_active":"2024-09-05T11:54:29.099402+0000","last_became_peered":"2024-09-05T11:54:29.099402+0000","last_unstale":"2024-09-05T11:56:19.600986+0000","last_undegraded":"2024-09-05T11:56:19.600986+0000","last_fullsized":"2024-09-05T11:56:19.600986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:00.548322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575257+0000","last_change":"2024-09-05T11:54:38.614246+0000","last_active":"2024-09-05T11:56:19.575257+0000","last_peered":"2024-09-05T11:56:19.575257+0000","last_clean":"2024-09-05T11:56:19.575257+0000","last_became_active":"2024-09-05T11:54:38.613696+0000","last_became_peered":"2024-09-05T11:54:38.613696+0000","last_unstale":"2024-09-05T11:56:19.575257+0000","last_undegraded":"2024-09-05T11:56:19.575257+0000","last_fullsized":"2024-09-05T11:56:19.575257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:52:36.726272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.865202+0000","last_change":"2024-09-05T11:54:31.524723+0000","last_active":"2024-09-05T11:56:46.865202+0000","last_peered":"2024-09-05T11:56:46.865202+0000","last_clean":"2024-09-05T11:56:46.865202+0000","last_became_active":"2024-09-05T11:54:31.524384+0000","last_became_peered":"2024-09-05T11:54:31.524384+0000","last_unstale":"2024-09-05T11:56:46.865202+0000","last_undegraded":"2024-09-05T11:56:46.865202+0000","last_fullsized":"2024-09-05T11:56:46.865202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:54:54.863757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":314,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:57:04.645035+0000","last_change":"2024-09-05T11:54:29.101346+0000","last_active":"2024-09-05T11:57:04.645035+0000","last_peered":"2024-09-05T11:57:04.645035+0000","last_clean":"2024-09-05T11:57:04.645035+0000","last_became_active":"2024-09-05T11:54:29.101031+0000","last_became_peered":"2024-09-05T11:54:29.101031+0000","last_unstale":"2024-09-05T11:57:04.645035+0000","last_undegraded":"2024-09-05T11:57:04.645035+0000","last_fullsized":"2024-09-05T11:57:04.645035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:55:57.166790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71064,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":235,"num_read_kb":187,"num_write":38,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577788+0000","last_change":"2024-09-05T11:54:38.610063+0000","last_active":"2024-09-05T11:56:19.577788+0000","last_peered":"2024-09-05T11:56:19.577788+0000","last_clean":"2024-09-05T11:56:19.577788+0000","last_became_active":"2024-09-05T11:54:38.608084+0000","last_became_peered":"2024-09-05T11:54:38.608084+0000","last_unstale":"2024-09-05T11:56:19.577788+0000","last_undegraded":"2024-09-05T11:56:19.577788+0000","last_fullsized":"2024-09-05T11:56:19.577788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:23:46.381020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876675+0000","last_change":"2024-09-05T11:54:31.523288+0000","last_active":"2024-09-05T11:56:46.876675+0000","last_peered":"2024-09-05T11:56:46.876675+0000","last_clean":"2024-09-05T11:56:46.876675+0000","last_became_active":"2024-09-05T11:54:31.522977+0000","last_became_peered":"2024-09-05T11:54:31.522977+0000","last_unstale":"2024-09-05T11:56:46.876675+0000","last_undegraded":"2024-09-05T11:56:46.876675+0000","last_fullsized":"2024-09-05T11:56:46.876675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:19:46.583777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"82'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575418+0000","last_change":"2024-09-05T11:54:29.543179+0000","last_active":"2024-09-05T11:56:19.575418+0000","last_peered":"2024-09-05T11:56:19.575418+0000","last_clean":"2024-09-05T11:56:19.575418+0000","last_became_active":"2024-09-05T11:54:29.542800+0000","last_became_peered":"2024-09-05T11:54:29.542800+0000","last_unstale":"2024-09-05T11:56:19.575418+0000","last_undegraded":"2024-09-05T11:56:19.575418+0000","last_fullsized":"2024-09-05T11:56:19.575418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:45:53.452760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.578665+0000","last_change":"2024-09-05T11:54:38.605257+0000","last_active":"2024-09-05T11:56:19.578665+0000","last_peered":"2024-09-05T11:56:19.578665+0000","last_clean":"2024-09-05T11:56:19.578665+0000","last_became_active":"2024-09-05T11:54:38.604822+0000","last_became_peered":"2024-09-05T11:54:38.604822+0000","last_unstale":"2024-09-05T11:56:19.578665+0000","last_undegraded":"2024-09-05T11:56:19.578665+0000","last_fullsized":"2024-09-05T11:56:19.578665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:34:16.907405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872639+0000","last_change":"2024-09-05T11:54:31.538911+0000","last_active":"2024-09-05T11:56:46.872639+0000","last_peered":"2024-09-05T11:56:46.872639+0000","last_clean":"2024-09-05T11:56:46.872639+0000","last_became_active":"2024-09-05T11:54:31.538551+0000","last_became_peered":"2024-09-05T11:54:31.538551+0000","last_unstale":"2024-09-05T11:56:46.872639+0000","last_undegraded":"2024-09-05T11:56:46.872639+0000","last_fullsized":"2024-09-05T11:56:46.872639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:37:40.392040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602529+0000","last_change":"2024-09-05T11:56:19.601711+0000","last_active":"2024-09-05T11:56:19.602529+0000","last_peered":"2024-09-05T11:56:19.602529+0000","last_clean":"2024-09-05T11:56:19.602529+0000","last_became_active":"2024-09-05T11:56:19.601334+0000","last_became_peered":"2024-09-05T11:56:19.601334+0000","last_unstale":"2024-09-05T11:56:19.602529+0000","last_undegraded":"2024-09-05T11:56:19.602529+0000","last_fullsized":"2024-09-05T11:56:19.602529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:47.376688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.577694+0000","last_change":"2024-09-05T11:54:38.606705+0000","last_active":"2024-09-05T11:56:19.577694+0000","last_peered":"2024-09-05T11:56:19.577694+0000","last_clean":"2024-09-05T11:56:19.577694+0000","last_became_active":"2024-09-05T11:54:38.606053+0000","last_became_peered":"2024-09-05T11:54:38.606053+0000","last_unstale":"2024-09-05T11:56:19.577694+0000","last_undegraded":"2024-09-05T11:56:19.577694+0000","last_fullsized":"2024-09-05T11:56:19.577694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:07:54.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870328+0000","last_change":"2024-09-05T11:54:31.524392+0000","last_active":"2024-09-05T11:56:46.870328+0000","last_peered":"2024-09-05T11:56:46.870328+0000","last_clean":"2024-09-05T11:56:46.870328+0000","last_became_active":"2024-09-05T11:54:31.524089+0000","last_became_peered":"2024-09-05T11:54:31.524089+0000","last_unstale":"2024-09-05T11:56:46.870328+0000","last_undegraded":"2024-09-05T11:56:46.870328+0000","last_fullsized":"2024-09-05T11:56:46.870328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:16:55.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.574486+0000","last_change":"2024-09-05T11:54:29.104864+0000","last_active":"2024-09-05T11:56:19.574486+0000","last_peered":"2024-09-05T11:56:19.574486+0000","last_clean":"2024-09-05T11:56:19.574486+0000","last_became_active":"2024-09-05T11:54:29.104461+0000","last_became_peered":"2024-09-05T11:54:29.104461+0000","last_unstale":"2024-09-05T11:56:19.574486+0000","last_undegraded":"2024-09-05T11:56:19.574486+0000","last_fullsized":"2024-09-05T11:56:19.574486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:31:04.758110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.579981+0000","last_change":"2024-09-05T11:54:38.612386+0000","last_active":"2024-09-05T11:56:19.579981+0000","last_peered":"2024-09-05T11:56:19.579981+0000","last_clean":"2024-09-05T11:56:19.579981+0000","last_became_active":"2024-09-05T11:54:38.611930+0000","last_became_peered":"2024-09-05T11:54:38.611930+0000","last_unstale":"2024-09-05T11:56:19.579981+0000","last_undegraded":"2024-09-05T11:56:19.579981+0000","last_fullsized":"2024-09-05T11:56:19.579981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:48.741456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600666+0000","last_change":"2024-09-05T11:54:38.603653+0000","last_active":"2024-09-05T11:56:19.600666+0000","last_peered":"2024-09-05T11:56:19.600666+0000","last_clean":"2024-09-05T11:56:19.600666+0000","last_became_active":"2024-09-05T11:54:38.603205+0000","last_became_peered":"2024-09-05T11:54:38.603205+0000","last_unstale":"2024-09-05T11:56:19.600666+0000","last_undegraded":"2024-09-05T11:56:19.600666+0000","last_fullsized":"2024-09-05T11:56:19.600666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:56:02.595671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.877264+0000","last_change":"2024-09-05T11:54:31.524936+0000","last_active":"2024-09-05T11:56:46.877264+0000","last_peered":"2024-09-05T11:56:46.877264+0000","last_clean":"2024-09-05T11:56:46.877264+0000","last_became_active":"2024-09-05T11:54:31.524521+0000","last_became_peered":"2024-09-05T11:54:31.524521+0000","last_unstale":"2024-09-05T11:56:46.877264+0000","last_undegraded":"2024-09-05T11:56:46.877264+0000","last_fullsized":"2024-09-05T11:56:46.877264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:31.969777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.575032+0000","last_change":"2024-09-05T11:54:29.102313+0000","last_active":"2024-09-05T11:56:19.575032+0000","last_peered":"2024-09-05T11:56:19.575032+0000","last_clean":"2024-09-05T11:56:19.575032+0000","last_became_active":"2024-09-05T11:54:29.101838+0000","last_became_peered":"2024-09-05T11:54:29.101838+0000","last_unstale":"2024-09-05T11:56:19.575032+0000","last_undegraded":"2024-09-05T11:56:19.575032+0000","last_fullsized":"2024-09-05T11:56:19.575032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:03.993747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.601411+0000","last_change":"2024-09-05T11:54:38.614646+0000","last_active":"2024-09-05T11:56:19.601411+0000","last_peered":"2024-09-05T11:56:19.601411+0000","last_clean":"2024-09-05T11:56:19.601411+0000","last_became_active":"2024-09-05T11:54:38.614201+0000","last_became_peered":"2024-09-05T11:54:38.614201+0000","last_unstale":"2024-09-05T11:56:19.601411+0000","last_undegraded":"2024-09-05T11:56:19.601411+0000","last_fullsized":"2024-09-05T11:56:19.601411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:50:06.878282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870033+0000","last_change":"2024-09-05T11:54:31.521618+0000","last_active":"2024-09-05T11:56:46.870033+0000","last_peered":"2024-09-05T11:56:46.870033+0000","last_clean":"2024-09-05T11:56:46.870033+0000","last_became_active":"2024-09-05T11:54:31.521290+0000","last_became_peered":"2024-09-05T11:54:31.521290+0000","last_unstale":"2024-09-05T11:56:46.870033+0000","last_undegraded":"2024-09-05T11:56:46.870033+0000","last_fullsized":"2024-09-05T11:56:46.870033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:04:25.053212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.573984+0000","last_change":"2024-09-05T11:54:29.096116+0000","last_active":"2024-09-05T11:56:19.573984+0000","last_peered":"2024-09-05T11:56:19.573984+0000","last_clean":"2024-09-05T11:56:19.573984+0000","last_became_active":"2024-09-05T11:54:29.095761+0000","last_became_peered":"2024-09-05T11:54:29.095761+0000","last_unstale":"2024-09-05T11:56:19.573984+0000","last_undegraded":"2024-09-05T11:56:19.573984+0000","last_fullsized":"2024-09-05T11:56:19.573984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:07:21.283226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.602775+0000","last_change":"2024-09-05T11:56:19.602278+0000","last_active":"2024-09-05T11:56:19.602775+0000","last_peered":"2024-09-05T11:56:19.602775+0000","last_clean":"2024-09-05T11:56:19.602775+0000","last_became_active":"2024-09-05T11:56:19.601847+0000","last_became_peered":"2024-09-05T11:56:19.601847+0000","last_unstale":"2024-09-05T11:56:19.602775+0000","last_undegraded":"2024-09-05T11:56:19.602775+0000","last_fullsized":"2024-09-05T11:56:19.602775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:12:30.600762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868826+0000","last_change":"2024-09-05T11:55:18.960761+0000","last_active":"2024-09-05T11:56:46.868826+0000","last_peered":"2024-09-05T11:56:46.868826+0000","last_clean":"2024-09-05T11:56:46.868826+0000","last_became_active":"2024-09-05T11:55:18.960285+0000","last_became_peered":"2024-09-05T11:55:18.960285+0000","last_unstale":"2024-09-05T11:56:46.868826+0000","last_undegraded":"2024-09-05T11:56:46.868826+0000","last_fullsized":"2024-09-05T11:56:46.868826+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:37:31.154498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600407+0000","last_change":"2024-09-05T11:54:38.611087+0000","last_active":"2024-09-05T11:56:19.600407+0000","last_peered":"2024-09-05T11:56:19.600407+0000","last_clean":"2024-09-05T11:56:19.600407+0000","last_became_active":"2024-09-05T11:54:38.610615+0000","last_became_peered":"2024-09-05T11:54:38.610615+0000","last_unstale":"2024-09-05T11:56:19.600407+0000","last_undegraded":"2024-09-05T11:56:19.600407+0000","last_fullsized":"2024-09-05T11:56:19.600407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:54:05.254190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"82'1","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.581507+0000","last_change":"2024-09-05T11:56:19.580692+0000","last_active":"2024-09-05T11:56:19.581507+0000","last_peered":"2024-09-05T11:56:19.581507+0000","last_clean":"2024-09-05T11:56:19.581507+0000","last_became_active":"2024-09-05T11:56:19.579882+0000","last_became_peered":"2024-09-05T11:56:19.579882+0000","last_unstale":"2024-09-05T11:56:19.581507+0000","last_undegraded":"2024-09-05T11:56:19.581507+0000","last_fullsized":"2024-09-05T11:56:19.581507+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:28.084000+0000","last_clean_scrub_stamp":"2024-09-05T11:54:28.084000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:05:34.221499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.861127+0000","last_change":"2024-09-05T11:55:18.754810+0000","last_active":"2024-09-05T11:56:46.861127+0000","last_peered":"2024-09-05T11:56:46.861127+0000","last_clean":"2024-09-05T11:56:46.861127+0000","last_became_active":"2024-09-05T11:55:18.754481+0000","last_became_peered":"2024-09-05T11:55:18.754481+0000","last_unstale":"2024-09-05T11:56:46.861127+0000","last_undegraded":"2024-09-05T11:56:46.861127+0000","last_fullsized":"2024-09-05T11:56:46.861127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:40:47.151281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:19.600960+0000","last_change":"2024-09-05T11:54:38.606031+0000","last_active":"2024-09-05T11:56:19.600960+0000","last_peered":"2024-09-05T11:56:19.600960+0000","last_clean":"2024-09-05T11:56:19.600960+0000","last_became_active":"2024-09-05T11:54:38.605520+0000","last_became_peered":"2024-09-05T11:54:38.605520+0000","last_unstale":"2024-09-05T11:56:19.600960+0000","last_undegraded":"2024-09-05T11:56:19.600960+0000","last_fullsized":"2024-09-05T11:56:19.600960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:37.575887+0000","last_clean_scrub_stamp":"2024-09-05T11:54:37.575887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:57:24.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868426+0000","last_change":"2024-09-05T11:54:31.527148+0000","last_active":"2024-09-05T11:56:46.868426+0000","last_peered":"2024-09-05T11:56:46.868426+0000","last_clean":"2024-09-05T11:56:46.868426+0000","last_became_active":"2024-09-05T11:54:31.526838+0000","last_became_peered":"2024-09-05T11:54:31.526838+0000","last_unstale":"2024-09-05T11:56:46.868426+0000","last_undegraded":"2024-09-05T11:56:46.868426+0000","last_fullsized":"2024-09-05T11:56:46.868426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:43:53.929328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870762+0000","last_change":"2024-09-05T11:54:31.528087+0000","last_active":"2024-09-05T11:56:46.870762+0000","last_peered":"2024-09-05T11:56:46.870762+0000","last_clean":"2024-09-05T11:56:46.870762+0000","last_became_active":"2024-09-05T11:54:31.527560+0000","last_became_peered":"2024-09-05T11:54:31.527560+0000","last_unstale":"2024-09-05T11:56:46.870762+0000","last_undegraded":"2024-09-05T11:56:46.870762+0000","last_fullsized":"2024-09-05T11:56:46.870762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:16:57.848644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.862687+0000","last_change":"2024-09-05T11:54:31.534890+0000","last_active":"2024-09-05T11:56:46.862687+0000","last_peered":"2024-09-05T11:56:46.862687+0000","last_clean":"2024-09-05T11:56:46.862687+0000","last_became_active":"2024-09-05T11:54:31.534453+0000","last_became_peered":"2024-09-05T11:54:31.534453+0000","last_unstale":"2024-09-05T11:56:46.862687+0000","last_undegraded":"2024-09-05T11:56:46.862687+0000","last_fullsized":"2024-09-05T11:56:46.862687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:42:44.249131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872631+0000","last_change":"2024-09-05T11:54:31.543780+0000","last_active":"2024-09-05T11:56:46.872631+0000","last_peered":"2024-09-05T11:56:46.872631+0000","last_clean":"2024-09-05T11:56:46.872631+0000","last_became_active":"2024-09-05T11:54:31.541660+0000","last_became_peered":"2024-09-05T11:54:31.541660+0000","last_unstale":"2024-09-05T11:56:46.872631+0000","last_undegraded":"2024-09-05T11:56:46.872631+0000","last_fullsized":"2024-09-05T11:56:46.872631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:56:18.088276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870845+0000","last_change":"2024-09-05T11:54:31.539793+0000","last_active":"2024-09-05T11:56:46.870845+0000","last_peered":"2024-09-05T11:56:46.870845+0000","last_clean":"2024-09-05T11:56:46.870845+0000","last_became_active":"2024-09-05T11:54:31.539445+0000","last_became_peered":"2024-09-05T11:54:31.539445+0000","last_unstale":"2024-09-05T11:56:46.870845+0000","last_undegraded":"2024-09-05T11:56:46.870845+0000","last_fullsized":"2024-09-05T11:56:46.870845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:26:43.839963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868744+0000","last_change":"2024-09-05T11:54:31.524899+0000","last_active":"2024-09-05T11:56:46.868744+0000","last_peered":"2024-09-05T11:56:46.868744+0000","last_clean":"2024-09-05T11:56:46.868744+0000","last_became_active":"2024-09-05T11:54:31.524581+0000","last_became_peered":"2024-09-05T11:54:31.524581+0000","last_unstale":"2024-09-05T11:56:46.868744+0000","last_undegraded":"2024-09-05T11:56:46.868744+0000","last_fullsized":"2024-09-05T11:56:46.868744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:14.461633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.859801+0000","last_change":"2024-09-05T11:54:31.530917+0000","last_active":"2024-09-05T11:56:46.859801+0000","last_peered":"2024-09-05T11:56:46.859801+0000","last_clean":"2024-09-05T11:56:46.859801+0000","last_became_active":"2024-09-05T11:54:31.529610+0000","last_became_peered":"2024-09-05T11:54:31.529610+0000","last_unstale":"2024-09-05T11:56:46.859801+0000","last_undegraded":"2024-09-05T11:56:46.859801+0000","last_fullsized":"2024-09-05T11:56:46.859801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:25:54.770355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868917+0000","last_change":"2024-09-05T11:54:31.532733+0000","last_active":"2024-09-05T11:56:46.868917+0000","last_peered":"2024-09-05T11:56:46.868917+0000","last_clean":"2024-09-05T11:56:46.868917+0000","last_became_active":"2024-09-05T11:54:31.525827+0000","last_became_peered":"2024-09-05T11:54:31.525827+0000","last_unstale":"2024-09-05T11:56:46.868917+0000","last_undegraded":"2024-09-05T11:56:46.868917+0000","last_fullsized":"2024-09-05T11:56:46.868917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:44.422744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869714+0000","last_change":"2024-09-05T11:54:31.530313+0000","last_active":"2024-09-05T11:56:46.869714+0000","last_peered":"2024-09-05T11:56:46.869714+0000","last_clean":"2024-09-05T11:56:46.869714+0000","last_became_active":"2024-09-05T11:54:31.529682+0000","last_became_peered":"2024-09-05T11:54:31.529682+0000","last_unstale":"2024-09-05T11:56:46.869714+0000","last_undegraded":"2024-09-05T11:56:46.869714+0000","last_fullsized":"2024-09-05T11:56:46.869714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:27:57.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871156+0000","last_change":"2024-09-05T11:54:31.529983+0000","last_active":"2024-09-05T11:56:46.871156+0000","last_peered":"2024-09-05T11:56:46.871156+0000","last_clean":"2024-09-05T11:56:46.871156+0000","last_became_active":"2024-09-05T11:54:31.529638+0000","last_became_peered":"2024-09-05T11:54:31.529638+0000","last_unstale":"2024-09-05T11:56:46.871156+0000","last_undegraded":"2024-09-05T11:56:46.871156+0000","last_fullsized":"2024-09-05T11:56:46.871156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:30:40.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871551+0000","last_change":"2024-09-05T11:54:31.541081+0000","last_active":"2024-09-05T11:56:46.871551+0000","last_peered":"2024-09-05T11:56:46.871551+0000","last_clean":"2024-09-05T11:56:46.871551+0000","last_became_active":"2024-09-05T11:54:31.540687+0000","last_became_peered":"2024-09-05T11:54:31.540687+0000","last_unstale":"2024-09-05T11:56:46.871551+0000","last_undegraded":"2024-09-05T11:56:46.871551+0000","last_fullsized":"2024-09-05T11:56:46.871551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:55:40.274852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871369+0000","last_change":"2024-09-05T11:54:31.536926+0000","last_active":"2024-09-05T11:56:46.871369+0000","last_peered":"2024-09-05T11:56:46.871369+0000","last_clean":"2024-09-05T11:56:46.871369+0000","last_became_active":"2024-09-05T11:54:31.536621+0000","last_became_peered":"2024-09-05T11:54:31.536621+0000","last_unstale":"2024-09-05T11:56:46.871369+0000","last_undegraded":"2024-09-05T11:56:46.871369+0000","last_fullsized":"2024-09-05T11:56:46.871369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:47:55.920693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868831+0000","last_change":"2024-09-05T11:55:18.754299+0000","last_active":"2024-09-05T11:56:46.868831+0000","last_peered":"2024-09-05T11:56:46.868831+0000","last_clean":"2024-09-05T11:56:46.868831+0000","last_became_active":"2024-09-05T11:55:18.753738+0000","last_became_peered":"2024-09-05T11:55:18.753738+0000","last_unstale":"2024-09-05T11:56:46.868831+0000","last_undegraded":"2024-09-05T11:56:46.868831+0000","last_fullsized":"2024-09-05T11:56:46.868831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:53:39.842415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868535+0000","last_change":"2024-09-05T11:54:31.534571+0000","last_active":"2024-09-05T11:56:46.868535+0000","last_peered":"2024-09-05T11:56:46.868535+0000","last_clean":"2024-09-05T11:56:46.868535+0000","last_became_active":"2024-09-05T11:54:31.534171+0000","last_became_peered":"2024-09-05T11:54:31.534171+0000","last_unstale":"2024-09-05T11:56:46.868535+0000","last_undegraded":"2024-09-05T11:56:46.868535+0000","last_fullsized":"2024-09-05T11:56:46.868535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:50:34.525384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876996+0000","last_change":"2024-09-05T11:54:31.528759+0000","last_active":"2024-09-05T11:56:46.876996+0000","last_peered":"2024-09-05T11:56:46.876996+0000","last_clean":"2024-09-05T11:56:46.876996+0000","last_became_active":"2024-09-05T11:54:31.528277+0000","last_became_peered":"2024-09-05T11:54:31.528277+0000","last_unstale":"2024-09-05T11:56:46.876996+0000","last_undegraded":"2024-09-05T11:56:46.876996+0000","last_fullsized":"2024-09-05T11:56:46.876996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:44.171260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874675+0000","last_change":"2024-09-05T11:54:31.539782+0000","last_active":"2024-09-05T11:56:46.874675+0000","last_peered":"2024-09-05T11:56:46.874675+0000","last_clean":"2024-09-05T11:56:46.874675+0000","last_became_active":"2024-09-05T11:54:31.539395+0000","last_became_peered":"2024-09-05T11:54:31.539395+0000","last_unstale":"2024-09-05T11:56:46.874675+0000","last_undegraded":"2024-09-05T11:56:46.874675+0000","last_fullsized":"2024-09-05T11:56:46.874675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:57:26.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875711+0000","last_change":"2024-09-05T11:54:31.534915+0000","last_active":"2024-09-05T11:56:46.875711+0000","last_peered":"2024-09-05T11:56:46.875711+0000","last_clean":"2024-09-05T11:56:46.875711+0000","last_became_active":"2024-09-05T11:54:31.534586+0000","last_became_peered":"2024-09-05T11:54:31.534586+0000","last_unstale":"2024-09-05T11:56:46.875711+0000","last_undegraded":"2024-09-05T11:56:46.875711+0000","last_fullsized":"2024-09-05T11:56:46.875711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:44.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.872643+0000","last_change":"2024-09-05T11:54:31.531127+0000","last_active":"2024-09-05T11:56:46.872643+0000","last_peered":"2024-09-05T11:56:46.872643+0000","last_clean":"2024-09-05T11:56:46.872643+0000","last_became_active":"2024-09-05T11:54:31.530835+0000","last_became_peered":"2024-09-05T11:54:31.530835+0000","last_unstale":"2024-09-05T11:56:46.872643+0000","last_undegraded":"2024-09-05T11:56:46.872643+0000","last_fullsized":"2024-09-05T11:56:46.872643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:46:22.295355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.866488+0000","last_change":"2024-09-05T11:54:31.543143+0000","last_active":"2024-09-05T11:56:46.866488+0000","last_peered":"2024-09-05T11:56:46.866488+0000","last_clean":"2024-09-05T11:56:46.866488+0000","last_became_active":"2024-09-05T11:54:31.542634+0000","last_became_peered":"2024-09-05T11:54:31.542634+0000","last_unstale":"2024-09-05T11:56:46.866488+0000","last_undegraded":"2024-09-05T11:56:46.866488+0000","last_fullsized":"2024-09-05T11:56:46.866488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:52.674531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.871715+0000","last_change":"2024-09-05T11:54:31.529990+0000","last_active":"2024-09-05T11:56:46.871715+0000","last_peered":"2024-09-05T11:56:46.871715+0000","last_clean":"2024-09-05T11:56:46.871715+0000","last_became_active":"2024-09-05T11:54:31.528674+0000","last_became_peered":"2024-09-05T11:54:31.528674+0000","last_unstale":"2024-09-05T11:56:46.871715+0000","last_undegraded":"2024-09-05T11:56:46.871715+0000","last_fullsized":"2024-09-05T11:56:46.871715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:04:25.767754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.858446+0000","last_change":"2024-09-05T11:54:31.524280+0000","last_active":"2024-09-05T11:56:46.858446+0000","last_peered":"2024-09-05T11:56:46.858446+0000","last_clean":"2024-09-05T11:56:46.858446+0000","last_became_active":"2024-09-05T11:54:31.523962+0000","last_became_peered":"2024-09-05T11:54:31.523962+0000","last_unstale":"2024-09-05T11:56:46.858446+0000","last_undegraded":"2024-09-05T11:56:46.858446+0000","last_fullsized":"2024-09-05T11:56:46.858446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:34:34.653090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874263+0000","last_change":"2024-09-05T11:54:31.528851+0000","last_active":"2024-09-05T11:56:46.874263+0000","last_peered":"2024-09-05T11:56:46.874263+0000","last_clean":"2024-09-05T11:56:46.874263+0000","last_became_active":"2024-09-05T11:54:31.528492+0000","last_became_peered":"2024-09-05T11:54:31.528492+0000","last_unstale":"2024-09-05T11:56:46.874263+0000","last_undegraded":"2024-09-05T11:56:46.874263+0000","last_fullsized":"2024-09-05T11:56:46.874263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:14:47.724739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.870515+0000","last_change":"2024-09-05T11:54:31.523894+0000","last_active":"2024-09-05T11:56:46.870515+0000","last_peered":"2024-09-05T11:56:46.870515+0000","last_clean":"2024-09-05T11:56:46.870515+0000","last_became_active":"2024-09-05T11:54:31.523086+0000","last_became_peered":"2024-09-05T11:54:31.523086+0000","last_unstale":"2024-09-05T11:56:46.870515+0000","last_undegraded":"2024-09-05T11:56:46.870515+0000","last_fullsized":"2024-09-05T11:56:46.870515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:31:48.394477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875858+0000","last_change":"2024-09-05T11:54:31.527664+0000","last_active":"2024-09-05T11:56:46.875858+0000","last_peered":"2024-09-05T11:56:46.875858+0000","last_clean":"2024-09-05T11:56:46.875858+0000","last_became_active":"2024-09-05T11:54:31.526992+0000","last_became_peered":"2024-09-05T11:54:31.526992+0000","last_unstale":"2024-09-05T11:56:46.875858+0000","last_undegraded":"2024-09-05T11:56:46.875858+0000","last_fullsized":"2024-09-05T11:56:46.875858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:59:15.510768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.876450+0000","last_change":"2024-09-05T11:55:18.753244+0000","last_active":"2024-09-05T11:56:46.876450+0000","last_peered":"2024-09-05T11:56:46.876450+0000","last_clean":"2024-09-05T11:56:46.876450+0000","last_became_active":"2024-09-05T11:55:18.752783+0000","last_became_peered":"2024-09-05T11:55:18.752783+0000","last_unstale":"2024-09-05T11:56:46.876450+0000","last_undegraded":"2024-09-05T11:56:46.876450+0000","last_fullsized":"2024-09-05T11:56:46.876450+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:43.264041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.873689+0000","last_change":"2024-09-05T11:55:18.754442+0000","last_active":"2024-09-05T11:56:46.873689+0000","last_peered":"2024-09-05T11:56:46.873689+0000","last_clean":"2024-09-05T11:56:46.873689+0000","last_became_active":"2024-09-05T11:55:18.753999+0000","last_became_peered":"2024-09-05T11:55:18.753999+0000","last_unstale":"2024-09-05T11:56:46.873689+0000","last_undegraded":"2024-09-05T11:56:46.873689+0000","last_fullsized":"2024-09-05T11:56:46.873689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:25:59.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.875619+0000","last_change":"2024-09-05T11:55:18.751541+0000","last_active":"2024-09-05T11:56:46.875619+0000","last_peered":"2024-09-05T11:56:46.875619+0000","last_clean":"2024-09-05T11:56:46.875619+0000","last_became_active":"2024-09-05T11:55:18.751114+0000","last_became_peered":"2024-09-05T11:55:18.751114+0000","last_unstale":"2024-09-05T11:56:46.875619+0000","last_undegraded":"2024-09-05T11:56:46.875619+0000","last_fullsized":"2024-09-05T11:56:46.875619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:32:07.703108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.868370+0000","last_change":"2024-09-05T11:54:31.530297+0000","last_active":"2024-09-05T11:56:46.868370+0000","last_peered":"2024-09-05T11:56:46.868370+0000","last_clean":"2024-09-05T11:56:46.868370+0000","last_became_active":"2024-09-05T11:54:31.529959+0000","last_became_peered":"2024-09-05T11:54:31.529959+0000","last_unstale":"2024-09-05T11:56:46.868370+0000","last_undegraded":"2024-09-05T11:56:46.868370+0000","last_fullsized":"2024-09-05T11:56:46.868370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:37.591398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867358+0000","last_change":"2024-09-05T11:55:18.752031+0000","last_active":"2024-09-05T11:56:46.867358+0000","last_peered":"2024-09-05T11:56:46.867358+0000","last_clean":"2024-09-05T11:56:46.867358+0000","last_became_active":"2024-09-05T11:55:18.751719+0000","last_became_peered":"2024-09-05T11:55:18.751719+0000","last_unstale":"2024-09-05T11:56:46.867358+0000","last_undegraded":"2024-09-05T11:56:46.867358+0000","last_fullsized":"2024-09-05T11:56:46.867358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:47.339616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.867819+0000","last_change":"2024-09-05T11:54:31.537920+0000","last_active":"2024-09-05T11:56:46.867819+0000","last_peered":"2024-09-05T11:56:46.867819+0000","last_clean":"2024-09-05T11:56:46.867819+0000","last_became_active":"2024-09-05T11:54:31.537625+0000","last_became_peered":"2024-09-05T11:54:31.537625+0000","last_unstale":"2024-09-05T11:56:46.867819+0000","last_undegraded":"2024-09-05T11:56:46.867819+0000","last_fullsized":"2024-09-05T11:56:46.867819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:51.143195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.869063+0000","last_change":"2024-09-05T11:54:31.542736+0000","last_active":"2024-09-05T11:56:46.869063+0000","last_peered":"2024-09-05T11:56:46.869063+0000","last_clean":"2024-09-05T11:56:46.869063+0000","last_became_active":"2024-09-05T11:54:31.542446+0000","last_became_peered":"2024-09-05T11:54:31.542446+0000","last_unstale":"2024-09-05T11:56:46.869063+0000","last_undegraded":"2024-09-05T11:56:46.869063+0000","last_fullsized":"2024-09-05T11:56:46.869063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:36:59.424780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-05T11:56:46.874537+0000","last_change":"2024-09-05T11:54:31.539813+0000","last_active":"2024-09-05T11:56:46.874537+0000","last_peered":"2024-09-05T11:56:46.874537+0000","last_clean":"2024-09-05T11:56:46.874537+0000","last_became_active":"2024-09-05T11:54:31.539428+0000","last_became_peered":"2024-09-05T11:54:31.539428+0000","last_unstale":"2024-09-05T11:56:46.874537+0000","last_undegraded":"2024-09-05T11:56:46.874537+0000","last_fullsized":"2024-09-05T11:56:46.874537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T11:54:30.500978+0000","last_clean_scrub_stamp":"2024-09-05T11:54:30.500978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:16:10.632715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71862,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":242,"num_read_kb":194,"num_write":98,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":215586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":532334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37056,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708096,"statfs":{"total":95995035648,"available":95957090304,"internally_reserved":0,"allocated":1753088,"data_stored":1193960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907001,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135934,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":532938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1127380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1122884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953553,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":532356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":528342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":599302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":528328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T11:57:11.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:12.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:11 smithi059 ceph-mon[30025]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:12.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:11 smithi189 ceph-mon[34772]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:11 smithi089 ceph-mon[34499]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 -- 172.21.15.59:0/3942378773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520103670 msgr2=0x7f752010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 --2- 172.21.15.59:0/3942378773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520103670 0x7f752010bb60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a480 comp rx=0 tx=0).stop 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 -- 172.21.15.59:0/3942378773 shutdown_connections 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 --2- 172.21.15.59:0/3942378773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f752010c180 0x7f752010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 --2- 172.21.15.59:0/3942378773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520103670 0x7f752010bb60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 --2- 172.21.15.59:0/3942378773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7520102d50 0x7f7520103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 -- 172.21.15.59:0/3942378773 >> 172.21.15.59:0/3942378773 conn(0x7f75200fc7d0 msgr2=0x7f75200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 -- 172.21.15.59:0/3942378773 shutdown_connections 2024-09-05T11:57:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.451+0000 7f7526200640 1 -- 172.21.15.59:0/3942378773 wait complete. 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.452+0000 7f7526200640 1 Processor -- start 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.452+0000 7f7526200640 1 -- start start 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7520103670 0x7f7520100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752010c180 0x7f7520101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7520111fd0 con 0x7f7520102d50 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7520111e50 con 0x7f7520103670 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7526200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7520112150 con 0x7f752010c180 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f751f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f751f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52908/0 (socket says 172.21.15.59:52908) 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f751f400640 1 -- 172.21.15.59:0/1499561013 learned_addr learned my addr 172.21.15.59:0/1499561013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f751fe00640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752010c180 0x7f7520101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.453+0000 7f7516a00640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7520103670 0x7f7520100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 -- 172.21.15.59:0/1499561013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752010c180 msgr2=0x7f7520101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752010c180 0x7f7520101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 -- 172.21.15.59:0/1499561013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7520103670 msgr2=0x7f7520100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7520103670 0x7f7520100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 -- 172.21.15.59:0/1499561013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75201a69b0 con 0x7f7520102d50 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751f400640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f751000d770 tx=0x7f751000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510017070 con 0x7f7520102d50 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7510005500 con 0x7f7520102d50 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510004e00 con 0x7f7520102d50 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.454+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75201a6ca0 con 0x7f7520102d50 2024-09-05T11:57:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.455+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75201a71b0 con 0x7f7520102d50 2024-09-05T11:57:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.456+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7520103130 con 0x7f7520102d50 2024-09-05T11:57:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.457+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7510004fa0 con 0x7f7520102d50 2024-09-05T11:57:12.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.459+0000 7f751cc00640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 0x7f75040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:12.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.459+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f75100cee10 con 0x7f7520102d50 2024-09-05T11:57:12.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.459+0000 7f7516a00640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 0x7f75040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:12.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.460+0000 7f7516a00640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 0x7f75040a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:12.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.463+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75100d2030 con 0x7f7520102d50 2024-09-05T11:57:12.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.685+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75201071f0 con 0x7f7520102d50 2024-09-05T11:57:12.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.686+0000 7f751cc00640 1 -- 172.21.15.59:0/1499561013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f7510095e20 con 0x7f7520102d50 2024-09-05T11:57:12.688 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:12.688 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:12.688 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:12.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 msgr2=0x7f75040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:12.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 0x7f75040a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7508000c00 tx=0x7f7508005b50 comp rx=0 tx=0).stop 2024-09-05T11:57:12.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 msgr2=0x7f75201009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:12.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f751000d770 tx=0x7f751000dc30 comp rx=0 tx=0).stop 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 shutdown_connections 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f75040a2c00 0x7f75040a50c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752010c180 0x7f7520101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7520103670 0x7f7520100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 --2- 172.21.15.59:0/1499561013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7520102d50 0x7f75201009b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 >> 172.21.15.59:0/1499561013 conn(0x7f75200fc7d0 msgr2=0x7f75200fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 shutdown_connections 2024-09-05T11:57:12.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:12.689+0000 7f7526200640 1 -- 172.21.15.59:0/1499561013 wait complete. 2024-09-05T11:57:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:12 smithi059 ceph-mon[30025]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-05T11:57:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:12 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1499561013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:12 smithi189 ceph-mon[34772]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-05T11:57:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:12 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1499561013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:12 smithi089 ceph-mon[34499]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-05T11:57:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:12 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1499561013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:13.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:13.892 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 -- 172.21.15.59:0/2625665211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 msgr2=0x7f6e5c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 --2- 172.21.15.59:0/2625665211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c108a10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009900 tx=0x7f6e4402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 -- 172.21.15.59:0/2625665211 shutdown_connections 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 --2- 172.21.15.59:0/2625665211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 0x7f6e5c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 --2- 172.21.15.59:0/2625665211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c108a10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 --2- 172.21.15.59:0/2625665211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 -- 172.21.15.59:0/2625665211 >> 172.21.15.59:0/2625665211 conn(0x7f6e5c0ffa50 msgr2=0x7f6e5c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:14.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 -- 172.21.15.59:0/2625665211 shutdown_connections 2024-09-05T11:57:14.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.455+0000 7f6e60e00640 1 -- 172.21.15.59:0/2625665211 wait complete. 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.456+0000 7f6e60e00640 1 Processor -- start 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.456+0000 7f6e60e00640 1 -- start start 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 0x7f6e5c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1141b0 con 0x7f6e5c1049c0 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c114030 con 0x7f6e5c109140 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e60e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c114330 con 0x7f6e5c104010 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e59e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e59e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52204/0 (socket says 172.21.15.59:52204) 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e59e00640 1 -- 172.21.15.59:0/858703705 learned_addr learned my addr 172.21.15.59:0/858703705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e5a800640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 0x7f6e5c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 -- 172.21.15.59:0/858703705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 msgr2=0x7f6e5c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 -- 172.21.15.59:0/858703705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 msgr2=0x7f6e5c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 0x7f6e5c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e53e00640 1 -- 172.21.15.59:0/858703705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c19f6c0 con 0x7f6e5c1049c0 2024-09-05T11:57:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.457+0000 7f6e59e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e53e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c19ac50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e4402a910 tx=0x7f6e44003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4403f070 con 0x7f6e5c1049c0 2024-09-05T11:57:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e44031c90 con 0x7f6e5c1049c0 2024-09-05T11:57:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44035030 con 0x7f6e5c1049c0 2024-09-05T11:57:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5c19f950 con 0x7f6e5c1049c0 2024-09-05T11:57:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.458+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5c1a7290 con 0x7f6e5c1049c0 2024-09-05T11:57:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.459+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5c1054f0 con 0x7f6e5c1049c0 2024-09-05T11:57:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.461+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e44042070 con 0x7f6e5c1049c0 2024-09-05T11:57:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.462+0000 7f6e52a00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 0x7f6e400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.463+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6e440efda0 con 0x7f6e5c1049c0 2024-09-05T11:57:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.463+0000 7f6e59e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 0x7f6e400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.463+0000 7f6e59e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 0x7f6e400a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c011500 tx=0x7f6e4c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.466+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e440bc240 con 0x7f6e5c1049c0 2024-09-05T11:57:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.686+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e5c10c380 con 0x7f6e5c1049c0 2024-09-05T11:57:14.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.687+0000 7f6e52a00640 1 -- 172.21.15.59:0/858703705 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f6e440c0540 con 0x7f6e5c1049c0 2024-09-05T11:57:14.689 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:14.689 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:14.689 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.689+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 msgr2=0x7f6e400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.689+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 0x7f6e400a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c011500 tx=0x7f6e4c005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.689+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 msgr2=0x7f6e5c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.689+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c19ac50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e4402a910 tx=0x7f6e44003b20 comp rx=0 tx=0).stop 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 shutdown_connections 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6e400a2b80 0x7f6e400a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e5c109140 0x7f6e5c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e5c1049c0 0x7f6e5c19ac50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 --2- 172.21.15.59:0/858703705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c104010 0x7f6e5c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:14.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 >> 172.21.15.59:0/858703705 conn(0x7f6e5c0ffa50 msgr2=0x7f6e5c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:14.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 shutdown_connections 2024-09-05T11:57:14.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:14.690+0000 7f6e60e00640 1 -- 172.21.15.59:0/858703705 wait complete. 2024-09-05T11:57:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:14 smithi059 ceph-mon[30025]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:14 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/858703705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:14 smithi189 ceph-mon[34772]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:15.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:14 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/858703705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:14 smithi089 ceph-mon[34499]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:14 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/858703705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:15.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T11:57:15.819 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:16.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.392+0000 7f6feb600640 1 -- 172.21.15.59:0/875790487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4103ff0 msgr2=0x7f6fe41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:16.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.392+0000 7f6feb600640 1 --2- 172.21.15.59:0/875790487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4103ff0 0x7f6fe41043d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009900 tx=0x7f6fd802a410 comp rx=0 tx=0).stop 2024-09-05T11:57:16.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.392+0000 7f6feb600640 1 -- 172.21.15.59:0/875790487 shutdown_connections 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 --2- 172.21.15.59:0/875790487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe4109120 0x7f6fe4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 --2- 172.21.15.59:0/875790487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41049a0 0x7f6fe41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 --2- 172.21.15.59:0/875790487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4103ff0 0x7f6fe41043d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 -- 172.21.15.59:0/875790487 >> 172.21.15.59:0/875790487 conn(0x7f6fe40ffa90 msgr2=0x7f6fe4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 -- 172.21.15.59:0/875790487 shutdown_connections 2024-09-05T11:57:16.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.393+0000 7f6feb600640 1 -- 172.21.15.59:0/875790487 wait complete. 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 Processor -- start 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 -- start start 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 0x7f6fe419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4109120 0x7f6fe419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4114170 con 0x7f6fe4109120 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4113ff0 con 0x7f6fe41049a0 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.394+0000 7f6feb600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41142f0 con 0x7f6fe4103ff0 2024-09-05T11:57:16.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 0x7f6fe419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:16.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:16.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4109120 0x7f6fe419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:16.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 0x7f6fe419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52212/0 (socket says 172.21.15.59:52212) 2024-09-05T11:57:16.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe8c00640 1 -- 172.21.15.59:0/1226960767 learned_addr learned my addr 172.21.15.59:0/1226960767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:16.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46640/0 (socket says 172.21.15.59:46640) 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 -- 172.21.15.59:0/1226960767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 msgr2=0x7f6fe419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 0x7f6fe419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 -- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4109120 msgr2=0x7f6fe419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4109120 0x7f6fe419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 -- 172.21.15.59:0/1226960767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419f5c0 con 0x7f6fe41049a0 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.395+0000 7f6fe3e00640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005fd0 tx=0x7f6fd000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.396+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009c30 con 0x7f6fe41049a0 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.396+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0009dd0 con 0x7f6fe41049a0 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.396+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe419f8b0 con 0x7f6fe41049a0 2024-09-05T11:57:16.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.396+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0010460 con 0x7f6fe41049a0 2024-09-05T11:57:16.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.396+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a7190 con 0x7f6fe41049a0 2024-09-05T11:57:16.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.397+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe41043d0 con 0x7f6fe41049a0 2024-09-05T11:57:16.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.399+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fd0003f50 con 0x7f6fe41049a0 2024-09-05T11:57:16.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.401+0000 7f6fe1600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 0x7f6fcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:16.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.401+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6fd00cefe0 con 0x7f6fe41049a0 2024-09-05T11:57:16.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.401+0000 7f6fe8c00640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 0x7f6fcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:16.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.402+0000 7f6fe8c00640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 0x7f6fcc0a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80096c0 tx=0x7f6fd80046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.404+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fd0017020 con 0x7f6fe41049a0 2024-09-05T11:57:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.590+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fe402d7c0 con 0x7f6fe41049a0 2024-09-05T11:57:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.592+0000 7f6fe1600640 1 -- 172.21.15.59:0/1226960767 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7f6fd009b480 con 0x7f6fe41049a0 2024-09-05T11:57:16.594 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:16.594 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 msgr2=0x7f6fcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 0x7f6fcc0a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80096c0 tx=0x7f6fd80046c0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 msgr2=0x7f6fe419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005fd0 tx=0x7f6fd000dde0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 shutdown_connections 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6fcc0a2b80 0x7f6fcc0a5040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fe4109120 0x7f6fe419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fe41049a0 0x7f6fe419ab50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 --2- 172.21.15.59:0/1226960767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe4103ff0 0x7f6fe419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 >> 172.21.15.59:0/1226960767 conn(0x7f6fe40ffa90 msgr2=0x7f6fe4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 shutdown_connections 2024-09-05T11:57:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:16.595+0000 7f6feb600640 1 -- 172.21.15.59:0/1226960767 wait complete. 2024-09-05T11:57:16.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:16 smithi059 ceph-mon[30025]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:16.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:16 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1226960767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:16 smithi189 ceph-mon[34772]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:16 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1226960767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:16 smithi089 ceph-mon[34499]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:16 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1226960767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T11:57:17.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-05T11:57:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:18.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:18 smithi059 ceph-mon[30025]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:57:19.032 INFO:teuthology.orchestra.run.smithi059.stdout:{"run_dir":"/var/run/ceph"} 2024-09-05T11:57:19.033 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-05T11:57:19.033 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 8220 2024-09-05T11:57:19.033 DEBUG:tasks.quiescer:Yielding 2024-09-05T11:57:19.033 INFO:teuthology.run_tasks:Running task workunit... 2024-09-05T11:57:19.050 INFO:tasks.workunit:Pulling workunits from ref e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:57:19.050 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-05T11:57:19.050 INFO:tasks.workunit:timeout=3h 2024-09-05T11:57:19.050 INFO:tasks.workunit:cleanup=True 2024-09-05T11:57:19.051 DEBUG:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-05T11:57:19.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:19.087 INFO:teuthology.orchestra.run.smithi059.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-05T11:57:19.087 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-05 11:55:23.545325577 +0000 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-05 11:55:23.545325577 +0000 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-05 11:56:54.711008609 +0000 2024-09-05T11:57:19.088 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-05 11:55:23.545325577 +0000 2024-09-05T11:57:19.088 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-05T11:57:19.089 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-05T11:57:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:18 smithi189 ceph-mon[34772]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:57:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:18 smithi089 ceph-mon[34499]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:57:19.192 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T11:57:19.231 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-05T11:57:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 -- 172.21.15.59:0/1215134310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 msgr2=0x7f624c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 --2- 172.21.15.59:0/1215134310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c110ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6244009ca0 tx=0x7f624402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 -- 172.21.15.59:0/1215134310 shutdown_connections 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 --2- 172.21.15.59:0/1215134310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c110ae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 --2- 172.21.15.59:0/1215134310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c1047e0 0x7f624c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 --2- 172.21.15.59:0/1215134310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c103e30 0x7f624c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 -- 172.21.15.59:0/1215134310 >> 172.21.15.59:0/1215134310 conn(0x7f624c0ff8d0 msgr2=0x7f624c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 -- 172.21.15.59:0/1215134310 shutdown_connections 2024-09-05T11:57:19.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.970+0000 7f6252200640 1 -- 172.21.15.59:0/1215134310 wait complete. 2024-09-05T11:57:19.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.971+0000 7f6252200640 1 Processor -- start 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.971+0000 7f6252200640 1 -- start start 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 0x7f624c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f624c113fc0 con 0x7f624c108f60 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f624c113e40 con 0x7f624c103e30 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6252200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f624c114140 con 0x7f624c1047e0 2024-09-05T11:57:19.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6250e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6250e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46656/0 (socket says 172.21.15.59:46656) 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6250e00640 1 -- 172.21.15.59:0/133539269 learned_addr learned my addr 172.21.15.59:0/133539269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6251800640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6243e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 0x7f624c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6243e00640 1 -- 172.21.15.59:0/133539269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 msgr2=0x7f624c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.972+0000 7f6243e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6243e00640 1 -- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 msgr2=0x7f624c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6243e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:19.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6243e00640 1 -- 172.21.15.59:0/133539269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19f3b0 con 0x7f624c1047e0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6250e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6243e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 0x7f624c19a940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f623400e9f0 tx=0x7f623400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623400ccc0 con 0x7f624c1047e0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19f6a0 con 0x7f624c1047e0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6234010910 con 0x7f624c1047e0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234018960 con 0x7f624c1047e0 2024-09-05T11:57:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.973+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c1a6f80 con 0x7f624c1047e0 2024-09-05T11:57:19.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.974+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c104210 con 0x7f624c1047e0 2024-09-05T11:57:19.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.977+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6234010ac0 con 0x7f624c1047e0 2024-09-05T11:57:19.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.978+0000 7f624a000640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 0x7f62300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:19.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.978+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f62340cfb80 con 0x7f624c1047e0 2024-09-05T11:57:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.978+0000 7f6250e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 0x7f62300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.979+0000 7f6250e00640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 0x7f62300a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f623c011500 tx=0x7f623c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:19.980+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f623409c020 con 0x7f624c1047e0 2024-09-05T11:57:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.202+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f624c10c1a0 con 0x7f624c1047e0 2024-09-05T11:57:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.203+0000 7f624a000640 1 -- 172.21.15.59:0/133539269 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f62340a0320 con 0x7f624c1047e0 2024-09-05T11:57:20.205 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:20.205 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 msgr2=0x7f62300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 0x7f62300a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f623c011500 tx=0x7f623c005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 msgr2=0x7f624c19a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 0x7f624c19a940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f623400e9f0 tx=0x7f623400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:57:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 shutdown_connections 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f62300a2b80 0x7f62300a5040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f624c108f60 0x7f624c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c1047e0 0x7f624c19a940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 --2- 172.21.15.59:0/133539269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f624c103e30 0x7f624c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 >> 172.21.15.59:0/133539269 conn(0x7f624c0ff8d0 msgr2=0x7f624c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 shutdown_connections 2024-09-05T11:57:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:20.206+0000 7f6252200640 1 -- 172.21.15.59:0/133539269 wait complete. 2024-09-05T11:57:20.896 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:20 smithi059 ceph-mon[30025]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:20.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:20 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/133539269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:20.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T11:54:32.492433+0000', 'modified': '2024-09-05T11:55:14.725014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6837/2748476304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 2748476304}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 2748476304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4050620246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4050620246}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4050620246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24545]} 2024-09-05T11:57:20.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T11:57:20.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T11:57:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:20 smithi189 ceph-mon[34772]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:20 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/133539269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:20 smithi089 ceph-mon[34499]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:57:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:20 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/133539269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.880+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2277952280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4076720 msgr2=0x7fa1a4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.880+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2277952280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4076720 0x7fa1a4076b00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa194009ca0 tx=0x7fa19402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2277952280 shutdown_connections 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2277952280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2277952280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4077040 0x7fa1a41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2277952280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4076720 0x7fa1a4076b00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2277952280 >> 172.21.15.59:0/2277952280 conn(0x7fa1a40ffa90 msgr2=0x7fa1a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2277952280 shutdown_connections 2024-09-05T11:57:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.881+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2277952280 wait complete. 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 Processor -- start 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 -- start start 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4076720 0x7fa1a41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 0x7fa1a41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41185f0 con 0x7fa1a4077040 2024-09-05T11:57:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4118470 con 0x7fa1a4076720 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4118770 con 0x7fa1a4112a30 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.882+0000 7fa199400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 0x7fa1a41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4076720 0x7fa1a41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa199400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 0x7fa1a41a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45878/0 (socket says 172.21.15.59:45878) 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa199400640 1 -- 172.21.15.59:0/2535842366 learned_addr learned my addr 172.21.15.59:0/2535842366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 -- 172.21.15.59:0/2535842366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4076720 msgr2=0x7fa1a41a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4076720 0x7fa1a41a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 -- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 msgr2=0x7fa1a41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 0x7fa1a41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 -- 172.21.15.59:0/2535842366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419efb0 con 0x7fa1a4112a30 2024-09-05T11:57:21.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa1a2800640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a419e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1a407aa10 tx=0x7fa19c00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.883+0000 7fa19b400640 1 -- 172.21.15.59:0/2535842366 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00ccc0 con 0x7fa1a4112a30 2024-09-05T11:57:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.884+0000 7fa19b400640 1 -- 172.21.15.59:0/2535842366 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c005020 con 0x7fa1a4112a30 2024-09-05T11:57:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.884+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419f2a0 con 0x7fa1a4112a30 2024-09-05T11:57:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.884+0000 7fa19b400640 1 -- 172.21.15.59:0/2535842366 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004a00 con 0x7fa1a4112a30 2024-09-05T11:57:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.884+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41ab590 con 0x7fa1a4112a30 2024-09-05T11:57:21.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa19b400640 1 -- 172.21.15.59:0/2535842366 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa19c0051d0 con 0x7fa1a4112a30 2024-09-05T11:57:21.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 Processor -- start 2024-09-05T11:57:21.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 -- start start 2024-09-05T11:57:21.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 0x55f7933defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f79333a0b0 con 0x55f7933df510 2024-09-05T11:57:21.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7932e1ab0 con 0x55f7933d4330 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.887+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f793336070 con 0x55f7933d4c50 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:54488/0 (socket says 172.21.15.59:54488) 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 -- 172.21.15.59:0/224668293 learned_addr learned my addr 172.21.15.59:0/224668293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 0x55f7933defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 -- 172.21.15.59:0/224668293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 msgr2=0x55f7933defd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 0x55f7933defd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 -- 172.21.15.59:0/224668293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 msgr2=0x55f7933e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 -- 172.21.15.59:0/224668293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7932ee4a0 con 0x55f7933d4330 2024-09-05T11:57:21.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 0x55f7933defd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa19b400640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 0x7fa1880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa19b400640 1 -- 172.21.15.59:0/2535842366 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa19c0cf260 con 0x7fa1a4112a30 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.888+0000 7fa199400640 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009620 tx=0x7fa18c009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917893f8c09591bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.889+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 0x7fa1880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.889+0000 7fa193000640 1 -- 172.21.15.59:0/224668293 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004030 con 0x55f7933d4330 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.889+0000 7fa193000640 1 -- 172.21.15.59:0/224668293 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa18c005500 con 0x55f7933d4330 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.889+0000 7fa193000640 1 -- 172.21.15.59:0/224668293 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004ca0 con 0x55f7933d4330 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/224668293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 msgr2=0x55f7933d4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009620 tx=0x7fa18c009590 comp rx=0 tx=0).stop 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 0x7fa1880a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1a419f9c0 tx=0x7fa194009820 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/224668293 shutdown_connections 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4c50 0x55f7933defd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/224668293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4330 0x55f7933d4710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/224668293 >> 172.21.15.59:0/224668293 conn(0x55f7933d08c0 msgr2=0x55f7933d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/224668293 shutdown_connections 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/224668293 wait complete. 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.890+0000 7fa1ab7dc740 1 -- start start 2024-09-05T11:57:21.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 Processor -- start 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 0x55f7933dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7933e2060 con 0x55f7933d4c50 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7932c2920 con 0x55f7933df510 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f7933e21e0 con 0x55f7933d4330 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1ab7dc740 10 client.0 _ll_register_callbacks cb 0x55f7932e68a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa199400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa199400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45896/0 (socket says 172.21.15.59:45896) 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa199400640 1 -- 172.21.15.59:0/303324596 learned_addr learned my addr 172.21.15.59:0/303324596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1a2800640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 0x55f7933dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.891+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 -- 172.21.15.59:0/303324596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 msgr2=0x55f7933dc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933dc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 -- 172.21.15.59:0/303324596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 msgr2=0x55f7933dcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 -- 172.21.15.59:0/303324596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7930ab2b0 con 0x55f7933df510 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa199400640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1a2800640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 0x55f7933dd070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa19c097ab0 tx=0x7fa19c09be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a55e2222f942aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa187400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c01c070 con 0x55f7933df510 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18c005500 con 0x55f7933df510 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.892+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa19c0b45a0 con 0x55f7933df510 2024-09-05T11:57:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa19c0a42c0 con 0x55f7933df510 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 10 client.25226 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f7933ede80 con 0x55f7933df510 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa18c005500 con 0x55f7933df510 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa19c0a3e30 con 0x55f7933df510 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 10 client.25226 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 10 client.25226 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 20 client.25226 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f7933f04a0 0x55f7933f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 4 client.25226 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.893+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f793347620 con 0x55f7933f04a0 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.894+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f7933f04a0 0x55f7933f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.894+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f7933f04a0 0x55f7933f1910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fb82619a6e4a14 server_cookie=22fe1f7a2793bf0 in_seq=0 out_seq=0 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.894+0000 7fa187400640 10 client.25226 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa187400640 1 -- 172.21.15.59:0/303324596 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f793347620 con 0x55f7933f04a0 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa187400640 10 client.25226 handle_command_reply: tid=0 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 client.25226 shutdown 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 msgr2=0x55f7933dd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 0x55f7933dd070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa19c097ab0 tx=0x7fa19c09be90 comp rx=0 tx=0).stop 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 shutdown_connections 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933df510 0x55f7933dd070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f7933f04a0 0x55f7933f1910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4c50 0x55f7933dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/303324596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa187400640 0 client.25226 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.895+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 >> 172.21.15.59:0/303324596 conn(0x55f7933d08c0 msgr2=0x55f7933d65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.896+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 shutdown_connections 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.896+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/303324596 wait complete. 2024-09-05T11:57:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.896+0000 7fa1ab7dc740 20 client.25226 trim_cache size 0 max 16384 2024-09-05T11:57:21.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.914+0000 7fa1ab7dc740 1 Processor -- start 2024-09-05T11:57:21.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 -- start start 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933df510 0x55f7933d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7933efc50 con 0x55f7933d4330 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7933f0320 con 0x55f7933d4c50 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f7933f09f0 con 0x55f7933df510 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45898/0 (socket says 172.21.15.59:45898) 2024-09-05T11:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1a1e00640 1 -- 172.21.15.59:0/1147196642 learned_addr learned my addr 172.21.15.59:0/1147196642 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa1a2800640 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933df510 0x55f7933d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.915+0000 7fa199400640 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 -- 172.21.15.59:0/1147196642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933df510 msgr2=0x55f7933d9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933df510 0x55f7933d9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 -- 172.21.15.59:0/1147196642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 msgr2=0x55f7933d9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 -- 172.21.15.59:0/1147196642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7933d1400 con 0x55f7933d4330 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa194004af0 tx=0x7fa194041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d11ae24f350b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa192600640 1 -- 172.21.15.59:0/1147196642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194003c30 con 0x55f7933d4330 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa192600640 1 -- 172.21.15.59:0/1147196642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19405a070 con 0x55f7933d4330 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.916+0000 7fa192600640 1 -- 172.21.15.59:0/1147196642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940046a0 con 0x55f7933d4330 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1147196642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 msgr2=0x55f7933d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa194004af0 tx=0x7fa194041000 comp rx=0 tx=0).stop 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1147196642 shutdown_connections 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933df510 0x55f7933d9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1147196642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933d4330 0x55f7933d0210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1147196642 >> 172.21.15.59:0/1147196642 conn(0x55f7933d08c0 msgr2=0x55f7933daec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1147196642 shutdown_connections 2024-09-05T11:57:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1147196642 wait complete. 2024-09-05T11:57:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.917+0000 7fa1ab7dc740 1 -- start start 2024-09-05T11:57:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 Processor -- start 2024-09-05T11:57:21.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7933f1790 con 0x55f7933df510 2024-09-05T11:57:21.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7933f10c0 con 0x55f7933d4c50 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f7933f1e60 con 0x55f7933d4330 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1ab7dc740 10 client.0 _ll_register_callbacks cb 0x7fa19c09afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.918+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45912/0 (socket says 172.21.15.59:45912) 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa1a2800640 1 -- 172.21.15.59:0/1167940907 learned_addr learned my addr 172.21.15.59:0/1167940907 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933d9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 -- 172.21.15.59:0/1167940907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 msgr2=0x55f7933d9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933d9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 -- 172.21.15.59:0/1167940907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 msgr2=0x55f7933da1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 -- 172.21.15.59:0/1167940907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7933f8210 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa1a2800640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa199400640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55f7933dbe80 tx=0x7fa18c004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789922a739017ede server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa187400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.919+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004d10 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19405a070 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa18c004030 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa18c033050 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 10 client.25232 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f79345af00 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19405a070 con 0x55f7933d4c50 2024-09-05T11:57:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa18c0035f0 con 0x55f7933d4c50 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 10 client.25232 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 10 client.25232 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 20 client.25232 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f79345f7a0 0x55f79345e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 4 client.25232 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.920+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "status"}) -- 0x55f793341b30 con 0x55f79345f7a0 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f79345f7a0 0x55f79345e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa1a1e00640 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f79345f7a0 0x55f79345e4b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1344158f68fbba server_cookie=505838185e361998 in_seq=0 out_seq=0 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa187400640 10 client.25232 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa187400640 1 -- 172.21.15.59:0/1167940907 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f793341b30 con 0x55f79345f7a0 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa187400640 10 client.25232 handle_command_reply: tid=0 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.921+0000 7fa1ab7dc740 1 client.25232 shutdown 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 msgr2=0x55f7933d9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55f7933dbe80 tx=0x7fa18c004950 comp rx=0 tx=0).stop 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 shutdown_connections 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f79345f7a0 0x55f79345e4b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f7933df510 0x55f7933da1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7933d4c50 0x55f7933d9c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 --2- 172.21.15.59:0/1167940907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7933d4330 0x55f7933d9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa187400640 0 client.25232 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 >> 172.21.15.59:0/1167940907 conn(0x55f7933d08c0 msgr2=0x55f7933eac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 shutdown_connections 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 1 -- 172.21.15.59:0/1167940907 wait complete. 2024-09-05T11:57:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.922+0000 7fa1ab7dc740 20 client.25232 trim_cache size 0 max 16384 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "53d560d4-6b7c-11ef-bcd6-c7b262605968", 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24545, 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 169.375670267, 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 21, 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 93, 2024-09-05T11:57:21.929 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 89, 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 182.070245981 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 msgr2=0x7fa1880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 0x7fa1880a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1a419f9c0 tx=0x7fa194009820 comp rx=0 tx=0).stop 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 msgr2=0x7fa1a419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a419e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1a407aa10 tx=0x7fa19c00edd0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 shutdown_connections 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa1880a2c00 0x7fa1880a50c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4112a30 0x7fa1a419e880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4077040 0x7fa1a41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 --2- 172.21.15.59:0/2535842366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1a4076720 0x7fa1a41a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 >> 172.21.15.59:0/2535842366 conn(0x7fa1a40ffa90 msgr2=0x7fa1a4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 shutdown_connections 2024-09-05T11:57:21.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:21.923+0000 7fa1a8c00640 1 -- 172.21.15.59:0/2535842366 wait complete. 2024-09-05T11:57:22.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:21 smithi189 ceph-mon[34772]: pgmap v385: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-05T11:57:22.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:21 smithi089 ceph-mon[34499]: pgmap v385: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-05T11:57:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:21 smithi059 ceph-mon[30025]: pgmap v385: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-05T11:57:22.734 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-05T11:57:24.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:23 smithi059 ceph-mon[30025]: pgmap v386: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:24.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:23 smithi189 ceph-mon[34772]: pgmap v386: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:24.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:23 smithi089 ceph-mon[34499]: pgmap v386: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:26.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:25 smithi189 ceph-mon[34772]: pgmap v387: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:26.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:25 smithi089 ceph-mon[34499]: pgmap v387: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:26.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:25 smithi059 ceph-mon[30025]: pgmap v387: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:28.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:27 smithi189 ceph-mon[34772]: pgmap v388: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:28.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:27 smithi089 ceph-mon[34499]: pgmap v388: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:27 smithi059 ceph-mon[30025]: pgmap v388: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:30.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:29 smithi189 ceph-mon[34772]: pgmap v389: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:30.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:29 smithi089 ceph-mon[34499]: pgmap v389: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:30.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:29 smithi059 ceph-mon[30025]: pgmap v389: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:32.237 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:31 smithi089 ceph-mon[34499]: pgmap v390: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:32.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:31 smithi189 ceph-mon[34772]: pgmap v390: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:32.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:31 smithi059 ceph-mon[30025]: pgmap v390: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:34.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:33 smithi189 ceph-mon[34772]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:34.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:33 smithi089 ceph-mon[34499]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:34.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:33 smithi059 ceph-mon[30025]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:36.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:35 smithi189 ceph-mon[34772]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:36.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:35 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:57:36.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:35 smithi089 ceph-mon[34499]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:36.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:35 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:57:36.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:35 smithi059 ceph-mon[30025]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:57:36.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:35 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:57:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:57:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:57:37.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:57:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:57:37.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:57:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:57:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:57:37.736 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.0 2024-09-05T11:57:37.736 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:38.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:38 smithi059 ceph-mon[30025]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:38.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:38 smithi189 ceph-mon[34772]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:38 smithi089 ceph-mon[34499]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:38.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 -- 172.21.15.59:0/1573367898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454102d20 msgr2=0x7f8454103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:38.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 --2- 172.21.15.59:0/1573367898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454102d20 0x7f8454103100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8444009900 tx=0x7f844402a480 comp rx=0 tx=0).stop 2024-09-05T11:57:38.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 -- 172.21.15.59:0/1573367898 shutdown_connections 2024-09-05T11:57:38.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 --2- 172.21.15.59:0/1573367898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 --2- 172.21.15.59:0/1573367898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454103640 0x7f845410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 --2- 172.21.15.59:0/1573367898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454102d20 0x7f8454103100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.829+0000 7f8459000640 1 -- 172.21.15.59:0/1573367898 >> 172.21.15.59:0/1573367898 conn(0x7f84540fc780 msgr2=0x7f84540feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.830+0000 7f8459000640 1 -- 172.21.15.59:0/1573367898 shutdown_connections 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.830+0000 7f8459000640 1 -- 172.21.15.59:0/1573367898 wait complete. 2024-09-05T11:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.831+0000 7f8459000640 1 Processor -- start 2024-09-05T11:57:38.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.831+0000 7f8459000640 1 -- start start 2024-09-05T11:57:38.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:38.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 0x7f845419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8454111fa0 con 0x7f845410c150 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8454111e20 con 0x7f8454102d20 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8459000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8454112120 con 0x7f8454103640 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8451e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8451e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:33356/0 (socket says 172.21.15.59:33356) 2024-09-05T11:57:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8451e00640 1 -- 172.21.15.59:0/3745226211 learned_addr learned my addr 172.21.15.59:0/3745226211 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.832+0000 7f8451400640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 0x7f845419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8452800640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451400640 1 -- 172.21.15.59:0/3745226211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 msgr2=0x7f845419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451400640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451400640 1 -- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 msgr2=0x7f845419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451400640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:38.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451400640 1 -- 172.21.15.59:0/3745226211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845419f640 con 0x7f8454103640 2024-09-05T11:57:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8452800640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.833+0000 7f8451e00640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f8451400640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 0x7f845419abd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f844000e7c0 tx=0x7f844000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440009530 con 0x7f8454103640 2024-09-05T11:57:38.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844000ccc0 con 0x7f8454103640 2024-09-05T11:57:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845419f930 con 0x7f8454103640 2024-09-05T11:57:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440004a00 con 0x7f8454103640 2024-09-05T11:57:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.834+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a7210 con 0x7f8454103640 2024-09-05T11:57:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.836+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8454103100 con 0x7f8454103640 2024-09-05T11:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.838+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8440004ba0 con 0x7f8454103640 2024-09-05T11:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.840+0000 7f844aa00640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 0x7f84380a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.841+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f84400cf3b0 con 0x7f8454103640 2024-09-05T11:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.841+0000 7f8451e00640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 0x7f84380a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.842+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f844009bfb0 con 0x7f8454103640 2024-09-05T11:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:38.842+0000 7f8451e00640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 0x7f84380a5190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f84440096c0 tx=0x7f84440046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:39.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.066+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f845419b9f0 con 0x7f8454103640 2024-09-05T11:57:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.068+0000 7f844aa00640 1 -- 172.21.15.59:0/3745226211 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f844009b8b0 con 0x7f8454103640 2024-09-05T11:57:39.070 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:39.070 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.071+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 msgr2=0x7f84380a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.071+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 0x7f84380a5190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f84440096c0 tx=0x7f84440046c0 comp rx=0 tx=0).stop 2024-09-05T11:57:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.071+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 msgr2=0x7f845419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.071+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 0x7f845419abd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f844000e7c0 tx=0x7f844000ec80 comp rx=0 tx=0).stop 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 shutdown_connections 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f84380a2cd0 0x7f84380a5190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f845410c150 0x7f845419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8454103640 0x7f845419abd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 --2- 172.21.15.59:0/3745226211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8454102d20 0x7f845419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.072+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 >> 172.21.15.59:0/3745226211 conn(0x7f84540fc780 msgr2=0x7f84541008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.073+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 shutdown_connections 2024-09-05T11:57:39.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:39.076+0000 7f8459000640 1 -- 172.21.15.59:0/3745226211 wait complete. 2024-09-05T11:57:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:39 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3745226211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:39.432 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:39 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3745226211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:39.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:39 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3745226211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:39.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:57:39.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0' 2024-09-05T11:57:39.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0 2024-09-05T11:57:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:40.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:40 smithi189 ceph-mon[34772]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:40.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:40 smithi059 ceph-mon[30025]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:40.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:40 smithi089 ceph-mon[34499]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.891+0000 7fcc1ca00640 1 -- 172.21.15.59:0/1051370622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 msgr2=0x7fcc1810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.891+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/1051370622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b9a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009860 tx=0x7fcc0402a3d0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 -- 172.21.15.59:0/1051370622 shutdown_connections 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/1051370622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc1810bfc0 0x7fcc1810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/1051370622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b9a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/1051370622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc18102b90 0x7fcc18102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 -- 172.21.15.59:0/1051370622 >> 172.21.15.59:0/1051370622 conn(0x7fcc180fc610 msgr2=0x7fcc180fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 -- 172.21.15.59:0/1051370622 shutdown_connections 2024-09-05T11:57:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.892+0000 7fcc1ca00640 1 -- 172.21.15.59:0/1051370622 wait complete. 2024-09-05T11:57:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.893+0000 7fcc1ca00640 1 Processor -- start 2024-09-05T11:57:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 -- start start 2024-09-05T11:57:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc18102b90 0x7fcc1810ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 0x7fcc181a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18111e10 con 0x7fcc181034b0 2024-09-05T11:57:40.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18111c90 con 0x7fcc18102b90 2024-09-05T11:57:40.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.894+0000 7fcc1ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18111f90 con 0x7fcc1810bfc0 2024-09-05T11:57:40.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.895+0000 7fcc16800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc18102b90 0x7fcc1810ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.895+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.895+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57914/0 (socket says 172.21.15.59:57914) 2024-09-05T11:57:40.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.895+0000 7fcc15e00640 1 -- 172.21.15.59:0/9079520 learned_addr learned my addr 172.21.15.59:0/9079520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.895+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 0x7fcc181a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.896+0000 7fcc17200640 1 -- 172.21.15.59:0/9079520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc18102b90 msgr2=0x7fcc1810ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.896+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc18102b90 0x7fcc1810ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.896+0000 7fcc17200640 1 -- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 msgr2=0x7fcc1810b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.896+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.896+0000 7fcc17200640 1 -- 172.21.15.59:0/9079520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc181a3940 con 0x7fcc1810bfc0 2024-09-05T11:57:40.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.897+0000 7fcc15e00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:40.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.897+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 0x7fcc181a3240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fd0 tx=0x7fcc1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.898+0000 7fcc0f400640 1 -- 172.21.15.59:0/9079520 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000ce20 con 0x7fcc1810bfc0 2024-09-05T11:57:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.898+0000 7fcc0f400640 1 -- 172.21.15.59:0/9079520 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc10005020 con 0x7fcc1810bfc0 2024-09-05T11:57:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.898+0000 7fcc0f400640 1 -- 172.21.15.59:0/9079520 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10010720 con 0x7fcc1810bfc0 2024-09-05T11:57:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.898+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc181a3c30 con 0x7fcc1810bfc0 2024-09-05T11:57:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.898+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc181ab4f0 con 0x7fcc1810bfc0 2024-09-05T11:57:40.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.903+0000 7fcc1fffe740 1 Processor -- start 2024-09-05T11:57:40.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.904+0000 7fcc0f400640 1 -- 172.21.15.59:0/9079520 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc10005540 con 0x7fcc1810bfc0 2024-09-05T11:57:40.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.905+0000 7fcc1fffe740 1 -- start start 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0bd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cba0163e0 con 0x555cba0afd50 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cb9fbdd80 con 0x555cba0b0670 2024-09-05T11:57:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.906+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cba0123a0 con 0x555cba0baf30 2024-09-05T11:57:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.907+0000 7fcc16800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.907+0000 7fcc16800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42980/0 (socket says 172.21.15.59:42980) 2024-09-05T11:57:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.907+0000 7fcc16800640 1 -- 172.21.15.59:0/382316599 learned_addr learned my addr 172.21.15.59:0/382316599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.907+0000 7fcc15e00640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.907+0000 7fcc15e00640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc0f400640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 0x7fcbfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc15e00640 1 -- 172.21.15.59:0/382316599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 msgr2=0x555cba0bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc15e00640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc15e00640 1 -- 172.21.15.59:0/382316599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 msgr2=0x555cba0ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc15e00640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc15e00640 1 -- 172.21.15.59:0/382316599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb9fdcae0 con 0x555cba0afd50 2024-09-05T11:57:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.908+0000 7fcc0f400640 1 -- 172.21.15.59:0/9079520 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcc100cfa60 con 0x7fcc1810bfc0 2024-09-05T11:57:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.910+0000 7fcc16800640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.910+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 0x7fcbfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.910+0000 7fcc15e00640 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0b0130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc04031000 tx=0x7fcc04005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93168e968f7e9d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.911+0000 7fcc17200640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 0x7fcbfc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005ba0 tx=0x7fcc10005940 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.911+0000 7fcc03400640 1 -- 172.21.15.59:0/382316599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04030580 con 0x555cba0afd50 2024-09-05T11:57:40.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.911+0000 7fcc03400640 1 -- 172.21.15.59:0/382316599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc04030720 con 0x555cba0afd50 2024-09-05T11:57:40.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.911+0000 7fcc03400640 1 -- 172.21.15.59:0/382316599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004920 con 0x555cba0afd50 2024-09-05T11:57:40.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.912+0000 7fcc1fffe740 1 -- 172.21.15.59:0/382316599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 msgr2=0x555cba0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.912+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0b0130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc04031000 tx=0x7fcc04005a90 comp rx=0 tx=0).stop 2024-09-05T11:57:40.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 -- 172.21.15.59:0/382316599 shutdown_connections 2024-09-05T11:57:40.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0b0670 0x555cba0ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/382316599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0b0130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 -- 172.21.15.59:0/382316599 >> 172.21.15.59:0/382316599 conn(0x555cba0ac2e0 msgr2=0x555cba0ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 -- 172.21.15.59:0/382316599 shutdown_connections 2024-09-05T11:57:40.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.913+0000 7fcc1fffe740 1 -- 172.21.15.59:0/382316599 wait complete. 2024-09-05T11:57:40.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.914+0000 7fcc1fffe740 1 -- start start 2024-09-05T11:57:40.916 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.914+0000 7fcc1fffe740 1 Processor -- start 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cba0bdb40 con 0x555cba0b0670 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cb9fd10f0 con 0x555cba0afd50 2024-09-05T11:57:40.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cba0bdcc0 con 0x555cba0baf30 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.915+0000 7fcc1fffe740 10 client.0 _ll_register_callbacks cb 0x555cb9fc29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42986/0 (socket says 172.21.15.59:42986) 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 -- 172.21.15.59:0/3080076507 learned_addr learned my addr 172.21.15.59:0/3080076507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 -- 172.21.15.59:0/3080076507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 msgr2=0x555cba0b89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0b89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.916+0000 7fcc15e00640 1 -- 172.21.15.59:0/3080076507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 msgr2=0x555cba0b8470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcc16800640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcc15e00640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcc15e00640 1 -- 172.21.15.59:0/3080076507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb9d873c0 con 0x555cba0afd50 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcc16800640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcc15e00640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc04005940 tx=0x7fcc040053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b014071925764b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.917+0000 7fcbf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.918+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04053020 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.918+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcc040055e0 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.918+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc04030720 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.919+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc04004d10 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.919+0000 7fcc1fffe740 10 client.25250 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.919+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555cba0bdb40 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.920+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc04030720 con 0x555cba0afd50 2024-09-05T11:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.920+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcc04003a70 con 0x555cba0afd50 2024-09-05T11:57:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.920+0000 7fcc1fffe740 10 client.25250 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.920+0000 7fcc1fffe740 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.921+0000 7fcc1fffe740 20 client.25250 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.921+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.921+0000 7fcc1fffe740 4 client.25250 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.921+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cba023950 con 0x555cba0caa90 2024-09-05T11:57:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.922+0000 7fcc16800640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.922+0000 7fcc16800640 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba0cce80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386abb4a6cb20034 server_cookie=8ec801b41eb1f808 in_seq=0 out_seq=0 2024-09-05T11:57:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.922+0000 7fcbf7e00640 10 client.25250 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.923+0000 7fcbf7e00640 1 -- 172.21.15.59:0/3080076507 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cba023950 con 0x555cba0caa90 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.923+0000 7fcbf7e00640 10 client.25250 handle_command_reply: tid=0 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.923+0000 7fcc1fffe740 1 client.25250 shutdown 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 msgr2=0x555cba0b7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc04005940 tx=0x7fcc040053a0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 shutdown_connections 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0baf30 0x555cba0b89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba0cce80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcbf7e00640 0 client.25250 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/3080076507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0afd50 0x555cba0b7f30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 >> 172.21.15.59:0/3080076507 conn(0x555cba0ac2e0 msgr2=0x555cba0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.924+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 shutdown_connections 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.925+0000 7fcc1fffe740 1 -- 172.21.15.59:0/3080076507 wait complete. 2024-09-05T11:57:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.925+0000 7fcc1fffe740 20 client.25250 trim_cache size 0 max 16384 2024-09-05T11:57:40.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.962+0000 7fcc1fffe740 1 Processor -- start 2024-09-05T11:57:40.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 -- start start 2024-09-05T11:57:40.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0afd50 0x555cba0b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b33f0 con 0x555cba0b0670 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b5c70 con 0x555cba0baf30 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.963+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b3a60 con 0x555cba0afd50 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57928/0 (socket says 172.21.15.59:57928) 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 -- 172.21.15.59:0/264156987 learned_addr learned my addr 172.21.15.59:0/264156987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 -- 172.21.15.59:0/264156987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0afd50 msgr2=0x555cba0b9e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0afd50 0x555cba0b9e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 -- 172.21.15.59:0/264156987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 msgr2=0x555cba0b5190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0b5190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.964+0000 7fcc16800640 1 -- 172.21.15.59:0/264156987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cba0b04a0 con 0x555cba0b0670 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.965+0000 7fcc16800640 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca10873cde39264 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.966+0000 7fcc02a00640 1 -- 172.21.15.59:0/264156987 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004020 con 0x555cba0b0670 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.966+0000 7fcc02a00640 1 -- 172.21.15.59:0/264156987 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc080041c0 con 0x555cba0b0670 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 -- 172.21.15.59:0/264156987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 msgr2=0x555cba0b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc02a00640 1 -- 172.21.15.59:0/264156987 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004020 con 0x555cba0b0670 2024-09-05T11:57:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08005a70 comp rx=0 tx=0).stop 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 -- 172.21.15.59:0/264156987 shutdown_connections 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0b0670 0x555cba0b27b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/264156987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0afd50 0x555cba0b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 -- 172.21.15.59:0/264156987 >> 172.21.15.59:0/264156987 conn(0x555cba0ac2e0 msgr2=0x555cba0c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 -- 172.21.15.59:0/264156987 shutdown_connections 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.967+0000 7fcc1fffe740 1 -- 172.21.15.59:0/264156987 wait complete. 2024-09-05T11:57:40.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.968+0000 7fcc1fffe740 1 -- start start 2024-09-05T11:57:40.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.968+0000 7fcc1fffe740 1 Processor -- start 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 0x555cba0ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b4130 con 0x555cba0afd50 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b55a0 con 0x555cba0baf30 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b4800 con 0x555cba0b0670 2024-09-05T11:57:40.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.969+0000 7fcc1fffe740 10 client.0 _ll_register_callbacks cb 0x7fcc040096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.970+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.970+0000 7fcc15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57944/0 (socket says 172.21.15.59:57944) 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.970+0000 7fcc15e00640 1 -- 172.21.15.59:0/1109387656 learned_addr learned my addr 172.21.15.59:0/1109387656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.971+0000 7fcc15e00640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.971+0000 7fcc15e00640 1 -- 172.21.15.59:0/1109387656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 msgr2=0x555cba0ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:40.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.971+0000 7fcc16800640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 0x555cba0ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.972+0000 7fcc15e00640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 0x555cba0ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.972+0000 7fcc15e00640 1 -- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 msgr2=0x555cba0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.972+0000 7fcc15e00640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.972+0000 7fcc15e00640 1 -- 172.21.15.59:0/1109387656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cba1345d0 con 0x555cba0afd50 2024-09-05T11:57:40.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.972+0000 7fcc16800640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 0x555cba0ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:40.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.973+0000 7fcc15e00640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcc0403d5b0 tx=0x7fcc04030a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0eeefe4aa126c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:40.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.973+0000 7fcbf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:57:40.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.974+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04053030 con 0x555cba0afd50 2024-09-05T11:57:40.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.974+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcc04003fe0 con 0x555cba0afd50 2024-09-05T11:57:40.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.975+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc080041c0 con 0x555cba0afd50 2024-09-05T11:57:40.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.975+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc040041e0 con 0x555cba0afd50 2024-09-05T11:57:40.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.975+0000 7fcc1fffe740 10 client.15249 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:40.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.975+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555cba1358a0 con 0x555cba0afd50 2024-09-05T11:57:40.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.976+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc080041c0 con 0x555cba0afd50 2024-09-05T11:57:40.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.976+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcc040604e0 con 0x555cba0afd50 2024-09-05T11:57:40.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.977+0000 7fcc1fffe740 10 client.15249 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:40.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.977+0000 7fcc1fffe740 10 client.15249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:40.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.977+0000 7fcc1fffe740 20 client.15249 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:40.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.977+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba137d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:40.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.978+0000 7fcc1fffe740 4 client.15249 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:57:40.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.978+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cba01de60 con 0x555cba0caa90 2024-09-05T11:57:40.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.978+0000 7fcc16800640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:40.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.979+0000 7fcc16800640 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba137d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d8a51cc11deb7f server_cookie=58fc41d9ce9ef8e4 in_seq=0 out_seq=0 2024-09-05T11:57:40.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.979+0000 7fcbf7e00640 10 client.15249 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:40.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.979+0000 7fcbf7e00640 1 -- 172.21.15.59:0/1109387656 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555cba01de60 con 0x555cba0caa90 2024-09-05T11:57:40.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.979+0000 7fcbf7e00640 10 client.15249 handle_command_reply: tid=0 2024-09-05T11:57:40.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.980+0000 7fcc1fffe740 1 client.15249 shutdown 2024-09-05T11:57:40.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.980+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 msgr2=0x555cba0cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.980+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcc0403d5b0 tx=0x7fcc04030a10 comp rx=0 tx=0).stop 2024-09-05T11:57:40.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 shutdown_connections 2024-09-05T11:57:40.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cba0baf30 0x555cba0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cba0b0670 0x555cba0ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555cba0caa90 0x555cba137d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 --2- 172.21.15.59:0/1109387656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cba0afd50 0x555cba0cdec0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 >> 172.21.15.59:0/1109387656 conn(0x555cba0ac2e0 msgr2=0x555cba0b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.981+0000 7fcbf7e00640 0 client.15249 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.982+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 shutdown_connections 2024-09-05T11:57:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.983+0000 7fcc1fffe740 1 -- 172.21.15.59:0/1109387656 wait complete. 2024-09-05T11:57:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.983+0000 7fcc1fffe740 20 client.15249 trim_cache size 0 max 16384 2024-09-05T11:57:40.986 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:57:40.986 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:57:40.986 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.984+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 msgr2=0x7fcbfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.984+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 0x7fcbfc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005ba0 tx=0x7fcc10005940 comp rx=0 tx=0).stop 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.984+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 msgr2=0x7fcc181a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.984+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 0x7fcc181a3240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fd0 tx=0x7fcc1000ef10 comp rx=0 tx=0).stop 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.985+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 shutdown_connections 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.985+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fcbfc0a2c00 0x7fcbfc0a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.985+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1810bfc0 0x7fcc181a3240 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.986+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc181034b0 0x7fcc1810b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.986+0000 7fcc1ca00640 1 --2- 172.21.15.59:0/9079520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc18102b90 0x7fcc1810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.986+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 >> 172.21.15.59:0/9079520 conn(0x7fcc180fc610 msgr2=0x7fcc1810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.986+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 shutdown_connections 2024-09-05T11:57:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:40.986+0000 7fcc1ca00640 1 -- 172.21.15.59:0/9079520 wait complete. 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 -- 172.21.15.59:0/3329446392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 msgr2=0x7f2b4c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 --2- 172.21.15.59:0/3329446392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c076b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009900 tx=0x7f2b3c02a410 comp rx=0 tx=0).stop 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 -- 172.21.15.59:0/3329446392 shutdown_connections 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 --2- 172.21.15.59:0/3329446392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 --2- 172.21.15.59:0/3329446392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 --2- 172.21.15.59:0/3329446392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c076b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.849+0000 7f2b51a00640 1 -- 172.21.15.59:0/3329446392 >> 172.21.15.59:0/3329446392 conn(0x7f2b4c0ffa90 msgr2=0x7f2b4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.850+0000 7f2b51a00640 1 -- 172.21.15.59:0/3329446392 shutdown_connections 2024-09-05T11:57:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.850+0000 7f2b51a00640 1 -- 172.21.15.59:0/3329446392 wait complete. 2024-09-05T11:57:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.851+0000 7f2b51a00640 1 Processor -- start 2024-09-05T11:57:41.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.851+0000 7f2b51a00640 1 -- start start 2024-09-05T11:57:41.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c118610 con 0x7f2b4c112a30 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c118490 con 0x7f2b4c077040 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b51a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c118790 con 0x7f2b4c076720 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.852+0000 7f2b4a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43020/0 (socket says 172.21.15.59:43020) 2024-09-05T11:57:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4a000640 1 -- 172.21.15.59:0/959049788 learned_addr learned my addr 172.21.15.59:0/959049788 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4b400640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4a000640 1 -- 172.21.15.59:0/959049788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 msgr2=0x7f2b4c1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4a000640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.853+0000 7f2b4a000640 1 -- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 msgr2=0x7f2b4c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.854+0000 7f2b4a000640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.854+0000 7f2b4a000640 1 -- 172.21.15.59:0/959049788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c19efd0 con 0x7f2b4c077040 2024-09-05T11:57:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.854+0000 7f2b4b400640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.854+0000 7f2b4a000640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800e9f0 tx=0x7f2b3800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.854+0000 7f2b43400640 1 -- 172.21.15.59:0/959049788 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800cd80 con 0x7f2b4c077040 2024-09-05T11:57:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.855+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c19f2c0 con 0x7f2b4c077040 2024-09-05T11:57:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.855+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c1ab5b0 con 0x7f2b4c077040 2024-09-05T11:57:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.855+0000 7f2b43400640 1 -- 172.21.15.59:0/959049788 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b38005500 con 0x7f2b4c077040 2024-09-05T11:57:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.855+0000 7f2b43400640 1 -- 172.21.15.59:0/959049788 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38010690 con 0x7f2b4c077040 2024-09-05T11:57:41.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.859+0000 7f2b43400640 1 -- 172.21.15.59:0/959049788 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b380046c0 con 0x7f2b4c077040 2024-09-05T11:57:41.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.860+0000 7f2b544c5740 1 Processor -- start 2024-09-05T11:57:41.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.860+0000 7f2b544c5740 1 -- start start 2024-09-05T11:57:41.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.860+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.860+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10297050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10299980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b544c5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae101f2530 con 0x55ae1028ccd0 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b544c5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ae10199860 con 0x55ae1028c3b0 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b544c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae101ee4f0 con 0x55ae10297590 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43030/0 (socket says 172.21.15.59:43030) 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 -- 172.21.15.59:0/1324493083 learned_addr learned my addr 172.21.15.59:0/1324493083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 -- 172.21.15.59:0/1324493083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 msgr2=0x55ae10299980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10299980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.861+0000 7f2b4b400640 1 -- 172.21.15.59:0/1324493083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 msgr2=0x55ae10297050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:41.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10297050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b4b400640 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10297050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b4b400640 1 -- 172.21.15.59:0/1324493083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae0ff63b90 con 0x55ae1028c3b0 2024-09-05T11:57:41.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10297050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:41.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b43400640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 0x7f2b300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.862+0000 7f2b43400640 1 -- 172.21.15.59:0/959049788 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2b38010430 con 0x7f2b4c077040 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.863+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 0x7f2b300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.863+0000 7f2b4b400640 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009ea0 tx=0x7f2b44009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f692719439d024 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.863+0000 7f2b37e00640 1 -- 172.21.15.59:0/1324493083 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44016070 con 0x55ae1028c3b0 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.864+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 0x7f2b300a5240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005c20 tx=0x7f2b3c005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.864+0000 7f2b37e00640 1 -- 172.21.15.59:0/1324493083 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4401bb40 con 0x55ae1028c3b0 2024-09-05T11:57:41.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.864+0000 7f2b37e00640 1 -- 172.21.15.59:0/1324493083 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44012d10 con 0x55ae1028c3b0 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 -- 172.21.15.59:0/1324493083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 msgr2=0x55ae1028c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009ea0 tx=0x7f2b44009f90 comp rx=0 tx=0).stop 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 -- 172.21.15.59:0/1324493083 shutdown_connections 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10299980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10297050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 --2- 172.21.15.59:0/1324493083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae1028c790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.865+0000 7f2b544c5740 1 -- 172.21.15.59:0/1324493083 >> 172.21.15.59:0/1324493083 conn(0x55ae10288940 msgr2=0x55ae10288d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.866+0000 7f2b544c5740 1 -- 172.21.15.59:0/1324493083 shutdown_connections 2024-09-05T11:57:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.866+0000 7f2b544c5740 1 -- 172.21.15.59:0/1324493083 wait complete. 2024-09-05T11:57:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.866+0000 7f2b544c5740 1 -- start start 2024-09-05T11:57:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 Processor -- start 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10294ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10295010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae1029a0d0 con 0x55ae1028ccd0 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ae101ace60 con 0x55ae1028c3b0 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae1029a250 con 0x55ae10297590 2024-09-05T11:57:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.867+0000 7f2b544c5740 10 client.0 _ll_register_callbacks cb 0x55ae101b1cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43034/0 (socket says 172.21.15.59:43034) 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 -- 172.21.15.59:0/315322286 learned_addr learned my addr 172.21.15.59:0/315322286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4a000640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10294ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 -- 172.21.15.59:0/315322286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 msgr2=0x55ae10295010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10295010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.868+0000 7f2b4aa00640 1 -- 172.21.15.59:0/315322286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 msgr2=0x55ae10294ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.869+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10294ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.869+0000 7f2b4aa00640 1 -- 172.21.15.59:0/315322286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae101b8990 con 0x55ae1028c3b0 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.869+0000 7f2b4a000640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10294ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.869+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c033000 tx=0x7f2b3c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442fa08d14eddc4c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.869+0000 7f2b34c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c041070 con 0x55ae1028c3b0 2024-09-05T11:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2b3c0046c0 con 0x55ae1028c3b0 2024-09-05T11:57:41.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4401bb40 con 0x55ae1028c3b0 2024-09-05T11:57:41.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c035050 con 0x55ae1028c3b0 2024-09-05T11:57:41.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b544c5740 10 client.25265 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:41.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.870+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae102a5eb0 con 0x55ae1028c3b0 2024-09-05T11:57:41.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.871+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4401bb40 con 0x55ae1028c3b0 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.871+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2b3c0035f0 con 0x55ae1028c3b0 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.871+0000 7f2b544c5740 10 client.25265 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.871+0000 7f2b544c5740 10 client.25265 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.872+0000 7f2b544c5740 20 client.25265 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.872+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae102a7370 0x55ae102a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.872+0000 7f2b544c5740 4 client.25265 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.872+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae10195310 con 0x55ae102a7370 2024-09-05T11:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.872+0000 7f2b4a000640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae102a7370 0x55ae102a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.873+0000 7f2b4a000640 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae102a7370 0x55ae102a9830 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0c18cf0c31d9ba server_cookie=19485b175587b6f in_seq=0 out_seq=0 2024-09-05T11:57:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.873+0000 7f2b34c00640 10 client.25265 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.874+0000 7f2b34c00640 1 -- 172.21.15.59:0/315322286 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae10195310 con 0x55ae102a7370 2024-09-05T11:57:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.874+0000 7f2b34c00640 10 client.25265 handle_command_reply: tid=0 2024-09-05T11:57:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.874+0000 7f2b544c5740 1 client.25265 shutdown 2024-09-05T11:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.874+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 msgr2=0x55ae10294590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.874+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c033000 tx=0x7f2b3c02a410 comp rx=0 tx=0).stop 2024-09-05T11:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 shutdown_connections 2024-09-05T11:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae10297590 0x55ae10295010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae102a7370 0x55ae102a9830 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae10294ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 --2- 172.21.15.59:0/315322286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae1028c3b0 0x55ae10294590 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 >> 172.21.15.59:0/315322286 conn(0x55ae10288940 msgr2=0x55ae10287c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b34c00640 0 client.25265 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 shutdown_connections 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.875+0000 7f2b544c5740 1 -- 172.21.15.59:0/315322286 wait complete. 2024-09-05T11:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.876+0000 7f2b544c5740 20 client.25265 trim_cache size 0 max 16384 2024-09-05T11:57:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.912+0000 7f2b544c5740 1 Processor -- start 2024-09-05T11:57:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.912+0000 7f2b544c5740 1 -- start start 2024-09-05T11:57:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.912+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.912+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae1028e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.912+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae102af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b544c5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c00dea0 con 0x55ae1028ccd0 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b544c5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c010760 con 0x55ae10297590 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b544c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c00e550 con 0x55ae1028c3b0 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44238/0 (socket says 172.21.15.59:44238) 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b4aa00640 1 -- 172.21.15.59:0/2750359280 learned_addr learned my addr 172.21.15.59:0/2750359280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b4b400640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae102af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.913+0000 7f2b4a000640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae1028e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4aa00640 1 -- 172.21.15.59:0/2750359280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 msgr2=0x55ae102af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:41.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae102af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4aa00640 1 -- 172.21.15.59:0/2750359280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 msgr2=0x55ae1028e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae1028e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4aa00640 1 -- 172.21.15.59:0/2750359280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae10293540 con 0x55ae1028c3b0 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.914+0000 7f2b4a000640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae1028e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.915+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c031550 tx=0x7f2b3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37479e3da0d9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.915+0000 7f2b37400640 1 -- 172.21.15.59:0/2750359280 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c035050 con 0x55ae1028c3b0 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.915+0000 7f2b37400640 1 -- 172.21.15.59:0/2750359280 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3c060070 con 0x55ae1028c3b0 2024-09-05T11:57:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.915+0000 7f2b37400640 1 -- 172.21.15.59:0/2750359280 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0325c0 con 0x55ae1028c3b0 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.916+0000 7f2b544c5740 1 -- 172.21.15.59:0/2750359280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 msgr2=0x55ae10292e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.916+0000 7f2b544c5740 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c031550 tx=0x7f2b3c009780 comp rx=0 tx=0).stop 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 -- 172.21.15.59:0/2750359280 shutdown_connections 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae102af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028ccd0 0x55ae1028e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 --2- 172.21.15.59:0/2750359280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028c3b0 0x55ae10292e60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 -- 172.21.15.59:0/2750359280 >> 172.21.15.59:0/2750359280 conn(0x55ae10288940 msgr2=0x55ae1028de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 -- 172.21.15.59:0/2750359280 shutdown_connections 2024-09-05T11:57:41.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.917+0000 7f2b544c5740 1 -- 172.21.15.59:0/2750359280 wait complete. 2024-09-05T11:57:41.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.918+0000 7f2b544c5740 1 -- start start 2024-09-05T11:57:41.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.918+0000 7f2b544c5740 1 Processor -- start 2024-09-05T11:57:41.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028ccd0 0x55ae1029c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae1029cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c00ec20 con 0x55ae1028c3b0 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c010090 con 0x55ae10297590 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c00f2f0 con 0x55ae1028ccd0 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b544c5740 10 client.0 _ll_register_callbacks cb 0x7f2b38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b4a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028ccd0 0x55ae1029c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.919+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58006/0 (socket says 172.21.15.59:58006) 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.920+0000 7f2b4aa00640 1 -- 172.21.15.59:0/3398220602 learned_addr learned my addr 172.21.15.59:0/3398220602 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.920+0000 7f2b4b400640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae1029cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 -- 172.21.15.59:0/3398220602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028ccd0 msgr2=0x55ae1029c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028ccd0 0x55ae1029c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 -- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 msgr2=0x55ae1029cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:41.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae1029cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 -- 172.21.15.59:0/3398220602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae1029d3c0 con 0x55ae1028c3b0 2024-09-05T11:57:41.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.921+0000 7f2b4aa00640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005ab0 tx=0x7f2b3c004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1420908b7b61811f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:41.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.922+0000 7f2b34c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:57:41.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c068020 con 0x55ae1028c3b0 2024-09-05T11:57:41.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2b3c0048c0 con 0x55ae1028c3b0 2024-09-05T11:57:41.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3c060070 con 0x55ae1028c3b0 2024-09-05T11:57:41.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c035040 con 0x55ae1028c3b0 2024-09-05T11:57:41.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b544c5740 10 client.15264 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:41.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.923+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae103131c0 con 0x55ae1028c3b0 2024-09-05T11:57:41.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.924+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3c060070 con 0x55ae1028c3b0 2024-09-05T11:57:41.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2b3c048580 con 0x55ae1028c3b0 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 10 client.15264 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 10 client.15264 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 20 client.15264 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae10313710 0x55ae10315b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 4 client.15264 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}] 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.925+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}) -- 0x55ae101ffc60 con 0x55ae10313710 2024-09-05T11:57:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.926+0000 7f2b4a000640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae10313710 0x55ae10315b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.926+0000 7f2b34c00640 10 client.15264 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.926+0000 7f2b4a000640 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae10313710 0x55ae10315b30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=189a5f209b4d7bc8 server_cookie=3d03ca9713c4d82e in_seq=0 out_seq=0 2024-09-05T11:57:41.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.936+0000 7f2b34c00640 1 -- 172.21.15.59:0/3398220602 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ae101ffc60 con 0x55ae10313710 2024-09-05T11:57:41.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.936+0000 7f2b34c00640 10 client.15264 handle_command_reply: tid=0 2024-09-05T11:57:41.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.936+0000 7f2b544c5740 1 client.15264 shutdown 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 msgr2=0x55ae1029fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005ab0 tx=0x7f2b3c004f70 comp rx=0 tx=0).stop 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 shutdown_connections 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae10297590 0x55ae1029cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae1028ccd0 0x55ae1029c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae10313710 0x55ae10315b30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 --2- 172.21.15.59:0/3398220602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae1028c3b0 0x55ae1029fe00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 >> 172.21.15.59:0/3398220602 conn(0x55ae10288940 msgr2=0x55ae10294530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b34c00640 0 client.15264 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:41.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.937+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 shutdown_connections 2024-09-05T11:57:41.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.938+0000 7f2b544c5740 1 -- 172.21.15.59:0/3398220602 wait complete. 2024-09-05T11:57:41.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.938+0000 7f2b544c5740 20 client.15264 trim_cache size 0 max 16384 2024-09-05T11:57:41.940 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:41.940 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 2, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "526b5f58": { 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 138.0, 2024-09-05T11:57:41.941 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:57:41.942 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 msgr2=0x7f2b300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 0x7f2b300a5240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005c20 tx=0x7f2b3c005b50 comp rx=0 tx=0).stop 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 msgr2=0x7f2b4c1a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800e9f0 tx=0x7f2b3800edf0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 shutdown_connections 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2b300a2d80 0x7f2b300a5240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b4c112a30 0x7f2b4c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b4c077040 0x7f2b4c1a4c20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 --2- 172.21.15.59:0/959049788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b4c076720 0x7f2b4c1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 >> 172.21.15.59:0/959049788 conn(0x7f2b4c0ffa90 msgr2=0x7f2b4c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.939+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 shutdown_connections 2024-09-05T11:57:41.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:41.940+0000 7f2b51a00640 1 -- 172.21.15.59:0/959049788 wait complete. 2024-09-05T11:57:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:42 smithi059 ceph-mon[30025]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:42 smithi189 ceph-mon[34772]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:42.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:57:42.355 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:42 smithi089 ceph-mon[34499]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T11:57:43.213 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '526b5f58', quiesce took 5.5 seconds. Will release after: 12.5 2024-09-05T11:57:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 -- 172.21.15.59:0/2347105857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f010e430 msgr2=0x7fe9f0110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/2347105857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f010e430 0x7fe9f0110820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009ca0 tx=0x7fe9e802a950 comp rx=0 tx=0).stop 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 -- 172.21.15.59:0/2347105857 shutdown_connections 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/2347105857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f010e430 0x7fe9f0110820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/2347105857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/2347105857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f00745c0 0x7fe9f00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 -- 172.21.15.59:0/2347105857 >> 172.21.15.59:0/2347105857 conn(0x7fe9f00fd8b0 msgr2=0x7fe9f00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 -- 172.21.15.59:0/2347105857 shutdown_connections 2024-09-05T11:57:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.927+0000 7fe9f5e00640 1 -- 172.21.15.59:0/2347105857 wait complete. 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.928+0000 7fe9f5e00640 1 Processor -- start 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.928+0000 7fe9f5e00640 1 -- start start 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f00745c0 0x7fe9f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f010e430 0x7fe9f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f01141d0 con 0x7fe9f00745c0 2024-09-05T11:57:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0114050 con 0x7fe9f010e430 2024-09-05T11:57:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9f5e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0114350 con 0x7fe9f0074ee0 2024-09-05T11:57:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9ee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9ee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44262/0 (socket says 172.21.15.59:44262) 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9ee000640 1 -- 172.21.15.59:0/3850704352 learned_addr learned my addr 172.21.15.59:0/3850704352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.929+0000 7fe9ef400640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f010e430 0x7fe9f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9eea00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f00745c0 0x7fe9f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 -- 172.21.15.59:0/3850704352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f010e430 msgr2=0x7fe9f019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f010e430 0x7fe9f019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 -- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f00745c0 msgr2=0x7fe9f019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f00745c0 0x7fe9f019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 -- 172.21.15.59:0/3850704352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f019f5e0 con 0x7fe9f0074ee0 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.930+0000 7fe9ee000640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b340 tx=0x7fe9dc00b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.931+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc005020 con 0x7fe9f0074ee0 2024-09-05T11:57:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.931+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc0056a0 con 0x7fe9f0074ee0 2024-09-05T11:57:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.931+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00a810 con 0x7fe9f0074ee0 2024-09-05T11:57:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.931+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f019f8d0 con 0x7fe9f0074ee0 2024-09-05T11:57:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.932+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f0101a80 con 0x7fe9f0074ee0 2024-09-05T11:57:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.933+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0075c70 con 0x7fe9f0074ee0 2024-09-05T11:57:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.935+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9dc024020 con 0x7fe9f0074ee0 2024-09-05T11:57:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.937+0000 7fe9f4a00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 0x7fe9d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.937+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe9dc0d7bc0 con 0x7fe9f0074ee0 2024-09-05T11:57:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.938+0000 7fe9eea00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 0x7fe9d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.939+0000 7fe9eea00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 0x7fe9d40a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009aa0 tx=0x7fe9e0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:43.939+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9dc02b030 con 0x7fe9f0074ee0 2024-09-05T11:57:44.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:44 smithi059 ceph-mon[30025]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:44.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.167+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9f00749a0 con 0x7fe9f0074ee0 2024-09-05T11:57:44.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:44.170 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:44.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.168+0000 7fe9f4a00640 1 -- 172.21.15.59:0/3850704352 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fe9f00749a0 con 0x7fe9f0074ee0 2024-09-05T11:57:44.170 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 msgr2=0x7fe9d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 0x7fe9d40a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009aa0 tx=0x7fe9e0005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 msgr2=0x7fe9f019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b340 tx=0x7fe9dc00b800 comp rx=0 tx=0).stop 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 shutdown_connections 2024-09-05T11:57:44.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9d40a2c00 0x7fe9d40a50c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f010e430 0x7fe9f019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f0074ee0 0x7fe9f019ab70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 --2- 172.21.15.59:0/3850704352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f00745c0 0x7fe9f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 >> 172.21.15.59:0/3850704352 conn(0x7fe9f00fd8b0 msgr2=0x7fe9f00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.171+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 shutdown_connections 2024-09-05T11:57:44.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:44.172+0000 7fe9f5e00640 1 -- 172.21.15.59:0/3850704352 wait complete. 2024-09-05T11:57:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:44 smithi089 ceph-mon[34499]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:44.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:44 smithi189 ceph-mon[34772]: pgmap v396: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:44.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:57:45.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3850704352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:45.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3850704352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:45.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3850704352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:45.612 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 -- 172.21.15.59:0/2007923582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581019c0 msgr2=0x7fee58101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 --2- 172.21.15.59:0/2007923582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581019c0 0x7fee58101e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee44009860 tx=0x7fee4402a340 comp rx=0 tx=0).stop 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 -- 172.21.15.59:0/2007923582 shutdown_connections 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 --2- 172.21.15.59:0/2007923582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee5810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 --2- 172.21.15.59:0/2007923582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581019c0 0x7fee58101e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 --2- 172.21.15.59:0/2007923582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581079c0 0x7fee58107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.227+0000 7fee5e200640 1 -- 172.21.15.59:0/2007923582 >> 172.21.15.59:0/2007923582 conn(0x7fee580fd740 msgr2=0x7fee580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.228+0000 7fee5e200640 1 -- 172.21.15.59:0/2007923582 shutdown_connections 2024-09-05T11:57:46.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.228+0000 7fee5e200640 1 -- 172.21.15.59:0/2007923582 wait complete. 2024-09-05T11:57:46.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.229+0000 7fee5e200640 1 Processor -- start 2024-09-05T11:57:46.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 -- start start 2024-09-05T11:57:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581019c0 0x7fee581982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 0x7fee5819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111dc0 con 0x7fee58102360 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111c40 con 0x7fee581079c0 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.230+0000 7fee5e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111f40 con 0x7fee581019c0 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58040/0 (socket says 172.21.15.59:58040) 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee57e00640 1 -- 172.21.15.59:0/1259463281 learned_addr learned my addr 172.21.15.59:0/1259463281 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581019c0 0x7fee581982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 0x7fee5819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 -- 172.21.15.59:0/1259463281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581019c0 msgr2=0x7fee581982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581019c0 0x7fee581982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 -- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 msgr2=0x7fee58198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee5d800640 1 -- 172.21.15.59:0/1259463281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5819d290 con 0x7fee581079c0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.231+0000 7fee57e00640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 0x7fee5819cbb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fee5000efc0 tx=0x7fee5000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee55600640 1 -- 172.21.15.59:0/1259463281 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50003970 con 0x7fee581079c0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee55600640 1 -- 172.21.15.59:0/1259463281 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee50004020 con 0x7fee581079c0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee55600640 1 -- 172.21.15.59:0/1259463281 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000b9f0 con 0x7fee581079c0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5819d580 con 0x7fee581079c0 2024-09-05T11:57:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.232+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee58074420 con 0x7fee581079c0 2024-09-05T11:57:46.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.237+0000 7fee62285740 1 Processor -- start 2024-09-05T11:57:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.237+0000 7fee55600640 1 -- 172.21.15.59:0/1259463281 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee5001d020 con 0x7fee581079c0 2024-09-05T11:57:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.237+0000 7fee62285740 1 -- start start 2024-09-05T11:57:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba94410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee6b9ed3e0 con 0x55ee6ba92020 2024-09-05T11:57:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ee6b994d80 con 0x55ee6ba86e40 2024-09-05T11:57:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee62285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee6b9e93a0 con 0x55ee6ba87760 2024-09-05T11:57:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.238+0000 7fee5ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee5ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44288/0 (socket says 172.21.15.59:44288) 2024-09-05T11:57:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee5ce00640 1 -- 172.21.15.59:0/301957699 learned_addr learned my addr 172.21.15.59:0/301957699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee57e00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba94410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee57e00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee55600640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 0x7fee400a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.239+0000 7fee55600640 1 -- 172.21.15.59:0/1259463281 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee500cf2e0 con 0x7fee581079c0 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 -- 172.21.15.59:0/301957699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 msgr2=0x55ee6ba87220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba87220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 -- 172.21.15.59:0/301957699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 msgr2=0x55ee6ba94410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba94410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 -- 172.21.15.59:0/301957699 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6b9b3ae0 con 0x55ee6ba87760 2024-09-05T11:57:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee57e00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba94410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 0x7fee400a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.240+0000 7fee5ce00640 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c70 tx=0x7fee4c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f7cebb106e3f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.241+0000 7fee4a000640 1 -- 172.21.15.59:0/301957699 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c004230 con 0x55ee6ba87760 2024-09-05T11:57:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.241+0000 7fee4a000640 1 -- 172.21.15.59:0/301957699 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c0043d0 con 0x55ee6ba87760 2024-09-05T11:57:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.241+0000 7fee4a000640 1 -- 172.21.15.59:0/301957699 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c019760 con 0x55ee6ba87760 2024-09-05T11:57:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 -- 172.21.15.59:0/301957699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 msgr2=0x55ee6ba91ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c70 tx=0x7fee4c011810 comp rx=0 tx=0).stop 2024-09-05T11:57:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee5d800640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 0x7fee400a5140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee500a3e10 tx=0x7fee500cf530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 -- 172.21.15.59:0/301957699 shutdown_connections 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba94410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba91ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 --2- 172.21.15.59:0/301957699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba87220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.242+0000 7fee62285740 1 -- 172.21.15.59:0/301957699 >> 172.21.15.59:0/301957699 conn(0x55ee6ba833d0 msgr2=0x55ee6ba837b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.243+0000 7fee62285740 1 -- 172.21.15.59:0/301957699 shutdown_connections 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.243+0000 7fee62285740 1 -- 172.21.15.59:0/301957699 wait complete. 2024-09-05T11:57:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.243+0000 7fee62285740 1 -- start start 2024-09-05T11:57:46.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.244+0000 7fee62285740 1 Processor -- start 2024-09-05T11:57:46.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.244+0000 7fee62285740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.244+0000 7fee62285740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 0x55ee6ba8f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee62285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6ba8f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee62285740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee6ba94bf0 con 0x55ee6ba87760 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee62285740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ee6b9a80f0 con 0x55ee6ba86e40 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee62285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee6ba94d70 con 0x55ee6ba92020 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee62285740 10 client.0 _ll_register_callbacks cb 0x55ee6b98fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee57e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee57e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43118/0 (socket says 172.21.15.59:43118) 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee57e00640 1 -- 172.21.15.59:0/3477413138 learned_addr learned my addr 172.21.15.59:0/3477413138 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:46.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.245+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 0x55ee6ba8f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee57e00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6ba8f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee5ce00640 1 -- 172.21.15.59:0/3477413138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 msgr2=0x55ee6ba8f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6ba8f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee5ce00640 1 -- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 msgr2=0x55ee6ba8ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.246+0000 7fee5ce00640 1 -- 172.21.15.59:0/3477413138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6b75e3c0 con 0x55ee6ba87760 2024-09-05T11:57:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.247+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 0x55ee6ba8f4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fee4c004650 tx=0x7fee4c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34da55eb0114d071 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.247+0000 7fee3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:57:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.247+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c027070 con 0x55ee6ba87760 2024-09-05T11:57:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.248+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee4c0035f0 con 0x55ee6ba87760 2024-09-05T11:57:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.248+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c0043d0 con 0x55ee6ba87760 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.248+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee4c033050 con 0x55ee6ba87760 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.248+0000 7fee62285740 10 client.15279 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.248+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee6baa0c80 con 0x55ee6ba87760 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c0043d0 con 0x55ee6ba87760 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fee4c018070 con 0x55ee6ba87760 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 10 client.15279 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 10 client.15279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 20 client.15279 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6baa1fd0 0x55ee6baa43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 4 client.15279 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.249+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6b9fa950 con 0x55ee6baa1fd0 2024-09-05T11:57:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.250+0000 7fee57e00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6baa1fd0 0x55ee6baa43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.250+0000 7fee57e00640 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6baa1fd0 0x55ee6baa43c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e587e37ae2b9561 server_cookie=5b721b7c530e0be3 in_seq=0 out_seq=0 2024-09-05T11:57:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.250+0000 7fee3ea00640 10 client.15279 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.251+0000 7fee3ea00640 1 -- 172.21.15.59:0/3477413138 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee6b9fa950 con 0x55ee6baa1fd0 2024-09-05T11:57:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.251+0000 7fee3ea00640 10 client.15279 handle_command_reply: tid=0 2024-09-05T11:57:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.251+0000 7fee62285740 1 client.15279 shutdown 2024-09-05T11:57:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.251+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 msgr2=0x55ee6ba8f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.251+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 0x55ee6ba8f4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fee4c004650 tx=0x7fee4c005d30 comp rx=0 tx=0).stop 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 shutdown_connections 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6ba8f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6baa1fd0 0x55ee6baa43c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba87760 0x55ee6ba8f4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 --2- 172.21.15.59:0/3477413138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee3ea00640 0 client.15279 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 >> 172.21.15.59:0/3477413138 conn(0x55ee6ba833d0 msgr2=0x55ee6ba890b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 shutdown_connections 2024-09-05T11:57:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.252+0000 7fee62285740 1 -- 172.21.15.59:0/3477413138 wait complete. 2024-09-05T11:57:46.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.253+0000 7fee62285740 20 client.15279 trim_cache size 0 max 16384 2024-09-05T11:57:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 Processor -- start 2024-09-05T11:57:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 -- start start 2024-09-05T11:57:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba86e40 0x55ee6ba898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6baa9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee581af080 con 0x55ee6ba86e40 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b1900 con 0x55ee6ba87760 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.280+0000 7fee62285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee581af6f0 con 0x55ee6ba92020 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43128/0 (socket says 172.21.15.59:43128) 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 -- 172.21.15.59:0/2645856631 learned_addr learned my addr 172.21.15.59:0/2645856631 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee57e00640 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba86e40 0x55ee6ba898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 -- 172.21.15.59:0/2645856631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 msgr2=0x55ee6baa9b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6baa9b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 -- 172.21.15.59:0/2645856631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba86e40 msgr2=0x55ee6ba898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba86e40 0x55ee6ba898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.281+0000 7fee5ce00640 1 -- 172.21.15.59:0/2645856631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6ba8d4d0 con 0x55ee6ba87760 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.282+0000 7fee5ce00640 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009820 tx=0x7fee4c011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d8afb77c911e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.282+0000 7fee49600640 1 -- 172.21.15.59:0/2645856631 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c015070 con 0x55ee6ba87760 2024-09-05T11:57:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.282+0000 7fee49600640 1 -- 172.21.15.59:0/2645856631 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c01ad70 con 0x55ee6ba87760 2024-09-05T11:57:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.282+0000 7fee49600640 1 -- 172.21.15.59:0/2645856631 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c027070 con 0x55ee6ba87760 2024-09-05T11:57:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.283+0000 7fee62285740 1 -- 172.21.15.59:0/2645856631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 msgr2=0x55ee6ba8d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.283+0000 7fee62285740 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009820 tx=0x7fee4c011e40 comp rx=0 tx=0).stop 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 -- 172.21.15.59:0/2645856631 shutdown_connections 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba92020 0x55ee6baa9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba87760 0x55ee6ba8d850 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 --2- 172.21.15.59:0/2645856631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba86e40 0x55ee6ba898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 -- 172.21.15.59:0/2645856631 >> 172.21.15.59:0/2645856631 conn(0x55ee6ba833d0 msgr2=0x55ee6ba90fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 -- 172.21.15.59:0/2645856631 shutdown_connections 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 -- 172.21.15.59:0/2645856631 wait complete. 2024-09-05T11:57:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.284+0000 7fee62285740 1 -- start start 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 Processor -- start 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba8c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee581afdc0 con 0x55ee6ba92020 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b1230 con 0x55ee6ba86e40 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b0490 con 0x55ee6ba87760 2024-09-05T11:57:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.285+0000 7fee62285740 10 client.0 _ll_register_callbacks cb 0x7fee50009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.286+0000 7fee57e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.286+0000 7fee57e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43144/0 (socket says 172.21.15.59:43144) 2024-09-05T11:57:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.286+0000 7fee57e00640 1 -- 172.21.15.59:0/3896695642 learned_addr learned my addr 172.21.15.59:0/3896695642 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.286+0000 7fee57e00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba8c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.286+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 -- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 msgr2=0x55ee6ba8bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 -- 172.21.15.59:0/3896695642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 msgr2=0x55ee6ba8c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba8c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 -- 172.21.15.59:0/3896695642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6bb14310 con 0x55ee6ba87760 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee57e00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba8c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee5ce00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba8c400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55ee6ba84330 tx=0x7fee4c011eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca65b8100e9ac4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c015040 con 0x55ee6ba87760 2024-09-05T11:57:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee4c01a890 con 0x55ee6ba87760 2024-09-05T11:57:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.287+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4c01ad70 con 0x55ee6ba87760 2024-09-05T11:57:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee4c043050 con 0x55ee6ba87760 2024-09-05T11:57:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 10 client.25237 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee6bb15530 con 0x55ee6ba87760 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c01ad70 con 0x55ee6ba87760 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fee4c018030 con 0x55ee6ba87760 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 10 client.25237 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 20 client.25237 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6bb15b90 0x55ee6bb17fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.288+0000 7fee62285740 4 client.25237 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.289+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee6b9f4e60 con 0x55ee6bb15b90 2024-09-05T11:57:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.289+0000 7fee57e00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6bb15b90 0x55ee6bb17fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:46.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee57e00640 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6bb15b90 0x55ee6bb17fb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f444fa79e87501 server_cookie=8ab899750fe594a8 in_seq=0 out_seq=0 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee3ea00640 10 client.25237 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee3ea00640 1 -- 172.21.15.59:0/3896695642 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee6b9f4e60 con 0x55ee6bb15b90 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee3ea00640 10 client.25237 handle_command_reply: tid=0 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 client.25237 shutdown 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 msgr2=0x55ee6ba8c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba8c400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55ee6ba84330 tx=0x7fee4c011eb0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 shutdown_connections 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ee6bb15b90 0x55ee6bb17fb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.290+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee6ba92020 0x55ee6ba8c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee6ba87760 0x55ee6ba8c400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 1 --2- 172.21.15.59:0/3896695642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ee6ba86e40 0x55ee6ba8bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 >> 172.21.15.59:0/3896695642 conn(0x55ee6ba833d0 msgr2=0x55ee6ba8eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee3ea00640 0 client.25237 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 shutdown_connections 2024-09-05T11:57:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 1 -- 172.21.15.59:0/3896695642 wait complete. 2024-09-05T11:57:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.291+0000 7fee62285740 20 client.25237 trim_cache size 0 max 16384 2024-09-05T11:57:46.294 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 msgr2=0x7fee400a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 0x7fee400a5140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee500a3e10 tx=0x7fee500cf530 comp rx=0 tx=0).stop 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 msgr2=0x7fee5819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 0x7fee5819cbb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fee5000efc0 tx=0x7fee5000c800 comp rx=0 tx=0).stop 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 shutdown_connections 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fee400a2c80 0x7fee400a5140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee581079c0 0x7fee5819cbb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee58102360 0x7fee58198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.292+0000 7fee5e200640 1 --2- 172.21.15.59:0/1259463281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee581019c0 0x7fee581982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.293+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 >> 172.21.15.59:0/1259463281 conn(0x7fee580fd740 msgr2=0x7fee580fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.293+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 shutdown_connections 2024-09-05T11:57:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:46.293+0000 7fee5e200640 1 -- 172.21.15.59:0/1259463281 wait complete. 2024-09-05T11:57:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:46 smithi089 ceph-mon[34499]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:46.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:46 smithi059 ceph-mon[30025]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:46.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:46 smithi189 ceph-mon[34772]: pgmap v397: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T11:57:48.129 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:57:48.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:57:48.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:48 smithi089 ceph-mon[34499]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:48 smithi059 ceph-mon[30025]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:48.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:48 smithi189 ceph-mon[34772]: pgmap v398: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:48.697 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:49.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 -- 172.21.15.59:0/1480472553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 msgr2=0x7fd68c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 --2- 172.21.15.59:0/1480472553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c110ae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd684009ca0 tx=0x7fd68402a8c0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 -- 172.21.15.59:0/1480472553 shutdown_connections 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 --2- 172.21.15.59:0/1480472553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c110ae0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 --2- 172.21.15.59:0/1480472553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c1047e0 0x7fd68c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 --2- 172.21.15.59:0/1480472553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c103e30 0x7fd68c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.340+0000 7fd68b200640 1 -- 172.21.15.59:0/1480472553 >> 172.21.15.59:0/1480472553 conn(0x7fd68c0ff910 msgr2=0x7fd68c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.341+0000 7fd68b200640 1 -- 172.21.15.59:0/1480472553 shutdown_connections 2024-09-05T11:57:49.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.341+0000 7fd68b200640 1 -- 172.21.15.59:0/1480472553 wait complete. 2024-09-05T11:57:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.342+0000 7fd68b200640 1 Processor -- start 2024-09-05T11:57:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.342+0000 7fd68b200640 1 -- start start 2024-09-05T11:57:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c103e30 0x7fd68c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c114010 con 0x7fd68c108f60 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c113e90 con 0x7fd68c1047e0 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd68b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c114190 con 0x7fd68c103e30 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd689400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd689400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43172/0 (socket says 172.21.15.59:43172) 2024-09-05T11:57:49.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.343+0000 7fd689400640 1 -- 172.21.15.59:0/308491269 learned_addr learned my addr 172.21.15.59:0/308491269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd68a800640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689e00640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c103e30 0x7fd68c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689400640 1 -- 172.21.15.59:0/308491269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c103e30 msgr2=0x7fd68c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689400640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c103e30 0x7fd68c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689400640 1 -- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 msgr2=0x7fd68c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689400640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd689400640 1 -- 172.21.15.59:0/308491269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c19f3c0 con 0x7fd68c1047e0 2024-09-05T11:57:49.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.344+0000 7fd68a800640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:49.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.345+0000 7fd689400640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.345+0000 7fd682a00640 1 -- 172.21.15.59:0/308491269 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800cd80 con 0x7fd68c1047e0 2024-09-05T11:57:49.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.346+0000 7fd682a00640 1 -- 172.21.15.59:0/308491269 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678005500 con 0x7fd68c1047e0 2024-09-05T11:57:49.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.346+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c19f6b0 con 0x7fd68c1047e0 2024-09-05T11:57:49.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.346+0000 7fd682a00640 1 -- 172.21.15.59:0/308491269 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678010690 con 0x7fd68c1047e0 2024-09-05T11:57:49.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.346+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a6f90 con 0x7fd68c1047e0 2024-09-05T11:57:49.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.350+0000 7fd693bd6740 1 Processor -- start 2024-09-05T11:57:49.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 -- start start 2024-09-05T11:57:49.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0ecb0 0x557540c0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c19e90 0x557540c1c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557540b75290 con 0x557540c0ecb0 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557540b1cb40 con 0x557540c0f5d0 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.351+0000 7fd693bd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557540b71250 con 0x557540c19e90 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd68a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0ecb0 0x557540c0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43176/0 (socket says 172.21.15.59:43176) 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 -- 172.21.15.59:0/2622490931 learned_addr learned my addr 172.21.15.59:0/2622490931 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 -- 172.21.15.59:0/2622490931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c19e90 msgr2=0x557540c1c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c19e90 0x557540c1c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 -- 172.21.15.59:0/2622490931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0ecb0 msgr2=0x557540c0f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0ecb0 0x557540c0f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd689e00640 1 -- 172.21.15.59:0/2622490931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575408e66a0 con 0x557540c0f5d0 2024-09-05T11:57:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.352+0000 7fd682a00640 1 -- 172.21.15.59:0/308491269 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6780046c0 con 0x7fd68c1047e0 2024-09-05T11:57:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.353+0000 7fd689e00640 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009cf0 tx=0x7fd67c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31e616572a22ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.353+0000 7fd677400640 1 -- 172.21.15.59:0/2622490931 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004230 con 0x557540c0f5d0 2024-09-05T11:57:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.353+0000 7fd677400640 1 -- 172.21.15.59:0/2622490931 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c0043d0 con 0x557540c0f5d0 2024-09-05T11:57:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.353+0000 7fd677400640 1 -- 172.21.15.59:0/2622490931 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c019890 con 0x557540c0f5d0 2024-09-05T11:57:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 -- 172.21.15.59:0/2622490931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 msgr2=0x557540c19950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009cf0 tx=0x7fd67c011810 comp rx=0 tx=0).stop 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 -- 172.21.15.59:0/2622490931 shutdown_connections 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c19e90 0x557540c1c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c19950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2622490931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0ecb0 0x557540c0f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.354+0000 7fd693bd6740 1 -- 172.21.15.59:0/2622490931 >> 172.21.15.59:0/2622490931 conn(0x557540c0b240 msgr2=0x557540c0b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.355+0000 7fd693bd6740 1 -- 172.21.15.59:0/2622490931 shutdown_connections 2024-09-05T11:57:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.355+0000 7fd693bd6740 1 -- 172.21.15.59:0/2622490931 wait complete. 2024-09-05T11:57:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.355+0000 7fd693bd6740 1 -- start start 2024-09-05T11:57:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.356+0000 7fd693bd6740 1 Processor -- start 2024-09-05T11:57:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.356+0000 7fd682a00640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 0x7fd6700a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c19730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557540c1cb00 con 0x557540c19e90 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557540b2ffa0 con 0x557540c0f5d0 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557540c1cc80 con 0x557540c0ecb0 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd68a800640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 0x7fd6700a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd693bd6740 10 client.0 _ll_register_callbacks cb 0x557540883ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd682a00640 1 -- 172.21.15.59:0/308491269 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd6780d2170 con 0x7fd68c1047e0 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd689e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c19730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd689e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c19730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44336/0 (socket says 172.21.15.59:44336) 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd689e00640 1 -- 172.21.15.59:0/2921859833 learned_addr learned my addr 172.21.15.59:0/2921859833 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.357+0000 7fd689400640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd68a800640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd689400640 1 -- 172.21.15.59:0/2921859833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 msgr2=0x557540c19730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd689400640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c19730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd689400640 1 -- 172.21.15.59:0/2921859833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 msgr2=0x557540c16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd689400640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd689400640 1 -- 172.21.15.59:0/2921859833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557540c17720 con 0x557540c0f5d0 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.358+0000 7fd68a800640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.359+0000 7fd689400640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c169e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd68c19f880 tx=0x7fd67809ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef04dc6494880e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.359+0000 7fd66be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.359+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780d3060 con 0x557540c0f5d0 2024-09-05T11:57:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.359+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd67809a7d0 con 0x557540c0f5d0 2024-09-05T11:57:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.359+0000 7fd68a800640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 0x7fd6700a4f50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd684006010 tx=0x7fd684005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.360+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c0043d0 con 0x557540c0f5d0 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.360+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd67809f730 con 0x557540c0f5d0 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.360+0000 7fd693bd6740 10 client.25304 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.360+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557540c31220 con 0x557540c0f5d0 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.360+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c0043d0 con 0x557540c0f5d0 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd6780ccad0 con 0x557540c0f5d0 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 10 client.25304 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 10 client.25304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 20 client.25304 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:49.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c32730 0x557540c34b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 4 client.25304 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557540b82800 con 0x557540c32730 2024-09-05T11:57:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.361+0000 7fd689e00640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c32730 0x557540c34b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.362+0000 7fd689e00640 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c32730 0x557540c34b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd8ee2ec29a6b0c server_cookie=6b8c0f7b426a211d in_seq=0 out_seq=0 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.362+0000 7fd66be00640 10 client.25304 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd66be00640 1 -- 172.21.15.59:0/2921859833 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557540b82800 con 0x557540c32730 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd66be00640 10 client.25304 handle_command_reply: tid=0 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 client.25304 shutdown 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 msgr2=0x557540c169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c169e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd68c19f880 tx=0x7fd67809ace0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 shutdown_connections 2024-09-05T11:57:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c32730 0x557540c34b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0f5d0 0x557540c169e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2921859833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c19730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 >> 172.21.15.59:0/2921859833 conn(0x557540c0b240 msgr2=0x557540c10f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.363+0000 7fd66be00640 0 client.25304 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.364+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 shutdown_connections 2024-09-05T11:57:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.364+0000 7fd693bd6740 1 -- 172.21.15.59:0/2921859833 wait complete. 2024-09-05T11:57:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.364+0000 7fd693bd6740 20 client.25304 trim_cache size 0 max 16384 2024-09-05T11:57:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 Processor -- start 2024-09-05T11:57:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 -- start start 2024-09-05T11:57:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 0x557540c12110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 0x557540c10e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00ddf0 con 0x557540c19e90 2024-09-05T11:57:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c010670 con 0x557540c0ecb0 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.383+0000 7fd693bd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00e460 con 0x557540c0f5d0 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 0x557540c10e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 0x557540c10e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44338/0 (socket says 172.21.15.59:44338) 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689400640 1 -- 172.21.15.59:0/2075838781 learned_addr learned my addr 172.21.15.59:0/2075838781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 0x557540c12110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd68a800640 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 -- 172.21.15.59:0/2075838781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 msgr2=0x557540c10e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 0x557540c10e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 -- 172.21.15.59:0/2075838781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 msgr2=0x557540c19250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c19250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.384+0000 7fd689e00640 1 -- 172.21.15.59:0/2075838781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557540c0bd80 con 0x557540c0ecb0 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.385+0000 7fd689e00640 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 0x557540c12110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd67c003db0 tx=0x7fd67c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e916fc4426025 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.385+0000 7fd676a00640 1 -- 172.21.15.59:0/2075838781 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004130 con 0x557540c0ecb0 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.385+0000 7fd676a00640 1 -- 172.21.15.59:0/2075838781 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c003ad0 con 0x557540c0ecb0 2024-09-05T11:57:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.385+0000 7fd676a00640 1 -- 172.21.15.59:0/2075838781 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004600 con 0x557540c0ecb0 2024-09-05T11:57:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 -- 172.21.15.59:0/2075838781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 msgr2=0x557540c12110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 0x557540c12110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd67c003db0 tx=0x7fd67c011810 comp rx=0 tx=0).stop 2024-09-05T11:57:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 -- 172.21.15.59:0/2075838781 shutdown_connections 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c19e90 0x557540c19250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0f5d0 0x557540c10e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 --2- 172.21.15.59:0/2075838781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c0ecb0 0x557540c12110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 -- 172.21.15.59:0/2075838781 >> 172.21.15.59:0/2075838781 conn(0x557540c0b240 msgr2=0x557540c10750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 -- 172.21.15.59:0/2075838781 shutdown_connections 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.386+0000 7fd693bd6740 1 -- 172.21.15.59:0/2075838781 wait complete. 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 -- start start 2024-09-05T11:57:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 Processor -- start 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 0x557540c1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c19e90 0x557540c9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00eb30 con 0x557540c0f5d0 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00ffa0 con 0x557540c19e90 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00f200 con 0x557540c0ecb0 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.387+0000 7fd693bd6740 10 client.0 _ll_register_callbacks cb 0x7fd6780d1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd689400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 0x557540c1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd689e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd689400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 0x557540c1e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58140/0 (socket says 172.21.15.59:58140) 2024-09-05T11:57:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd689e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44342/0 (socket says 172.21.15.59:44342) 2024-09-05T11:57:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd689400640 1 -- 172.21.15.59:0/3025413815 learned_addr learned my addr 172.21.15.59:0/3025413815 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.388+0000 7fd68a800640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c19e90 0x557540c9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 -- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c19e90 msgr2=0x557540c9c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c19e90 0x557540c9c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 -- 172.21.15.59:0/3025413815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 msgr2=0x557540c1e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 0x557540c1e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 -- 172.21.15.59:0/3025413815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557540c9c940 con 0x557540c0ecb0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd689e00640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5595fc577d7d162e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd66be00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c03b060 con 0x557540c0ecb0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.389+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c003ad0 con 0x557540c0ecb0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.390+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd67c0035f0 con 0x557540c0ecb0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.390+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd67c042050 con 0x557540c0ecb0 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.390+0000 7fd693bd6740 10 client.25252 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.390+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557540c9db40 con 0x557540c0ecb0 2024-09-05T11:57:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.390+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c003ad0 con 0x557540c0ecb0 2024-09-05T11:57:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd67c02b030 con 0x557540c0ecb0 2024-09-05T11:57:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 10 client.25252 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 10 client.25252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 20 client.25252 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c9dfb0 0x557540ca03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 4 client.25252 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557540b7cd10 con 0x557540c9dfb0 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.391+0000 7fd689400640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c9dfb0 0x557540ca03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.392+0000 7fd689400640 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c9dfb0 0x557540ca03a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4b6926309d8271 server_cookie=781b52753319b9e9 in_seq=0 out_seq=0 2024-09-05T11:57:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.392+0000 7fd66be00640 10 client.25252 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.393+0000 7fd66be00640 1 -- 172.21.15.59:0/3025413815 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557540b7cd10 con 0x557540c9dfb0 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.393+0000 7fd66be00640 10 client.25252 handle_command_reply: tid=0 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.393+0000 7fd693bd6740 1 client.25252 shutdown 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 msgr2=0x557540c1e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c0096d0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 shutdown_connections 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557540c19e90 0x557540c9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x557540c9dfb0 0x557540ca03a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557540c0f5d0 0x557540c1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 --2- 172.21.15.59:0/3025413815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557540c0ecb0 0x557540c1e0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 >> 172.21.15.59:0/3025413815 conn(0x557540c0b240 msgr2=0x557540c0a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.394+0000 7fd66be00640 0 client.25252 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.395+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 shutdown_connections 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.395+0000 7fd693bd6740 1 -- 172.21.15.59:0/3025413815 wait complete. 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.395+0000 7fd693bd6740 20 client.25252 trim_cache size 0 max 16384 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "730cc1ac-1c84-4f32-bb31-87b0f91d6dd9", 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:57:49.397 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.396+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 msgr2=0x7fd6700a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.396+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 0x7fd6700a4f50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd684006010 tx=0x7fd684005d80 comp rx=0 tx=0).stop 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.396+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 msgr2=0x7fd68c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.396+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800edf0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 shutdown_connections 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd6700a2a90 0x7fd6700a4f50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c108f60 0x7fd68c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd68c1047e0 0x7fd68c19a950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 --2- 172.21.15.59:0/308491269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd68c103e30 0x7fd68c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:49.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 >> 172.21.15.59:0/308491269 conn(0x7fd68c0ff910 msgr2=0x7fd68c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:49.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 shutdown_connections 2024-09-05T11:57:49.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:49.397+0000 7fd68b200640 1 -- 172.21.15.59:0/308491269 wait complete. 2024-09-05T11:57:50.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:57:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:57:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:57:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:50 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:50 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: pgmap v399: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:57:50.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:50 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:57:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:51.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 -- 172.21.15.59:0/293137906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 msgr2=0x7f4130108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 --2- 172.21.15.59:0/293137906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f4130108830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f411c009900 tx=0x7f411c02a410 comp rx=0 tx=0).stop 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 -- 172.21.15.59:0/293137906 shutdown_connections 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 --2- 172.21.15.59:0/293137906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f4130110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 --2- 172.21.15.59:0/293137906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f4130108830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 --2- 172.21.15.59:0/293137906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f4130104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 -- 172.21.15.59:0/293137906 >> 172.21.15.59:0/293137906 conn(0x7f41300ff910 msgr2=0x7f4130101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.521+0000 7f4135e00640 1 -- 172.21.15.59:0/293137906 shutdown_connections 2024-09-05T11:57:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.522+0000 7f4135e00640 1 -- 172.21.15.59:0/293137906 wait complete. 2024-09-05T11:57:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.523+0000 7f4135e00640 1 Processor -- start 2024-09-05T11:57:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.523+0000 7f4135e00640 1 -- start start 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f413019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f413019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f413019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114010 con 0x7f41301047e0 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4130113e90 con 0x7f4130108f60 2024-09-05T11:57:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114190 con 0x7f4130103e30 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4134a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f413019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4134a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f413019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45392/0 (socket says 172.21.15.59:45392) 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4134a00640 1 -- 172.21.15.59:0/2499251619 learned_addr learned my addr 172.21.15.59:0/2499251619 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f4135400640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f413019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.524+0000 7f412fe00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f413019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.525+0000 7f4135400640 1 -- 172.21.15.59:0/2499251619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 msgr2=0x7f413019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.525+0000 7f4135400640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f413019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.525+0000 7f4135400640 1 -- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 msgr2=0x7f413019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.526+0000 7f4135400640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f413019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.526+0000 7f4135400640 1 -- 172.21.15.59:0/2499251619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413019f460 con 0x7f4130108f60 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.526+0000 7f412fe00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f413019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.526+0000 7f4135400640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f413019ed80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f412800ec50 tx=0x7f412800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.527+0000 7f412d600640 1 -- 172.21.15.59:0/2499251619 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128003b70 con 0x7f4130108f60 2024-09-05T11:57:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.527+0000 7f412d600640 1 -- 172.21.15.59:0/2499251619 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4128003d10 con 0x7f4130108f60 2024-09-05T11:57:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.527+0000 7f412d600640 1 -- 172.21.15.59:0/2499251619 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128010890 con 0x7f4130108f60 2024-09-05T11:57:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.527+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413019f750 con 0x7f4130108f60 2024-09-05T11:57:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.527+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301a7030 con 0x7f4130108f60 2024-09-05T11:57:51.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.531+0000 7f412d600640 1 -- 172.21.15.59:0/2499251619 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4128003ec0 con 0x7f4130108f60 2024-09-05T11:57:51.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.533+0000 7f4139e61740 1 Processor -- start 2024-09-05T11:57:51.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.533+0000 7f4139e61740 1 -- start start 2024-09-05T11:57:51.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.534+0000 7f412d600640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 0x7f41180a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.534+0000 7f412d600640 1 -- 172.21.15.59:0/2499251619 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f41280cf920 con 0x7f4130108f60 2024-09-05T11:57:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.534+0000 7f4139e61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7185130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.534+0000 7f4139e61740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.534+0000 7f4139e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4139e61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e8e70eb3e0 con 0x55e8e7184d50 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4139e61740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7092d80 con 0x55e8e7185670 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4139e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8e70e73a0 con 0x55e8e718ff30 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4135400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4135400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45400/0 (socket says 172.21.15.59:45400) 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f4135400640 1 -- 172.21.15.59:0/1405953059 learned_addr learned my addr 172.21.15.59:0/1405953059 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.535+0000 7f412fe00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 0x7f41180a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4134a00640 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7185130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4135400640 1 -- 172.21.15.59:0/1405953059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 msgr2=0x55e8e718f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4135400640 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4135400640 1 -- 172.21.15.59:0/1405953059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 msgr2=0x55e8e7185130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4135400640 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7185130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.536+0000 7f4135400640 1 -- 172.21.15.59:0/1405953059 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e70b1ae0 con 0x55e8e718ff30 2024-09-05T11:57:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.537+0000 7f4134a00640 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7185130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.537+0000 7f412fe00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 0x7f41180a5140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c009620 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.538+0000 7f4135400640 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f4128000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba11830e3f2319be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.538+0000 7f4121c00640 1 -- 172.21.15.59:0/1405953059 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412809c410 con 0x55e8e718ff30 2024-09-05T11:57:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.538+0000 7f4121c00640 1 -- 172.21.15.59:0/1405953059 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412809bd10 con 0x55e8e718ff30 2024-09-05T11:57:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.539+0000 7f4121c00640 1 -- 172.21.15.59:0/1405953059 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128004da0 con 0x55e8e718ff30 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.539+0000 7f4139e61740 1 -- 172.21.15.59:0/1405953059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 msgr2=0x55e8e7192320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.539+0000 7f4139e61740 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f4128000f40 comp rx=0 tx=0).stop 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.539+0000 7f4139e61740 1 -- 172.21.15.59:0/1405953059 shutdown_connections 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e7192320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 --2- 172.21.15.59:0/1405953059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7185130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 -- 172.21.15.59:0/1405953059 >> 172.21.15.59:0/1405953059 conn(0x55e8e71812e0 msgr2=0x55e8e71816c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 -- 172.21.15.59:0/1405953059 shutdown_connections 2024-09-05T11:57:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 -- 172.21.15.59:0/1405953059 wait complete. 2024-09-05T11:57:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.540+0000 7f4139e61740 1 -- start start 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.541+0000 7f4139e61740 1 Processor -- start 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 0x55e8e718ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7185670 0x55e8e718d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e718d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4134a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 0x55e8e718ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4134a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 0x55e8e718ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43718/0 (socket says 172.21.15.59:43718) 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7192b80 con 0x55e8e7185670 2024-09-05T11:57:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4134a00640 1 -- 172.21.15.59:0/3919434727 learned_addr learned my addr 172.21.15.59:0/3919434727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f412fe00640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7185670 0x55e8e718d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e8e70a60f0 con 0x55e8e7184d50 2024-09-05T11:57:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7192d00 con 0x55e8e718ff30 2024-09-05T11:57:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.542+0000 7f4139e61740 10 client.0 _ll_register_callbacks cb 0x55e8e70979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e718d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 -- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 msgr2=0x55e8e718ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 0x55e8e718ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 -- 172.21.15.59:0/3919434727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7185670 msgr2=0x55e8e718d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7185670 0x55e8e718d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.543+0000 7f4135400640 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e6e5c3c0 con 0x55e8e718ff30 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.544+0000 7f4135400640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e718d8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f41280979b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c956b1bf9010b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.544+0000 7f4116000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.544+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280a4980 con 0x55e8e718ff30 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.544+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f41280043c0 con 0x55e8e718ff30 2024-09-05T11:57:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.544+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412809bd10 con 0x55e8e718ff30 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.545+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f412809c410 con 0x55e8e718ff30 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.545+0000 7f4139e61740 10 client.25264 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.545+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e8e7192bd0 con 0x55e8e718ff30 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.545+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412809bd10 con 0x55e8e718ff30 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.545+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f412809b160 con 0x55e8e718ff30 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 10 client.25264 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 20 client.25264 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e719fac0 0x55e8e71a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 4 client.25264 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e70f8950 con 0x55e8e719fac0 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.546+0000 7f4134a00640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e719fac0 0x55e8e71a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.547+0000 7f4134a00640 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e719fac0 0x55e8e71a1eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20393a6e997213cb server_cookie=391a5ed180255aee in_seq=0 out_seq=0 2024-09-05T11:57:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.547+0000 7f4116000640 10 client.25264 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4116000640 1 -- 172.21.15.59:0/3919434727 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8e70f8950 con 0x55e8e719fac0 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4116000640 10 client.25264 handle_command_reply: tid=0 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 client.25264 shutdown 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 msgr2=0x55e8e718d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e718d8b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f41280979b0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 shutdown_connections 2024-09-05T11:57:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e718ff30 0x55e8e718d8b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e719fac0 0x55e8e71a1eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7185670 0x55e8e718d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 --2- 172.21.15.59:0/3919434727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7184d50 0x55e8e718ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.548+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 >> 172.21.15.59:0/3919434727 conn(0x55e8e71812e0 msgr2=0x55e8e7187010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.549+0000 7f4116000640 0 client.25264 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.549+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 shutdown_connections 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.549+0000 7f4139e61740 1 -- 172.21.15.59:0/3919434727 wait complete. 2024-09-05T11:57:51.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.549+0000 7f4139e61740 20 client.25264 trim_cache size 0 max 16384 2024-09-05T11:57:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 Processor -- start 2024-09-05T11:57:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 -- start start 2024-09-05T11:57:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 0x55e8e71a7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41301aefd0 con 0x55e8e718ff30 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b1850 con 0x55e8e7185670 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.585+0000 7f4139e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41301af640 con 0x55e8e7184d50 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f4134a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f4134a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45422/0 (socket says 172.21.15.59:45422) 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f4134a00640 1 -- 172.21.15.59:0/3018456961 learned_addr learned my addr 172.21.15.59:0/3018456961 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f412fe00640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f4135400640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 0x55e8e71a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f412fe00640 1 -- 172.21.15.59:0/3018456961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 msgr2=0x55e8e7188110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f412fe00640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f412fe00640 1 -- 172.21.15.59:0/3018456961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 msgr2=0x55e8e71a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.586+0000 7f412fe00640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 0x55e8e71a7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.587+0000 7f412fe00640 1 -- 172.21.15.59:0/3018456961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e7181110 con 0x55e8e7185670 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.587+0000 7f4135400640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 0x55e8e71a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.587+0000 7f4134a00640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.587+0000 7f412fe00640 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718d700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f411c02aa00 tx=0x7f411c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8c95b07b400a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.588+0000 7f4121200640 1 -- 172.21.15.59:0/3018456961 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c003ae0 con 0x55e8e7185670 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.588+0000 7f4121200640 1 -- 172.21.15.59:0/3018456961 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411c003c80 con 0x55e8e7185670 2024-09-05T11:57:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.588+0000 7f4121200640 1 -- 172.21.15.59:0/3018456961 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c03d7d0 con 0x55e8e7185670 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 -- 172.21.15.59:0/3018456961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 msgr2=0x55e8e718d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718d700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f411c02aa00 tx=0x7f411c03f000 comp rx=0 tx=0).stop 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 -- 172.21.15.59:0/3018456961 shutdown_connections 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e718ff30 0x55e8e71a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e7185670 0x55e8e718d700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 --2- 172.21.15.59:0/3018456961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7184d50 0x55e8e7188110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 -- 172.21.15.59:0/3018456961 >> 172.21.15.59:0/3018456961 conn(0x55e8e71812e0 msgr2=0x55e8e718eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.589+0000 7f4139e61740 1 -- 172.21.15.59:0/3018456961 shutdown_connections 2024-09-05T11:57:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.590+0000 7f4139e61740 1 -- 172.21.15.59:0/3018456961 wait complete. 2024-09-05T11:57:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.590+0000 7f4139e61740 1 -- start start 2024-09-05T11:57:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.590+0000 7f4139e61740 1 Processor -- start 2024-09-05T11:57:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 0x55e8e71984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 0x55e8e7194f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41301afd10 con 0x55e8e7184d50 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b1180 con 0x55e8e718ff30 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b03e0 con 0x55e8e7185670 2024-09-05T11:57:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.591+0000 7f4139e61740 10 client.0 _ll_register_callbacks cb 0x7f4124009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f4134a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f4134a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48956/0 (socket says 172.21.15.59:48956) 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f4134a00640 1 -- 172.21.15.59:0/1593247822 learned_addr learned my addr 172.21.15.59:0/1593247822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f412fe00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 0x55e8e71984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f4135400640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 0x55e8e7194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f412fe00640 1 -- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 msgr2=0x55e8e7194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.592+0000 7f412fe00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 0x55e8e7194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f412fe00640 1 -- 172.21.15.59:0/1593247822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 msgr2=0x55e8e7197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f412fe00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f412fe00640 1 -- 172.21.15.59:0/1593247822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e71862d0 con 0x55e8e7185670 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f4135400640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 0x55e8e7194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f412fe00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 0x55e8e71984e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e8e708dd10 tx=0x7f411c003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1522cddaecdc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.593+0000 7f4116000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c05d060 con 0x55e8e7185670 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f411c003c80 con 0x55e8e7185670 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f411c0315b0 con 0x55e8e7185670 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f411c064050 con 0x55e8e7185670 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4139e61740 10 client.25270 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.594+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e8e720ab40 con 0x55e8e7185670 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411c003c80 con 0x55e8e7185670 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f411c044050 con 0x55e8e7185670 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 10 client.25270 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 20 client.25270 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e720b250 0x55e8e720d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 4 client.25270 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8e70f2e60 con 0x55e8e720b250 2024-09-05T11:57:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.595+0000 7f4134a00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e720b250 0x55e8e720d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.596+0000 7f4134a00640 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e720b250 0x55e8e720d670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336eef7b229de29c server_cookie=c2af065656d92f0b in_seq=0 out_seq=0 2024-09-05T11:57:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.596+0000 7f4116000640 10 client.25270 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.596+0000 7f4116000640 1 -- 172.21.15.59:0/1593247822 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e8e70f2e60 con 0x55e8e720b250 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.596+0000 7f4116000640 10 client.25270 handle_command_reply: tid=0 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.596+0000 7f4139e61740 1 client.25270 shutdown 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 msgr2=0x55e8e71984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 0x55e8e71984e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e8e708dd10 tx=0x7f411c003e30 comp rx=0 tx=0).stop 2024-09-05T11:57:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 shutdown_connections 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e8e718ff30 0x55e8e7194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8e7185670 0x55e8e71984e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e8e720b250 0x55e8e720d670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4116000640 0 client.25270 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 --2- 172.21.15.59:0/1593247822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8e7184d50 0x55e8e7197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 >> 172.21.15.59:0/1593247822 conn(0x55e8e71812e0 msgr2=0x55e8e71867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.597+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 shutdown_connections 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.598+0000 7f4139e61740 1 -- 172.21.15.59:0/1593247822 wait complete. 2024-09-05T11:57:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.598+0000 7f4139e61740 20 client.25270 trim_cache size 0 max 16384 2024-09-05T11:57:51.601 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:51.601 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:57:51.601 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:57:51.601 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 msgr2=0x7f41180a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 0x7f41180a5140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c009620 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 msgr2=0x7f413019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f413019ed80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f412800ec50 tx=0x7f412800c6a0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 shutdown_connections 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f41180a2c80 0x7f41180a5140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130108f60 0x7f413019ed80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41301047e0 0x7f413019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 --2- 172.21.15.59:0/2499251619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4130103e30 0x7f413019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 >> 172.21.15.59:0/2499251619 conn(0x7f41300ff910 msgr2=0x7f4130100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 shutdown_connections 2024-09-05T11:57:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:51.599+0000 7f4135e00640 1 -- 172.21.15.59:0/2499251619 wait complete. 2024-09-05T11:57:52.315 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:52 smithi059 ceph-mon[30025]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:52.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:57:52.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:52.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:52 smithi089 ceph-mon[34499]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:52.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:52 smithi189 ceph-mon[34772]: pgmap v400: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T11:57:52.811 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 -- 172.21.15.59:0/2613679215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc4102360 msgr2=0x7fabc410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 --2- 172.21.15.59:0/2613679215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc4102360 0x7fabc410e890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009cd0 tx=0x7fabbc02a8e0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 -- 172.21.15.59:0/2613679215 shutdown_connections 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 --2- 172.21.15.59:0/2613679215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc4102360 0x7fabc410e890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 --2- 172.21.15.59:0/2613679215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabc41019c0 0x7fabc4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 --2- 172.21.15.59:0/2613679215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 -- 172.21.15.59:0/2613679215 >> 172.21.15.59:0/2613679215 conn(0x7fabc40fd740 msgr2=0x7fabc40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:53.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 -- 172.21.15.59:0/2613679215 shutdown_connections 2024-09-05T11:57:53.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.456+0000 7fabc9600640 1 -- 172.21.15.59:0/2613679215 wait complete. 2024-09-05T11:57:53.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.458+0000 7fabc9600640 1 Processor -- start 2024-09-05T11:57:53.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.458+0000 7fabc9600640 1 -- start start 2024-09-05T11:57:53.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 0x7fabc4075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:53.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabc4102360 0x7fabc4070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:53.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:53.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4111e10 con 0x7fabc41019c0 2024-09-05T11:57:53.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4111c90 con 0x7fabc41079c0 2024-09-05T11:57:53.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabc4111f90 con 0x7fabc4102360 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 0x7fabc4075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43754/0 (socket says 172.21.15.59:43754) 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.459+0000 7fabc8c00640 1 -- 172.21.15.59:0/855591327 learned_addr learned my addr 172.21.15.59:0/855591327 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc8c00640 1 -- 172.21.15.59:0/855591327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabc4102360 msgr2=0x7fabc4070eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc8c00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabc4102360 0x7fabc4070eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc8c00640 1 -- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 msgr2=0x7fabc4075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc8c00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 0x7fabc4075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc8c00640 1 -- 172.21.15.59:0/855591327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc4071b20 con 0x7fabc41079c0 2024-09-05T11:57:53.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.460+0000 7fabc3e00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 0x7fabc4075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:53.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.461+0000 7fabc8c00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fabbc033040 tx=0x7fabbc031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:53.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.461+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc004040 con 0x7fabc41079c0 2024-09-05T11:57:53.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.461+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc0046c0 con 0x7fabc41079c0 2024-09-05T11:57:53.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.461+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc030e80 con 0x7fabc41079c0 2024-09-05T11:57:53.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.462+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41af2d0 con 0x7fabc41079c0 2024-09-05T11:57:53.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.462+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41af6d0 con 0x7fabc41079c0 2024-09-05T11:57:53.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.463+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc4103100 con 0x7fabc41079c0 2024-09-05T11:57:53.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.467+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabbc0305a0 con 0x7fabc41079c0 2024-09-05T11:57:53.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.470+0000 7fabc0c00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 0x7faba80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.470+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fabbc02e030 con 0x7fabc41079c0 2024-09-05T11:57:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.470+0000 7fabc3e00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 0x7faba80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.471+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabbc0b93d0 con 0x7fabc41079c0 2024-09-05T11:57:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.472+0000 7fabc3e00640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 0x7faba80a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009ba0 tx=0x7fabb4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.918+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc4101e20 con 0x7fabc41079c0 2024-09-05T11:57:53.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.919+0000 7fabc0c00640 1 -- 172.21.15.59:0/855591327 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fabc4101e20 con 0x7fabc41079c0 2024-09-05T11:57:53.921 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:53.921 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:53.922 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 msgr2=0x7faba80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 0x7faba80a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009ba0 tx=0x7fabb4005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 msgr2=0x7fabc40713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fabbc033040 tx=0x7fabbc031e60 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 shutdown_connections 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faba80a2c00 0x7faba80a50c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41079c0 0x7fabc40713f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabc4102360 0x7fabc4070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 --2- 172.21.15.59:0/855591327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc41019c0 0x7fabc4075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 >> 172.21.15.59:0/855591327 conn(0x7fabc40fd740 msgr2=0x7fabc40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 shutdown_connections 2024-09-05T11:57:53.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:53.922+0000 7fabc9600640 1 -- 172.21.15.59:0/855591327 wait complete. 2024-09-05T11:57:54.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:54 smithi059 ceph-mon[30025]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:54.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:54 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/855591327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:54.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:54 smithi089 ceph-mon[34499]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:54.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:54 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/855591327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:54.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:54 smithi189 ceph-mon[34772]: pgmap v401: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T11:57:54.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:54 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/855591327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:54.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:57:55.299 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:55.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:57:55.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.987+0000 7f676e400640 1 -- 172.21.15.59:0/820595894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768112a50 msgr2=0x7f6768114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.987+0000 7f676e400640 1 --2- 172.21.15.59:0/820595894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768112a50 0x7f6768114e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6760009ca0 tx=0x7f676002a8b0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 -- 172.21.15.59:0/820595894 shutdown_connections 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 --2- 172.21.15.59:0/820595894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768112a50 0x7f6768114e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 --2- 172.21.15.59:0/820595894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 0x7f6768112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 --2- 172.21.15.59:0/820595894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768076720 0x7f6768076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 -- 172.21.15.59:0/820595894 >> 172.21.15.59:0/820595894 conn(0x7f67680ffa50 msgr2=0x7f6768101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 -- 172.21.15.59:0/820595894 shutdown_connections 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.988+0000 7f676e400640 1 -- 172.21.15.59:0/820595894 wait complete. 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.989+0000 7f676e400640 1 Processor -- start 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.990+0000 7f676e400640 1 -- start start 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.990+0000 7f676e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.990+0000 7f676e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 0x7f67681a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f676e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768112a50 0x7f676819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f676e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67681185e0 con 0x7f6768076720 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f676e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6768118460 con 0x7f6768077040 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f676e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6768118760 con 0x7f6768112a50 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f6767400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f6767400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48996/0 (socket says 172.21.15.59:48996) 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f6767400640 1 -- 172.21.15.59:0/2361726764 learned_addr learned my addr 172.21.15.59:0/2361726764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f6766a00640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 0x7f67681a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.991+0000 7f6767e00640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768112a50 0x7f676819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.992+0000 7f6767400640 1 -- 172.21.15.59:0/2361726764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768112a50 msgr2=0x7f676819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.992+0000 7f6767400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768112a50 0x7f676819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.992+0000 7f6767400640 1 -- 172.21.15.59:0/2361726764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 msgr2=0x7f67681a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.992+0000 7f6767400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 0x7f67681a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:55.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.992+0000 7f6767400640 1 -- 172.21.15.59:0/2361726764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676819f020 con 0x7f6768076720 2024-09-05T11:57:55.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.993+0000 7f6767400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f675800e9f0 tx=0x7f675800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:55.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.993+0000 7f675fe00640 1 -- 172.21.15.59:0/2361726764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675800cd80 con 0x7f6768076720 2024-09-05T11:57:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.993+0000 7f675fe00640 1 -- 172.21.15.59:0/2361726764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6758010910 con 0x7f6768076720 2024-09-05T11:57:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.993+0000 7f675fe00640 1 -- 172.21.15.59:0/2361726764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758018990 con 0x7f6768076720 2024-09-05T11:57:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.994+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676819f310 con 0x7f6768076720 2024-09-05T11:57:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.994+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67681ab600 con 0x7f6768076720 2024-09-05T11:57:56.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:55.999+0000 7f6770e24740 1 Processor -- start 2024-09-05T11:57:56.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.000+0000 7f6770e24740 1 -- start start 2024-09-05T11:57:56.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.000+0000 7f675fe00640 1 -- 172.21.15.59:0/2361726764 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6758010ac0 con 0x7f6768076720 2024-09-05T11:57:56.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.000+0000 7f6770e24740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.000+0000 7f6770e24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d5590 0x55cd3e6df910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.001+0000 7f6770e24740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.001+0000 7f6770e24740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e63b3e0 con 0x55cd3e6d4c70 2024-09-05T11:57:56.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.001+0000 7f6770e24740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e5e2d80 con 0x55cd3e6dfe50 2024-09-05T11:57:56.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.001+0000 7f6770e24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e6373a0 con 0x55cd3e6d5590 2024-09-05T11:57:56.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.002+0000 7f6767e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.002+0000 7f6767e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49004/0 (socket says 172.21.15.59:49004) 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.002+0000 7f6767e00640 1 -- 172.21.15.59:0/4122476437 learned_addr learned my addr 172.21.15.59:0/4122476437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.002+0000 7f6766a00640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.002+0000 7f6767400640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d5590 0x55cd3e6df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6767e00640 1 -- 172.21.15.59:0/4122476437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d5590 msgr2=0x55cd3e6df910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6767e00640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d5590 0x55cd3e6df910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6767e00640 1 -- 172.21.15.59:0/4122476437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 msgr2=0x55cd3e6e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6767e00640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6767e00640 1 -- 172.21.15.59:0/4122476437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd3e601ae0 con 0x55cd3e6d4c70 2024-09-05T11:57:56.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.003+0000 7f6766a00640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:56.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.004+0000 7f6767e00640 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6760009ca0 tx=0x7f67600303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730b6ea59aee6a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:56.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.005+0000 7f675cc00640 1 -- 172.21.15.59:0/4122476437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760030710 con 0x55cd3e6d4c70 2024-09-05T11:57:56.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.005+0000 7f675cc00640 1 -- 172.21.15.59:0/4122476437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760036040 con 0x55cd3e6d4c70 2024-09-05T11:57:56.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.005+0000 7f675cc00640 1 -- 172.21.15.59:0/4122476437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760004460 con 0x55cd3e6d4c70 2024-09-05T11:57:56.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.005+0000 7f675fe00640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 0x7f674c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f675fe00640 1 -- 172.21.15.59:0/2361726764 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f67580cf990 con 0x7f6768076720 2024-09-05T11:57:56.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 -- 172.21.15.59:0/4122476437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 msgr2=0x55cd3e6d5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6760009ca0 tx=0x7f67600303a0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 -- 172.21.15.59:0/4122476437 shutdown_connections 2024-09-05T11:57:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d5590 0x55cd3e6df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 --2- 172.21.15.59:0/4122476437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6d5050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6770e24740 1 -- 172.21.15.59:0/4122476437 >> 172.21.15.59:0/4122476437 conn(0x55cd3e6d1200 msgr2=0x55cd3e6d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.006+0000 7f6767400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 0x7f674c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.007+0000 7f6767400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 0x7f674c0a5140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f675800e9c0 tx=0x7f6758018420 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.007+0000 7f6770e24740 1 -- 172.21.15.59:0/4122476437 shutdown_connections 2024-09-05T11:57:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.007+0000 7f6770e24740 1 -- 172.21.15.59:0/4122476437 wait complete. 2024-09-05T11:57:56.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.008+0000 7f6770e24740 1 -- start start 2024-09-05T11:57:56.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.008+0000 7f6770e24740 1 Processor -- start 2024-09-05T11:57:56.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6dd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e6e29c0 con 0x55cd3e6d5590 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e5f60f0 con 0x55cd3e6d4c70 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd3e6e2b40 con 0x55cd3e6dfe50 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.009+0000 7f6770e24740 10 client.0 _ll_register_callbacks cb 0x55cd3e5ddbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.010+0000 7f6766a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.010+0000 7f6766a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49008/0 (socket says 172.21.15.59:49008) 2024-09-05T11:57:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.010+0000 7f6766a00640 1 -- 172.21.15.59:0/2430050290 learned_addr learned my addr 172.21.15.59:0/2430050290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.011+0000 7f6766a00640 1 -- 172.21.15.59:0/2430050290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6dfe50 msgr2=0x55cd3e6dd840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:57:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.011+0000 7f6766a00640 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6dd840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.011+0000 7f6766a00640 1 -- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 msgr2=0x55cd3e6dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.011+0000 7f6766a00640 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.011+0000 7f6766a00640 1 -- 172.21.15.59:0/2430050290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd3e3ac3c0 con 0x55cd3e6d5590 2024-09-05T11:57:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.012+0000 7f6766a00640 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f675400b9f0 tx=0x7f675400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76128ec2206c4b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.012+0000 7f6751600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:57:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.012+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754003e20 con 0x55cd3e6d5590 2024-09-05T11:57:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.012+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6754011440 con 0x55cd3e6d5590 2024-09-05T11:57:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.012+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6760036040 con 0x55cd3e6d5590 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.013+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6754011640 con 0x55cd3e6d5590 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.013+0000 7f6770e24740 10 client.15339 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.013+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cd3e6eeb10 con 0x55cd3e6d5590 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760036040 con 0x55cd3e6d5590 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f675401b9f0 con 0x55cd3e6d5590 2024-09-05T11:57:56.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 10 client.15339 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 20 client.15339 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e6f0020 0x55cd3e6f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 4 client.15339 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.014+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd3e648950 con 0x55cd3e6f0020 2024-09-05T11:57:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.015+0000 7f6767e00640 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e6f0020 0x55cd3e6f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.015+0000 7f6751600640 10 client.15339 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.017+0000 7f6767e00640 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e6f0020 0x55cd3e6f24a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35b0b624f282bac server_cookie=1cc92907c225537e in_seq=0 out_seq=0 2024-09-05T11:57:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.018+0000 7f6751600640 1 -- 172.21.15.59:0/2430050290 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd3e648950 con 0x55cd3e6f0020 2024-09-05T11:57:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.018+0000 7f6751600640 10 client.15339 handle_command_reply: tid=0 2024-09-05T11:57:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.018+0000 7f6770e24740 1 client.15339 shutdown 2024-09-05T11:57:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 msgr2=0x55cd3e6dd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f675400b9f0 tx=0x7f675400beb0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 shutdown_connections 2024-09-05T11:57:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6dd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e6f0020 0x55cd3e6f24a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6d5590 0x55cd3e6dd300 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f675400b9f0 tx=0x7f675400beb0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 --2- 172.21.15.59:0/2430050290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 >> 172.21.15.59:0/2430050290 conn(0x55cd3e6d1200 msgr2=0x55cd3e6d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:56.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.019+0000 7f6751600640 0 client.15339 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:56.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.023+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 shutdown_connections 2024-09-05T11:57:56.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.023+0000 7f6770e24740 1 -- 172.21.15.59:0/2430050290 wait complete. 2024-09-05T11:57:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.023+0000 7f6770e24740 20 client.15339 trim_cache size 0 max 16384 2024-09-05T11:57:56.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.051+0000 7f6770e24740 1 Processor -- start 2024-09-05T11:57:56.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.051+0000 7f6770e24740 1 -- start start 2024-09-05T11:57:56.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d87a0 0x55cd3e6de620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b3790 con 0x55cd3e6dfe50 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b6010 con 0x55cd3e6d4c70 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.052+0000 7f6770e24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b3e00 con 0x55cd3e6d87a0 2024-09-05T11:57:56.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.053+0000 7f6766a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.053+0000 7f6766a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49010/0 (socket says 172.21.15.59:49010) 2024-09-05T11:57:56.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.053+0000 7f6766a00640 1 -- 172.21.15.59:0/3800038266 learned_addr learned my addr 172.21.15.59:0/3800038266 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:56.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.053+0000 7f6767e00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.053+0000 7f6767e00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d87a0 0x55cd3e6de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.054+0000 7f6767e00640 1 -- 172.21.15.59:0/3800038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d87a0 msgr2=0x55cd3e6de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.054+0000 7f6767e00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d87a0 0x55cd3e6de620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.054+0000 7f6767e00640 1 -- 172.21.15.59:0/3800038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 msgr2=0x55cd3e6f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.054+0000 7f6767e00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.054+0000 7f6767e00640 1 -- 172.21.15.59:0/3800038266 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd3e6cc770 con 0x55cd3e6d4c70 2024-09-05T11:57:56.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.055+0000 7f6766a00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:56.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.055+0000 7f6767e00640 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6f7d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f676002aea0 tx=0x7f6760000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9b9b05f1519ded server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:56.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.056+0000 7f6753e00640 1 -- 172.21.15.59:0/3800038266 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760055070 con 0x55cd3e6d4c70 2024-09-05T11:57:56.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.056+0000 7f6753e00640 1 -- 172.21.15.59:0/3800038266 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760036030 con 0x55cd3e6d4c70 2024-09-05T11:57:56.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.056+0000 7f6753e00640 1 -- 172.21.15.59:0/3800038266 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760055070 con 0x55cd3e6d4c70 2024-09-05T11:57:56.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.057+0000 7f6770e24740 1 -- 172.21.15.59:0/3800038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 msgr2=0x55cd3e6f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6f7d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f676002aea0 tx=0x7f6760000f40 comp rx=0 tx=0).stop 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 -- 172.21.15.59:0/3800038266 shutdown_connections 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6d87a0 0x55cd3e6de620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 --2- 172.21.15.59:0/3800038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6f7d70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 -- 172.21.15.59:0/3800038266 >> 172.21.15.59:0/3800038266 conn(0x55cd3e6d1200 msgr2=0x55cd3e6d8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:56.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 -- 172.21.15.59:0/3800038266 shutdown_connections 2024-09-05T11:57:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.058+0000 7f6770e24740 1 -- 172.21.15.59:0/3800038266 wait complete. 2024-09-05T11:57:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.059+0000 7f6770e24740 1 -- start start 2024-09-05T11:57:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.059+0000 7f6770e24740 1 Processor -- start 2024-09-05T11:57:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 0x55cd3e759d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b44d0 con 0x55cd3e6dfe50 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b5940 con 0x55cd3e6d4c70 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67681b4ba0 con 0x55cd3e6e47f0 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.060+0000 7f6770e24740 10 client.0 _ll_register_callbacks cb 0x7f6754004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6e3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43818/0 (socket says 172.21.15.59:43818) 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 -- 172.21.15.59:0/2244438518 learned_addr learned my addr 172.21.15.59:0/2244438518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 0x55cd3e759d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 -- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 msgr2=0x55cd3e6e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6e3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 -- 172.21.15.59:0/2244438518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 msgr2=0x55cd3e6e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.061+0000 7f6767e00640 1 -- 172.21.15.59:0/2244438518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd3e75a380 con 0x55cd3e6e47f0 2024-09-05T11:57:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.062+0000 7f6767e00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 0x55cd3e759d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f676002add0 tx=0x7f676002a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebbad690bfe952e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.062+0000 7f6751600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.063+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676004a040 con 0x55cd3e6e47f0 2024-09-05T11:57:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.063+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6760036030 con 0x55cd3e6e47f0 2024-09-05T11:57:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.063+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6760004d40 con 0x55cd3e6e47f0 2024-09-05T11:57:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.064+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6760055040 con 0x55cd3e6e47f0 2024-09-05T11:57:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.064+0000 7f6770e24740 10 client.25288 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.064+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cd3e75b660 con 0x55cd3e6e47f0 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760036030 con 0x55cd3e6e47f0 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f67600045c0 con 0x55cd3e6e47f0 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 10 client.25288 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 20 client.25288 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e75b8f0 0x55cd3e75dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 4 client.25288 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:57:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.065+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd3e642e60 con 0x55cd3e75b8f0 2024-09-05T11:57:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.066+0000 7f6766a00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e75b8f0 0x55cd3e75dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.067+0000 7f6766a00640 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e75b8f0 0x55cd3e75dce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565ccdcbf4244027 server_cookie=28e92b71a4a25092 in_seq=0 out_seq=0 2024-09-05T11:57:56.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.067+0000 7f6751600640 10 client.25288 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:56.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6751600640 1 -- 172.21.15.59:0/2244438518 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd3e642e60 con 0x55cd3e75b8f0 2024-09-05T11:57:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6751600640 10 client.25288 handle_command_reply: tid=0 2024-09-05T11:57:56.071 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:57:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 client.25288 shutdown 2024-09-05T11:57:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 msgr2=0x55cd3e759d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 0x55cd3e759d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f676002add0 tx=0x7f676002a8b0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 shutdown_connections 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd3e6e47f0 0x55cd3e759d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55cd3e75b8f0 0x55cd3e75dce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cd3e6dfe50 0x55cd3e6e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 --2- 172.21.15.59:0/2244438518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cd3e6d4c70 0x55cd3e6e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 >> 172.21.15.59:0/2244438518 conn(0x55cd3e6d1200 msgr2=0x55cd3e6d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.068+0000 7f6751600640 0 client.25288 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.069+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 shutdown_connections 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.069+0000 7f6770e24740 1 -- 172.21.15.59:0/2244438518 wait complete. 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.069+0000 7f6770e24740 20 client.25288 trim_cache size 0 max 16384 2024-09-05T11:57:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 msgr2=0x7f674c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 0x7f674c0a5140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f675800e9c0 tx=0x7f6758018420 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 msgr2=0x7f67681a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f675800e9f0 tx=0x7f675800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 shutdown_connections 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f674c0a2c80 0x7f674c0a5140 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6768112a50 0x7f676819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6768077040 0x7f67681a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 --2- 172.21.15.59:0/2361726764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6768076720 0x7f67681a4730 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.070+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 >> 172.21.15.59:0/2361726764 conn(0x7f67680ffa50 msgr2=0x7f6768101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.071+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 shutdown_connections 2024-09-05T11:57:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:56.071+0000 7f676e400640 1 -- 172.21.15.59:0/2361726764 wait complete. 2024-09-05T11:57:56.391 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:56 smithi189 ceph-mon[34772]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:56 smithi089 ceph-mon[34499]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:56.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:56 smithi059 ceph-mon[30025]: pgmap v402: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T11:57:56.821 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:57.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.452+0000 7f46a3e00640 1 -- 172.21.15.59:0/4259905054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 msgr2=0x7f46a4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.452+0000 7f46a3e00640 1 --2- 172.21.15.59:0/4259905054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a4102f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4694009900 tx=0x7f469402a480 comp rx=0 tx=0).stop 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 -- 172.21.15.59:0/4259905054 shutdown_connections 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 --2- 172.21.15.59:0/4259905054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 --2- 172.21.15.59:0/4259905054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 0x7f46a410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 --2- 172.21.15.59:0/4259905054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a4102f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 -- 172.21.15.59:0/4259905054 >> 172.21.15.59:0/4259905054 conn(0x7f46a40fc610 msgr2=0x7f46a40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 -- 172.21.15.59:0/4259905054 shutdown_connections 2024-09-05T11:57:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.453+0000 7f46a3e00640 1 -- 172.21.15.59:0/4259905054 wait complete. 2024-09-05T11:57:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.454+0000 7f46a3e00640 1 Processor -- start 2024-09-05T11:57:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.454+0000 7f46a3e00640 1 -- start start 2024-09-05T11:57:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 0x7f46a419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:57.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111dc0 con 0x7f46a4102b90 2024-09-05T11:57:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111c40 con 0x7f46a410bfc0 2024-09-05T11:57:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4111f40 con 0x7f46a41034b0 2024-09-05T11:57:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 0x7f46a419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49034/0 (socket says 172.21.15.59:49034) 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a2a00640 1 -- 172.21.15.59:0/3299097264 learned_addr learned my addr 172.21.15.59:0/3299097264 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.455+0000 7f46a3400640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 -- 172.21.15.59:0/3299097264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 msgr2=0x7f46a419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 0x7f46a419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 -- 172.21.15.59:0/3299097264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 msgr2=0x7f46a419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 -- 172.21.15.59:0/3299097264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a419f430 con 0x7f46a4102b90 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a3400640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.456+0000 7f46a2a00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4694006010 tx=0x7f4694004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.457+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469403f030 con 0x7f46a4102b90 2024-09-05T11:57:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.457+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4694004440 con 0x7f46a4102b90 2024-09-05T11:57:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.457+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a419f6c0 con 0x7f46a4102b90 2024-09-05T11:57:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.457+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a7000 con 0x7f46a4102b90 2024-09-05T11:57:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.457+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694035070 con 0x7f46a4102b90 2024-09-05T11:57:57.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.459+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4684005290 con 0x7f46a4102b90 2024-09-05T11:57:57.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.461+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4694042030 con 0x7f46a4102b90 2024-09-05T11:57:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.462+0000 7f46a8a00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 0x7f46880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.462+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f46940ef240 con 0x7f46a4102b90 2024-09-05T11:57:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.463+0000 7f46a2000640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 0x7f46880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:57.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.463+0000 7f46a2000640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 0x7f46880a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46a40fea50 tx=0x7f4690005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.467+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46940bbd30 con 0x7f46a4102b90 2024-09-05T11:57:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.706+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4684005580 con 0x7f46a4102b90 2024-09-05T11:57:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.707+0000 7f46a8a00640 1 -- 172.21.15.59:0/3299097264 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f46940bb630 con 0x7f46a4102b90 2024-09-05T11:57:57.709 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:57:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:57:57.710 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:57:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.710+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 msgr2=0x7f46880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.710+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 0x7f46880a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46a40fea50 tx=0x7f4690005f90 comp rx=0 tx=0).stop 2024-09-05T11:57:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.710+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 msgr2=0x7f46a419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.710+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4694006010 tx=0x7f4694004040 comp rx=0 tx=0).stop 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 shutdown_connections 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46880a2c00 0x7f46880a50c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46a410bfc0 0x7f46a419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46a41034b0 0x7f46a419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 --2- 172.21.15.59:0/3299097264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a4102b90 0x7f46a419a480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.711+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 >> 172.21.15.59:0/3299097264 conn(0x7f46a40fc610 msgr2=0x7f46a4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.712+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 shutdown_connections 2024-09-05T11:57:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:57.712+0000 7f46a3e00640 1 -- 172.21.15.59:0/3299097264 wait complete. 2024-09-05T11:57:57.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:57:57.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:57:58.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:58 smithi059 ceph-mon[30025]: pgmap v403: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-05T11:57:58.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:57:58 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3299097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:58.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:58 smithi089 ceph-mon[34499]: pgmap v403: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-05T11:57:58.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:57:58 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3299097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:58.455 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 526b5f58 --release --await' 2024-09-05T11:57:58.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 526b5f58 --release --await 2024-09-05T11:57:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:58 smithi189 ceph-mon[34772]: pgmap v403: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-09-05T11:57:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:57:58 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3299097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:57:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 -- 172.21.15.59:0/2511475954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074eb0 msgr2=0x7fc784112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 --2- 172.21.15.59:0/2511475954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074eb0 0x7fc784112650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc770009860 tx=0x7fc77002a3d0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 -- 172.21.15.59:0/2511475954 shutdown_connections 2024-09-05T11:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 --2- 172.21.15.59:0/2511475954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 0x7fc784114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 --2- 172.21.15.59:0/2511475954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074eb0 0x7fc784112650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 --2- 172.21.15.59:0/2511475954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074590 0x7fc784074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.290+0000 7fc78bc00640 1 -- 172.21.15.59:0/2511475954 >> 172.21.15.59:0/2511475954 conn(0x7fc7840fd8c0 msgr2=0x7fc7840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.291+0000 7fc78bc00640 1 -- 172.21.15.59:0/2511475954 shutdown_connections 2024-09-05T11:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.291+0000 7fc78bc00640 1 -- 172.21.15.59:0/2511475954 wait complete. 2024-09-05T11:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.293+0000 7fc78bc00640 1 Processor -- start 2024-09-05T11:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.293+0000 7fc78bc00640 1 -- start start 2024-09-05T11:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.294+0000 7fc78bc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074590 0x7fc7841a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.294+0000 7fc78bc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.294+0000 7fc78bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 0x7fc78419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc78bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc784118590 con 0x7fc784112b90 2024-09-05T11:57:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc78bc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc784118410 con 0x7fc784074eb0 2024-09-05T11:57:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc78bc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc784118710 con 0x7fc784074590 2024-09-05T11:57:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc789200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074590 0x7fc7841a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43860/0 (socket says 172.21.15.59:43860) 2024-09-05T11:57:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 -- 172.21.15.59:0/205277770 learned_addr learned my addr 172.21.15.59:0/205277770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc789c00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 0x7fc78419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 -- 172.21.15.59:0/205277770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074590 msgr2=0x7fc7841a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074590 0x7fc7841a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 -- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 msgr2=0x7fc78419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 0x7fc78419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.295+0000 7fc783e00640 1 -- 172.21.15.59:0/205277770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78419f010 con 0x7fc784074eb0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.296+0000 7fc783e00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc770009980 tx=0x7fc7700309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.296+0000 7fc781600640 1 -- 172.21.15.59:0/205277770 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770040070 con 0x7fc784074eb0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.296+0000 7fc781600640 1 -- 172.21.15.59:0/205277770 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770005620 con 0x7fc784074eb0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.296+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78419f2a0 con 0x7fc784074eb0 2024-09-05T11:57:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.297+0000 7fc781600640 1 -- 172.21.15.59:0/205277770 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770004920 con 0x7fc784074eb0 2024-09-05T11:57:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.297+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7841ab5f0 con 0x7fc784074eb0 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.301+0000 7fc78e5bd740 1 Processor -- start 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.301+0000 7fc781600640 1 -- 172.21.15.59:0/205277770 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc770043070 con 0x7fc784074eb0 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.302+0000 7fc78e5bd740 1 -- start start 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc73ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 0x55a29bc7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a29bbda290 con 0x55a29bc743e0 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a29bb81b40 con 0x55a29bc7eca0 2024-09-05T11:57:59.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a29bbd6250 con 0x55a29bc73ac0 2024-09-05T11:57:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:57:59.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43872/0 (socket says 172.21.15.59:43872) 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.303+0000 7fc783e00640 1 -- 172.21.15.59:0/1791662517 learned_addr learned my addr 172.21.15.59:0/1791662517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc781600640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 0x7fc76c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc789c00640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc73ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc781600640 1 -- 172.21.15.59:0/205277770 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc7700f0250 con 0x7fc784074eb0 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc783e00640 1 -- 172.21.15.59:0/1791662517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 msgr2=0x55a29bc73ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc783e00640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc73ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.304+0000 7fc783e00640 1 -- 172.21.15.59:0/1791662517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 msgr2=0x55a29bc7e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.305+0000 7fc789c00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 0x7fc76c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.305+0000 7fc789200640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 0x55a29bc7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.305+0000 7fc783e00640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 0x55a29bc7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.305+0000 7fc783e00640 1 -- 172.21.15.59:0/1791662517 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29b94b6a0 con 0x55a29bc7eca0 2024-09-05T11:57:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.306+0000 7fc789200640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 0x55a29bc7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.306+0000 7fc783e00640 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7700ed940 tx=0x7fc77002a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cb816f7010d755 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.306+0000 7fc776000640 1 -- 172.21.15.59:0/1791662517 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77004a070 con 0x55a29bc7eca0 2024-09-05T11:57:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.306+0000 7fc776000640 1 -- 172.21.15.59:0/1791662517 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7700f3940 con 0x55a29bc7eca0 2024-09-05T11:57:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.306+0000 7fc776000640 1 -- 172.21.15.59:0/1791662517 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700bc6a0 con 0x55a29bc7eca0 2024-09-05T11:57:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.307+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1791662517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 msgr2=0x55a29bc81090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.307+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7700ed940 tx=0x7fc77002a3d0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.307+0000 7fc789c00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 0x7fc76c0a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc784076600 tx=0x7fc77c009b10 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.309+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1791662517 shutdown_connections 2024-09-05T11:57:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.309+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc7eca0 0x55a29bc81090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.309+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc743e0 0x55a29bc7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.309+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1791662517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc73ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.309+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1791662517 >> 172.21.15.59:0/1791662517 conn(0x55a29bc70050 msgr2=0x55a29bc70430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.310+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1791662517 shutdown_connections 2024-09-05T11:57:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.310+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1791662517 wait complete. 2024-09-05T11:57:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.310+0000 7fc78e5bd740 1 -- start start 2024-09-05T11:57:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 Processor -- start 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bc7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bc7bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a29bc81630 con 0x55a29bc7eca0 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a29bb94fa0 con 0x55a29bc73ac0 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a29bc817b0 con 0x55a29bc743e0 2024-09-05T11:57:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.311+0000 7fc78e5bd740 10 client.0 _ll_register_callbacks cb 0x55a29b8e8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc789200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bc7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45548/0 (socket says 172.21.15.59:45548) 2024-09-05T11:57:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 -- 172.21.15.59:0/1860785951 learned_addr learned my addr 172.21.15.59:0/1860785951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc789c00640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bc7bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 -- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 msgr2=0x55a29bc7e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bc7e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 -- 172.21.15.59:0/1860785951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 msgr2=0x55a29bc7bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bc7bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.312+0000 7fc783e00640 1 -- 172.21.15.59:0/1860785951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29bc7c730 con 0x55a29bc743e0 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc783e00640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7700b8b00 tx=0x7fc7700bc470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba2d71a11253bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc76aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77004a040 con 0x55a29bc743e0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc770005140 con 0x55a29bc743e0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a29bbe7800 con 0x55a29bc743e0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7700bc000 con 0x55a29bc743e0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc78e5bd740 10 client.25303 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.313+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a29bc81630 con 0x55a29bc743e0 2024-09-05T11:57:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.314+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a29bbe7800 con 0x55a29bc743e0 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.314+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7700b8d10 con 0x55a29bc743e0 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 10 client.25303 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 10 client.25303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 20 client.25303 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bc9f5f0 0x55a29bca19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 4 client.25303 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.315+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a29bbe1d10 con 0x55a29bc9f5f0 2024-09-05T11:57:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.317+0000 7fc789200640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bc9f5f0 0x55a29bca19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.318+0000 7fc789200640 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bc9f5f0 0x55a29bca19e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a57dfbf3506ec0 server_cookie=7499d73bed9dab31 in_seq=0 out_seq=0 2024-09-05T11:57:59.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.318+0000 7fc76aa00640 10 client.25303 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.318+0000 7fc76aa00640 1 -- 172.21.15.59:0/1860785951 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a29bbe1d10 con 0x55a29bc9f5f0 2024-09-05T11:57:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.318+0000 7fc76aa00640 10 client.25303 handle_command_reply: tid=0 2024-09-05T11:57:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.318+0000 7fc78e5bd740 1 client.25303 shutdown 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 msgr2=0x55a29bc7b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7700b8b00 tx=0x7fc7700bc470 comp rx=0 tx=0).stop 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 shutdown_connections 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bc9f5f0 0x55a29bca19e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bc7bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc743e0 0x55a29bc7b910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1860785951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bc7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 >> 172.21.15.59:0/1860785951 conn(0x55a29bc70050 msgr2=0x55a29bc75d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.319+0000 7fc76aa00640 0 client.25303 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.320+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 shutdown_connections 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.320+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1860785951 wait complete. 2024-09-05T11:57:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.320+0000 7fc78e5bd740 20 client.25303 trim_cache size 0 max 16384 2024-09-05T11:57:59.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.356+0000 7fc78e5bd740 1 Processor -- start 2024-09-05T11:57:59.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 -- start start 2024-09-05T11:57:59.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bca7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bca7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410dcb0 con 0x55a29bc7eca0 2024-09-05T11:57:59.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ad720 con 0x55a29bc73ac0 2024-09-05T11:57:59.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.357+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410e320 con 0x55a29bc70050 2024-09-05T11:57:59.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bca7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45552/0 (socket says 172.21.15.59:45552) 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789200640 1 -- 172.21.15.59:0/3666326851 learned_addr learned my addr 172.21.15.59:0/3666326851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789c00640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bca7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 -- 172.21.15.59:0/3666326851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 msgr2=0x55a29bc6f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 -- 172.21.15.59:0/3666326851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 msgr2=0x55a29bca7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bca7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc783e00640 1 -- 172.21.15.59:0/3666326851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29bc7c270 con 0x55a29bc73ac0 2024-09-05T11:57:59.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789200640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:59.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.358+0000 7fc789c00640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bca7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.359+0000 7fc783e00640 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bca7120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55a29bc7ca00 tx=0x7fc7700c4f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce15dbb041bff4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.360+0000 7fc775600640 1 -- 172.21.15.59:0/3666326851 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77004a070 con 0x55a29bc73ac0 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.360+0000 7fc775600640 1 -- 172.21.15.59:0/3666326851 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7700d4070 con 0x55a29bc73ac0 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.360+0000 7fc775600640 1 -- 172.21.15.59:0/3666326851 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700f00b0 con 0x55a29bc73ac0 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 -- 172.21.15.59:0/3666326851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 msgr2=0x55a29bca7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bca7120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55a29bc7ca00 tx=0x7fc7700c4f20 comp rx=0 tx=0).stop 2024-09-05T11:57:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 -- 172.21.15.59:0/3666326851 shutdown_connections 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bca7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc73ac0 0x55a29bca7120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/3666326851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc70050 0x55a29bc6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 -- 172.21.15.59:0/3666326851 >> 172.21.15.59:0/3666326851 conn(0x55a29bc7dda0 msgr2=0x55a29bc76ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.361+0000 7fc78e5bd740 1 -- 172.21.15.59:0/3666326851 shutdown_connections 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.362+0000 7fc78e5bd740 1 -- 172.21.15.59:0/3666326851 wait complete. 2024-09-05T11:57:59.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.362+0000 7fc78e5bd740 1 -- start start 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.362+0000 7fc78e5bd740 1 Processor -- start 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bd090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841abbe0 con 0x55a29bc7eca0 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ad050 con 0x55a29bc70050 2024-09-05T11:57:59.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7841ac2b0 con 0x55a29bc73ac0 2024-09-05T11:57:59.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.363+0000 7fc78e5bd740 10 client.0 _ll_register_callbacks cb 0x7fc770004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:59.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.364+0000 7fc789200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.364+0000 7fc789200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43904/0 (socket says 172.21.15.59:43904) 2024-09-05T11:57:59.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.364+0000 7fc789200640 1 -- 172.21.15.59:0/1025024029 learned_addr learned my addr 172.21.15.59:0/1025024029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.364+0000 7fc789c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bd090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.364+0000 7fc783e00640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789200640 1 -- 172.21.15.59:0/1025024029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 msgr2=0x55a29bc83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789200640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789200640 1 -- 172.21.15.59:0/1025024029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 msgr2=0x55a29bd090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789200640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bd090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789200640 1 -- 172.21.15.59:0/1025024029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29bd1a670 con 0x55a29bc70050 2024-09-05T11:57:59.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc783e00640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:59.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.365+0000 7fc789c00640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bd090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:57:59.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.366+0000 7fc789200640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc778009c50 tx=0x7fc778005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332ac7d6aea13f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.366+0000 7fc76aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.366+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778009e60 con 0x55a29bc70050 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.366+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc778004020 con 0x55a29bc70050 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.367+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7700d4070 con 0x55a29bc70050 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.367+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc778004220 con 0x55a29bc70050 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.367+0000 7fc78e5bd740 10 client.25367 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:59.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.367+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a29bd1b9d0 con 0x55a29bc70050 2024-09-05T11:57:59.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.368+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7700d4070 con 0x55a29bc70050 2024-09-05T11:57:59.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.368+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc778010430 con 0x55a29bc70050 2024-09-05T11:57:59.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.368+0000 7fc78e5bd740 10 client.25367 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.368+0000 7fc78e5bd740 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.368+0000 7fc78e5bd740 20 client.25367 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.369+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bd1c100 0x55a29bd1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.369+0000 7fc78e5bd740 4 client.25367 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.369+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a29bb5dfb0 con 0x55a29bd1c100 2024-09-05T11:57:59.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.369+0000 7fc783e00640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bd1c100 0x55a29bd1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.371+0000 7fc783e00640 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bd1c100 0x55a29bd1e520 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8998b9b2ef1cc8 server_cookie=18cda4cf66c82d93 in_seq=0 out_seq=0 2024-09-05T11:57:59.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.371+0000 7fc76aa00640 10 client.25367 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.372+0000 7fc76aa00640 1 -- 172.21.15.59:0/1025024029 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a29bb5dfb0 con 0x55a29bd1c100 2024-09-05T11:57:59.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.372+0000 7fc76aa00640 10 client.25367 handle_command_reply: tid=0 2024-09-05T11:57:59.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.372+0000 7fc78e5bd740 1 client.25367 shutdown 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 msgr2=0x55a29bc83960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc778009c50 tx=0x7fc778005410 comp rx=0 tx=0).stop 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 shutdown_connections 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a29bd1c100 0x55a29bd1e520 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a29bc7eca0 0x55a29bd090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a29bc73ac0 0x55a29bc83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 --2- 172.21.15.59:0/1025024029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a29bc70050 0x55a29bc83960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 >> 172.21.15.59:0/1025024029 conn(0x55a29bc7dda0 msgr2=0x55a29bc7a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc76aa00640 0 client.25367 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 shutdown_connections 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 1 -- 172.21.15.59:0/1025024029 wait complete. 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.373+0000 7fc78e5bd740 20 client.25367 trim_cache size 0 max 16384 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:57:59.376 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "e8605844-7336-40f4-89be-ef5709400a30", 2024-09-05T11:57:59.377 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:57:59.377 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 msgr2=0x7fc76c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 0x7fc76c0a50c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc784076600 tx=0x7fc77c009b10 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 msgr2=0x7fc7841a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc770009980 tx=0x7fc7700309a0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 shutdown_connections 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc76c0a2c00 0x7fc76c0a50c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc784112b90 0x7fc78419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc784074eb0 0x7fc7841a4c60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 --2- 172.21.15.59:0/205277770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc784074590 0x7fc7841a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.375+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 >> 172.21.15.59:0/205277770 conn(0x7fc7840fd8c0 msgr2=0x7fc7840ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.376+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 shutdown_connections 2024-09-05T11:57:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.376+0000 7fc78bc00640 1 -- 172.21.15.59:0/205277770 wait complete. 2024-09-05T11:57:59.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 -- 172.21.15.59:0/122181097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc102360 msgr2=0x7fa0bc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/122181097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc102360 0x7fa0bc10e890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009cd0 tx=0x7fa0b402a8f0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 -- 172.21.15.59:0/122181097 shutdown_connections 2024-09-05T11:57:59.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/122181097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc102360 0x7fa0bc10e890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/122181097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/122181097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 0x7fa0bc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.969+0000 7fa0bbe00640 1 -- 172.21.15.59:0/122181097 >> 172.21.15.59:0/122181097 conn(0x7fa0bc0fd740 msgr2=0x7fa0bc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.970+0000 7fa0bbe00640 1 -- 172.21.15.59:0/122181097 shutdown_connections 2024-09-05T11:57:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.970+0000 7fa0bbe00640 1 -- 172.21.15.59:0/122181097 wait complete. 2024-09-05T11:57:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 Processor -- start 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 -- start start 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 0x7fa0bc1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 0x7fa0bc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10fc00 con 0x7fa0bc1019c0 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10fa80 con 0x7fa0bc102360 2024-09-05T11:57:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.971+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc10fd80 con 0x7fa0bc1079c0 2024-09-05T11:57:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.972+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.972+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49106/0 (socket says 172.21.15.59:49106) 2024-09-05T11:57:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.972+0000 7fa0baa00640 1 -- 172.21.15.59:0/2263312373 learned_addr learned my addr 172.21.15.59:0/2263312373 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.972+0000 7fa0bb400640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 0x7fa0bc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.972+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 0x7fa0bc1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 -- 172.21.15.59:0/2263312373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 msgr2=0x7fa0bc19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 0x7fa0bc19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 -- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 msgr2=0x7fa0bc1a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 -- 172.21.15.59:0/2263312373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc19ee40 con 0x7fa0bc102360 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0baa00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.973+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 0x7fa0bc1a4af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800bb20 tx=0x7fa0a80091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.974+0000 7fa0c0a00640 1 -- 172.21.15.59:0/2263312373 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8009af0 con 0x7fa0bc102360 2024-09-05T11:57:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.974+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc19f130 con 0x7fa0bc102360 2024-09-05T11:57:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.974+0000 7fa0c0a00640 1 -- 172.21.15.59:0/2263312373 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8009c90 con 0x7fa0bc102360 2024-09-05T11:57:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.974+0000 7fa0c0a00640 1 -- 172.21.15.59:0/2263312373 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8015620 con 0x7fa0bc102360 2024-09-05T11:57:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.974+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc195da0 con 0x7fa0bc102360 2024-09-05T11:57:59.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.977+0000 7fa0c0a00640 1 -- 172.21.15.59:0/2263312373 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0a8016420 con 0x7fa0bc102360 2024-09-05T11:57:59.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 Processor -- start 2024-09-05T11:57:59.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 -- start start 2024-09-05T11:57:59.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c0a00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 0x7fa0a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c0a00640 1 -- 172.21.15.59:0/2263312373 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa0a80cf210 con 0x7fa0bc102360 2024-09-05T11:57:59.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560dda04a650 con 0x560dda0ef700 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560dd9ff1f10 con 0x560dda0e4520 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.979+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560dda046610 con 0x560dda0e4e40 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.980+0000 7fa0ba000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.980+0000 7fa0ba000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45580/0 (socket says 172.21.15.59:45580) 2024-09-05T11:57:59.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.980+0000 7fa0ba000640 1 -- 172.21.15.59:0/2295642934 learned_addr learned my addr 172.21.15.59:0/2295642934 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.980+0000 7fa0bb400640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0bb400640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 0x7fa0a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0ba000640 1 -- 172.21.15.59:0/2295642934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 msgr2=0x560dda0e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0baa00640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0ba000640 1 -- 172.21.15.59:0/2295642934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 msgr2=0x560dda0f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0ba000640 1 -- 172.21.15.59:0/2295642934 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd9d59580 con 0x560dda0e4e40 2024-09-05T11:57:59.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0baa00640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:59.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.981+0000 7fa0bb400640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:57:59.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.982+0000 7fa0ba000640 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80a3d40 tx=0x7fa0a809b140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a768130fa5d929e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.982+0000 7fa0a7e00640 1 -- 172.21.15.59:0/2295642934 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8097040 con 0x560dda0e4e40 2024-09-05T11:57:59.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.982+0000 7fa0bb400640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 0x7fa0a00a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4004af0 tx=0x7fa0b4005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.982+0000 7fa0a7e00640 1 -- 172.21.15.59:0/2295642934 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8009e40 con 0x560dda0e4e40 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.982+0000 7fa0a7e00640 1 -- 172.21.15.59:0/2295642934 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80999d0 con 0x560dda0e4e40 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 -- 172.21.15.59:0/2295642934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 msgr2=0x560dda0ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80a3d40 tx=0x7fa0a809b140 comp rx=0 tx=0).stop 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 -- 172.21.15.59:0/2295642934 shutdown_connections 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0ef1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/2295642934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 -- 172.21.15.59:0/2295642934 >> 172.21.15.59:0/2295642934 conn(0x560dda0e0ab0 msgr2=0x560dda0e0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.983+0000 7fa0c48a6740 1 -- 172.21.15.59:0/2295642934 shutdown_connections 2024-09-05T11:57:59.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.984+0000 7fa0c48a6740 1 -- 172.21.15.59:0/2295642934 wait complete. 2024-09-05T11:57:59.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.984+0000 7fa0c48a6740 1 -- start start 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.984+0000 7fa0c48a6740 1 Processor -- start 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 0x560dda0ecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda0ed130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560dda0f2270 con 0x560dda0e4e40 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560dda005350 con 0x560dda0e4520 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560dda0f23f0 con 0x560dda0ef700 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0c48a6740 10 client.0 _ll_register_callbacks cb 0x560dda028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43942/0 (socket says 172.21.15.59:43942) 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.985+0000 7fa0baa00640 1 -- 172.21.15.59:0/3126181216 learned_addr learned my addr 172.21.15.59:0/3126181216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:57:59.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.986+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 0x560dda0ecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.986+0000 7fa0bb400640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda0ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.986+0000 7fa0baa00640 1 -- 172.21.15.59:0/3126181216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 msgr2=0x560dda0ed130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0baa00640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda0ed130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0baa00640 1 -- 172.21.15.59:0/3126181216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 msgr2=0x560dda0ecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0baa00640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 0x560dda0ecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0baa00640 1 -- 172.21.15.59:0/3126181216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd9e23960 con 0x560dda0e4520 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 0x560dda0ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0baa00640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac013d30 tx=0x7fa0ac0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe65bd7d999ed10 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.987+0000 7fa0a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac020060 con 0x560dda0e4520 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560dda057bc0 con 0x560dda0e4520 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa0ac01b630 con 0x560dda0e4520 2024-09-05T11:57:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa0ac027050 con 0x560dda0e4520 2024-09-05T11:57:59.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0c48a6740 10 client.25376 fetch_fsmap learned FSMap version 21 2024-09-05T11:57:59.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560dda0fdf70 con 0x560dda0e4520 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.988+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560dda057bc0 con 0x560dda0e4520 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa0ac00e680 con 0x560dda0e4520 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 10 client.25376 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 10 client.25376 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 20 client.25376 populate_metadata read hostname 'smithi059' 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda0ff380 0x560dda1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 4 client.25376 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:57:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.989+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dda0520d0 con 0x560dda0ff380 2024-09-05T11:57:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.990+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda0ff380 0x560dda1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:57:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.990+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda0ff380 0x560dda1017d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62aa3d6922a428cd server_cookie=f245436f798b5760 in_seq=0 out_seq=0 2024-09-05T11:57:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.990+0000 7fa0a4c00640 10 client.25376 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.991+0000 7fa0a4c00640 1 -- 172.21.15.59:0/3126181216 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dda0520d0 con 0x560dda0ff380 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.991+0000 7fa0a4c00640 10 client.25376 handle_command_reply: tid=0 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.991+0000 7fa0c48a6740 1 client.25376 shutdown 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 msgr2=0x560dda0ec6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac013d30 tx=0x7fa0ac0117b0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 shutdown_connections 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda0ed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda0ff380 0x560dda1017d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4e40 0x560dda0ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0a4c00640 0 client.25376 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:57:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3126181216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0ec6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:57:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.992+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 >> 172.21.15.59:0/3126181216 conn(0x560dda0e0ab0 msgr2=0x560dda0dfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:57:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.993+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 shutdown_connections 2024-09-05T11:57:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.993+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3126181216 wait complete. 2024-09-05T11:57:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:57:59.993+0000 7fa0c48a6740 20 client.25376 trim_cache size 0 max 16384 2024-09-05T11:58:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.011+0000 7fa0c48a6740 1 Processor -- start 2024-09-05T11:58:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.011+0000 7fa0c48a6740 1 -- start start 2024-09-05T11:58:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 0x560dda0e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda1070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc101e20 con 0x560dda0e4520 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1be420 con 0x560dda0e4e40 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1bc370 con 0x560dda0ef700 2024-09-05T11:58:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0ba000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0ba000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43944/0 (socket says 172.21.15.59:43944) 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0bb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda1070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0ba000640 1 -- 172.21.15.59:0/3530639695 learned_addr learned my addr 172.21.15.59:0/3530639695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.012+0000 7fa0baa00640 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 0x560dda0e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0ba000640 1 -- 172.21.15.59:0/3530639695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 msgr2=0x560dda1070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda1070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0ba000640 1 -- 172.21.15.59:0/3530639695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 msgr2=0x560dda0e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 0x560dda0e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0ba000640 1 -- 172.21.15.59:0/3530639695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dda0e74c0 con 0x560dda0e4e40 2024-09-05T11:58:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.013+0000 7fa0baa00640 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 0x560dda0e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.014+0000 7fa0ba000640 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa0a809b300 tx=0x7fa0a800d560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556d0d07ff323a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.014+0000 7fa0a7400640 1 -- 172.21.15.59:0/3530639695 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8097040 con 0x560dda0e4e40 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.014+0000 7fa0a7400640 1 -- 172.21.15.59:0/3530639695 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a809bd00 con 0x560dda0e4e40 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.014+0000 7fa0a7400640 1 -- 172.21.15.59:0/3530639695 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8084df0 con 0x560dda0e4e40 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3530639695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 msgr2=0x560dda0ee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa0a809b300 tx=0x7fa0a800d560 comp rx=0 tx=0).stop 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3530639695 shutdown_connections 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0ef700 0x560dda1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4e40 0x560dda0ee510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/3530639695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0e4520 0x560dda0e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3530639695 >> 172.21.15.59:0/3530639695 conn(0x560dda0e0ab0 msgr2=0x560dda0ede00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3530639695 shutdown_connections 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- 172.21.15.59:0/3530639695 wait complete. 2024-09-05T11:58:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.015+0000 7fa0c48a6740 1 -- start start 2024-09-05T11:58:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 Processor -- start 2024-09-05T11:58:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1bc8e0 con 0x560dda0ef700 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1bdd50 con 0x560dda0e4520 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1bcfb0 con 0x560dda0e4e40 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0c48a6740 10 client.0 _ll_register_callbacks cb 0x7fa0ac009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43946/0 (socket says 172.21.15.59:43946) 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.016+0000 7fa0baa00640 1 -- 172.21.15.59:0/4211148891 learned_addr learned my addr 172.21.15.59:0/4211148891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 -- 172.21.15.59:0/4211148891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 msgr2=0x560dda0f3810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0f3810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 -- 172.21.15.59:0/4211148891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 msgr2=0x560dda0f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 -- 172.21.15.59:0/4211148891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dda169070 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0baa00640 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ead0 tx=0x7fa0ac005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6b7c5235b1a03c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00e5c0 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a809bd00 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.017+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa0ac00e760 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.018+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa0ac039050 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.018+0000 7fa0c48a6740 10 client.25382 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.018+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560dda16a2d0 con 0x560dda0e4520 2024-09-05T11:58:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.018+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a809bd00 con 0x560dda0e4520 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa0ac003c30 con 0x560dda0e4520 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 10 client.25382 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 10 client.25382 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 20 client.25382 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda16a9b0 0x560dda16ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 4 client.25382 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "526b5f58", "release": true, "await": true}] 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "526b5f58", "release": true, "await": true}) -- 0x560dd9fce430 con 0x560dda16a9b0 2024-09-05T11:58:00.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.019+0000 7fa0ba000640 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda16a9b0 0x560dda16ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.020+0000 7fa0ba000640 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda16a9b0 0x560dda16ce30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e5247f83ed21b5 server_cookie=75bb62f0ebcdadac in_seq=0 out_seq=0 2024-09-05T11:58:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.020+0000 7fa0a4c00640 10 client.25382 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:00.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0a4c00640 1 -- 172.21.15.59:0/4211148891 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560dd9fce430 con 0x560dda16a9b0 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0a4c00640 10 client.25382 handle_command_reply: tid=0 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 client.25382 shutdown 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 msgr2=0x560dda0f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ead0 tx=0x7fa0ac005d50 comp rx=0 tx=0).stop 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 shutdown_connections 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560dda16a9b0 0x560dda16ce30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560dda0ef700 0x560dda0f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dda0e4e40 0x560dda0f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 --2- 172.21.15.59:0/4211148891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dda0e4520 0x560dda0f32d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 >> 172.21.15.59:0/4211148891 conn(0x560dda0e0ab0 msgr2=0x560dda0ee510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:00.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.021+0000 7fa0a4c00640 0 client.25382 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.022+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 shutdown_connections 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.022+0000 7fa0c48a6740 1 -- 172.21.15.59:0/4211148891 wait complete. 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.022+0000 7fa0c48a6740 20 client.25382 trim_cache size 0 max 16384 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 4, 2024-09-05T11:58:00.025 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "526b5f58": { 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4, 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 138.0, 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:58:00.026 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:00.027 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 msgr2=0x7fa0a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 0x7fa0a00a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4004af0 tx=0x7fa0b4005d80 comp rx=0 tx=0).stop 2024-09-05T11:58:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 msgr2=0x7fa0bc1a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 0x7fa0bc1a4af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800bb20 tx=0x7fa0a80091e0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 shutdown_connections 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0a00a2b80 0x7fa0a00a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0bc1079c0 0x7fa0bc19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0bc102360 0x7fa0bc1a4af0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800bb20 tx=0x7fa0a80091e0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 --2- 172.21.15.59:0/2263312373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc1019c0 0x7fa0bc1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 >> 172.21.15.59:0/2263312373 conn(0x7fa0bc0fd740 msgr2=0x7fa0bc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 shutdown_connections 2024-09-05T11:58:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:00.025+0000 7fa0bbe00640 1 -- 172.21.15.59:0/2263312373 wait complete. 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: pgmap v404: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:00 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:58:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: pgmap v404: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:58:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:58:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:00 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: pgmap v404: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:00.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:00 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:58:00.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:58:01.426 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '526b5f58', total seconds elapsed: 23.7 2024-09-05T11:58:01.489 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.181+0000 7f6414c00640 1 -- 172.21.15.59:0/760217366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64101049a0 msgr2=0x7f64101089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.181+0000 7f6414c00640 1 --2- 172.21.15.59:0/760217366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64101049a0 0x7f64101089f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 -- 172.21.15.59:0/760217366 shutdown_connections 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 --2- 172.21.15.59:0/760217366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f6410110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 --2- 172.21.15.59:0/760217366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64101049a0 0x7f64101089f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 --2- 172.21.15.59:0/760217366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6410103ff0 0x7f64101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 -- 172.21.15.59:0/760217366 >> 172.21.15.59:0/760217366 conn(0x7f64100ffa90 msgr2=0x7f6410101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 -- 172.21.15.59:0/760217366 shutdown_connections 2024-09-05T11:58:02.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.182+0000 7f6414c00640 1 -- 172.21.15.59:0/760217366 wait complete. 2024-09-05T11:58:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.183+0000 7f6414c00640 1 Processor -- start 2024-09-05T11:58:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 -- start start 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 0x7f641019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64101049a0 0x7f641019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6410114190 con 0x7f6410103ff0 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6410114010 con 0x7f64101049a0 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.184+0000 7f6414c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6410114310 con 0x7f6410109120 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.185+0000 7f640e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.185+0000 7f640e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44052/0 (socket says 172.21.15.59:44052) 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.185+0000 7f640e800640 1 -- 172.21.15.59:0/40982966 learned_addr learned my addr 172.21.15.59:0/40982966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.185+0000 7f640de00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 0x7f641019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.185+0000 7f640d400640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64101049a0 0x7f641019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 -- 172.21.15.59:0/40982966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64101049a0 msgr2=0x7f641019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64101049a0 0x7f641019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 -- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 msgr2=0x7f641019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 0x7f641019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 -- 172.21.15.59:0/40982966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641019f5a0 con 0x7f6410109120 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640de00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 0x7f641019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f640e800640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f640800eb10 tx=0x7f640800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.186+0000 7f6406a00640 1 -- 172.21.15.59:0/40982966 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408004b20 con 0x7f6410109120 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.187+0000 7f6406a00640 1 -- 172.21.15.59:0/40982966 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6408004cc0 con 0x7f6410109120 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.187+0000 7f6406a00640 1 -- 172.21.15.59:0/40982966 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408010760 con 0x7f6410109120 2024-09-05T11:58:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.187+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641019f890 con 0x7f6410109120 2024-09-05T11:58:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.187+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a7170 con 0x7f6410109120 2024-09-05T11:58:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.190+0000 7f6417675740 1 Processor -- start 2024-09-05T11:58:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.192+0000 7f6406a00640 1 -- 172.21.15.59:0/40982966 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6408005020 con 0x7f6410109120 2024-09-05T11:58:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.192+0000 7f6417675740 1 -- start start 2024-09-05T11:58:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3931f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca393c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dca38983e0 con 0x55dca393cd70 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dca383fd80 con 0x55dca3931b90 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f6417675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dca38943a0 con 0x55dca39324b0 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f640d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f640de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca393c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.193+0000 7f640d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60278/0 (socket says 172.21.15.59:60278) 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 -- 172.21.15.59:0/3723145132 learned_addr learned my addr 172.21.15.59:0/3723145132 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f6406a00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 0x7f63f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f6406a00640 1 -- 172.21.15.59:0/40982966 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f64080cfa00 con 0x7f6410109120 2024-09-05T11:58:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 -- 172.21.15.59:0/3723145132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 msgr2=0x55dca393c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca393c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 -- 172.21.15.59:0/3723145132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 msgr2=0x55dca3931f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3931f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.194+0000 7f640d400640 1 -- 172.21.15.59:0/3723145132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca385eae0 con 0x55dca393cd70 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.195+0000 7f640e800640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 0x7f63f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.195+0000 7f640d400640 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ea275dbbd7e923 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f63fb000640 1 -- 172.21.15.59:0/3723145132 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc035040 con 0x55dca393cd70 2024-09-05T11:58:02.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f63fb000640 1 -- 172.21.15.59:0/3723145132 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc005520 con 0x55dca393cd70 2024-09-05T11:58:02.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f63fb000640 1 -- 172.21.15.59:0/3723145132 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0035f0 con 0x55dca393cd70 2024-09-05T11:58:02.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f6417675740 1 -- 172.21.15.59:0/3723145132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 msgr2=0x55dca393f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f640e800640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 0x7f63f40a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6408098820 tx=0x7f64080058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.196+0000 7f6417675740 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc005de0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 -- 172.21.15.59:0/3723145132 shutdown_connections 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca393cd70 0x55dca393f160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca393c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 --2- 172.21.15.59:0/3723145132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3931f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 -- 172.21.15.59:0/3723145132 >> 172.21.15.59:0/3723145132 conn(0x55dca392e120 msgr2=0x55dca392e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 -- 172.21.15.59:0/3723145132 shutdown_connections 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 -- 172.21.15.59:0/3723145132 wait complete. 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.197+0000 7f6417675740 1 -- start start 2024-09-05T11:58:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 Processor -- start 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3939db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca393a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dca393f980 con 0x55dca39324b0 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dca38530f0 con 0x55dca3931b90 2024-09-05T11:58:02.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dca393fb00 con 0x55dca393cd70 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.198+0000 7f6417675740 10 client.0 _ll_register_callbacks cb 0x55dca38449e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44080/0 (socket says 172.21.15.59:44080) 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 -- 172.21.15.59:0/2572831901 learned_addr learned my addr 172.21.15.59:0/2572831901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3939db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 -- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 msgr2=0x55dca3939db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640de00640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca393a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3939db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 -- 172.21.15.59:0/2572831901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 msgr2=0x55dca393a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca393a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.199+0000 7f640d400640 1 -- 172.21.15.59:0/2572831901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca36093c0 con 0x55dca393cd70 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f640de00640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca393a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f640d400640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63fc004cc0 tx=0x7f63fc004b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3728049eebef6ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f63ef400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc035070 con 0x55dca393cd70 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc005520 con 0x55dca393cd70 2024-09-05T11:58:02.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.200+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f63fc005100 con 0x55dca393cd70 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.201+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f63fc059050 con 0x55dca393cd70 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.201+0000 7f6417675740 10 client.25336 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.201+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dca393f980 con 0x55dca393cd70 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.201+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc005520 con 0x55dca393cd70 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f63fc051020 con 0x55dca393cd70 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 10 client.25336 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 10 client.25336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:02.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 20 client.25336 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca3955100 0x55dca39574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 4 client.25336 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.202+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dca38a5950 con 0x55dca3955100 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.203+0000 7f640de00640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca3955100 0x55dca39574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.203+0000 7f63ef400640 10 client.25336 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.203+0000 7f640de00640 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca3955100 0x55dca39574f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd164733cee3a44d server_cookie=1758758103b353e3 in_seq=0 out_seq=0 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.204+0000 7f63ef400640 1 -- 172.21.15.59:0/2572831901 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dca38a5950 con 0x55dca3955100 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.204+0000 7f63ef400640 10 client.25336 handle_command_reply: tid=0 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.204+0000 7f6417675740 1 client.25336 shutdown 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.204+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 msgr2=0x55dca393a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63fc004cc0 tx=0x7f63fc004b20 comp rx=0 tx=0).stop 2024-09-05T11:58:02.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 shutdown_connections 2024-09-05T11:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca393a830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca3955100 0x55dca39574f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca393a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 --2- 172.21.15.59:0/2572831901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca3939db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 >> 172.21.15.59:0/2572831901 conn(0x55dca392e120 msgr2=0x55dca3933e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f63ef400640 0 client.25336 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:02.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 shutdown_connections 2024-09-05T11:58:02.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 1 -- 172.21.15.59:0/2572831901 wait complete. 2024-09-05T11:58:02.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.205+0000 7f6417675740 20 client.25336 trim_cache size 0 max 16384 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.242+0000 7f6417675740 1 Processor -- start 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.242+0000 7f6417675740 1 -- start start 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.242+0000 7f6417675740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.242+0000 7f6417675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca395cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.242+0000 7f6417675740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca393cd70 0x55dca395d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f6417675740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64101aace0 con 0x55dca3931b90 2024-09-05T11:58:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f6417675740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ad560 con 0x55dca393cd70 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f6417675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ab350 con 0x55dca39324b0 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60302/0 (socket says 172.21.15.59:60302) 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 -- 172.21.15.59:0/996265455 learned_addr learned my addr 172.21.15.59:0/996265455 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640de00640 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca395cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 -- 172.21.15.59:0/996265455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 msgr2=0x55dca395cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca395cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 -- 172.21.15.59:0/996265455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca393cd70 msgr2=0x55dca395d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca393cd70 0x55dca395d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.243+0000 7f640d400640 1 -- 172.21.15.59:0/996265455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca3938300 con 0x55dca3931b90 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.244+0000 7f640d400640 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0035f0 tx=0x7f63fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93079224dffb900e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.244+0000 7f63fa600640 1 -- 172.21.15.59:0/996265455 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc048070 con 0x55dca3931b90 2024-09-05T11:58:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.244+0000 7f63fa600640 1 -- 172.21.15.59:0/996265455 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc0057f0 con 0x55dca3931b90 2024-09-05T11:58:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.244+0000 7f63fa600640 1 -- 172.21.15.59:0/996265455 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc035040 con 0x55dca3931b90 2024-09-05T11:58:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.245+0000 7f6417675740 1 -- 172.21.15.59:0/996265455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 msgr2=0x55dca393a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.245+0000 7f6417675740 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0035f0 tx=0x7f63fc005fd0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 -- 172.21.15.59:0/996265455 shutdown_connections 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca393cd70 0x55dca395d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca39324b0 0x55dca395cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 --2- 172.21.15.59:0/996265455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca3931b90 0x55dca393a680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 -- 172.21.15.59:0/996265455 >> 172.21.15.59:0/996265455 conn(0x55dca392e120 msgr2=0x55dca393be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 -- 172.21.15.59:0/996265455 shutdown_connections 2024-09-05T11:58:02.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 -- 172.21.15.59:0/996265455 wait complete. 2024-09-05T11:58:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.246+0000 7f6417675740 1 -- start start 2024-09-05T11:58:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.247+0000 7f6417675740 1 Processor -- start 2024-09-05T11:58:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.247+0000 7f6417675740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca394db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.247+0000 7f6417675740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.247+0000 7f6417675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca39493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f6417675740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64101aba20 con 0x55dca39324b0 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f6417675740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ace90 con 0x55dca3931b90 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f6417675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ac0f0 con 0x55dca393cd70 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f6417675740 10 client.0 _ll_register_callbacks cb 0x7f63fc004df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60318/0 (socket says 172.21.15.59:60318) 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 -- 172.21.15.59:0/3973361032 learned_addr learned my addr 172.21.15.59:0/3973361032 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640d400640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca394db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 -- 172.21.15.59:0/3973361032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 msgr2=0x55dca39493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca39493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 -- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 msgr2=0x55dca394db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca394db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640de00640 1 -- 172.21.15.59:0/3973361032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca3949a10 con 0x55dca39324b0 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.248+0000 7f640d400640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca394db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.249+0000 7f640de00640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400010920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cce5052e6f21ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.249+0000 7f63ef400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:02.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.249+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640002a060 con 0x55dca39324b0 2024-09-05T11:58:02.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.249+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc0057f0 con 0x55dca39324b0 2024-09-05T11:58:02.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.250+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f640001e9a0 con 0x55dca39324b0 2024-09-05T11:58:02.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.250+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6400031050 con 0x55dca39324b0 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.250+0000 7f6417675740 10 client.15390 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.250+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dca39bfa10 con 0x55dca39324b0 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.250+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc0057f0 con 0x55dca39324b0 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f640000f3c0 con 0x55dca39324b0 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 10 client.15390 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 20 client.15390 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca39c0170 0x55dca39c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 4 client.15390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:58:02.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dca389fe60 con 0x55dca39c0170 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.251+0000 7f640d400640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca39c0170 0x55dca39c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.252+0000 7f63ef400640 10 client.15390 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.252+0000 7f640d400640 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca39c0170 0x55dca39c2590 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b853441fe413bbc server_cookie=9a24f4e07800140d in_seq=0 out_seq=0 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.252+0000 7f63ef400640 1 -- 172.21.15.59:0/3973361032 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dca389fe60 con 0x55dca39c0170 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.252+0000 7f63ef400640 10 client.15390 handle_command_reply: tid=0 2024-09-05T11:58:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.252+0000 7f6417675740 1 client.15390 shutdown 2024-09-05T11:58:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 msgr2=0x55dca3948e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6400000c00 tx=0x7f6400010920 comp rx=0 tx=0).stop 2024-09-05T11:58:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 shutdown_connections 2024-09-05T11:58:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca393cd70 0x55dca39493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dca39c0170 0x55dca39c2590 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dca39324b0 0x55dca3948e90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 --2- 172.21.15.59:0/3973361032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dca3931b90 0x55dca394db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 >> 172.21.15.59:0/3973361032 conn(0x55dca392e120 msgr2=0x55dca3938550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.253+0000 7f63ef400640 0 client.15390 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.254+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 shutdown_connections 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.254+0000 7f6417675740 1 -- 172.21.15.59:0/3973361032 wait complete. 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.254+0000 7f6417675740 20 client.15390 trim_cache size 0 max 16384 2024-09-05T11:58:02.256 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 msgr2=0x7f63f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 0x7f63f40a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6408098820 tx=0x7f64080058d0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 msgr2=0x7f641019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:02.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f640800eb10 tx=0x7f640800c2d0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 shutdown_connections 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63f40a2c00 0x7f63f40a50c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6410109120 0x7f641019eec0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64101049a0 0x7f641019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 --2- 172.21.15.59:0/40982966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6410103ff0 0x7f641019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.255+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 >> 172.21.15.59:0/40982966 conn(0x7f64100ffa90 msgr2=0x7f6410101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.256+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 shutdown_connections 2024-09-05T11:58:02.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:02.256+0000 7f6414c00640 1 -- 172.21.15.59:0/40982966 wait complete. 2024-09-05T11:58:02.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:02 smithi059 ceph-mon[30025]: pgmap v405: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:02.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:02 smithi089 ceph-mon[34499]: pgmap v405: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:02.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:02 smithi189 ceph-mon[34772]: pgmap v405: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:03.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:58:03.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:03.613 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 -- 172.21.15.59:0/1415157589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a141047e0 msgr2=0x7f6a14108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 --2- 172.21.15.59:0/1415157589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a141047e0 0x7f6a14108830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009900 tx=0x7f6a0002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 -- 172.21.15.59:0/1415157589 shutdown_connections 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 --2- 172.21.15.59:0/1415157589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a14110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 --2- 172.21.15.59:0/1415157589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a141047e0 0x7f6a14108830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 --2- 172.21.15.59:0/1415157589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a14103e30 0x7f6a14104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 -- 172.21.15.59:0/1415157589 >> 172.21.15.59:0/1415157589 conn(0x7f6a140ff910 msgr2=0x7f6a14101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 -- 172.21.15.59:0/1415157589 shutdown_connections 2024-09-05T11:58:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.253+0000 7f6a1a400640 1 -- 172.21.15.59:0/1415157589 wait complete. 2024-09-05T11:58:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.254+0000 7f6a1a400640 1 Processor -- start 2024-09-05T11:58:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 -- start start 2024-09-05T11:58:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 0x7f6a1419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a141047e0 0x7f6a1419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a1419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14113fd0 con 0x7f6a14103e30 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14113e50 con 0x7f6a14108f60 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.255+0000 7f6a1a400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14114150 con 0x7f6a141047e0 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 0x7f6a1419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a1419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a1419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50760/0 (socket says 172.21.15.59:50760) 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19a00640 1 -- 172.21.15.59:0/502770914 learned_addr learned my addr 172.21.15.59:0/502770914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a13e00640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a141047e0 0x7f6a1419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 -- 172.21.15.59:0/502770914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a141047e0 msgr2=0x7f6a1419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a141047e0 0x7f6a1419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 -- 172.21.15.59:0/502770914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 msgr2=0x7f6a1419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a1419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.256+0000 7f6a19000640 1 -- 172.21.15.59:0/502770914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1419f440 con 0x7f6a14103e30 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a19000640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 0x7f6a1419a490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800e9f0 tx=0x7f6a0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800ccc0 con 0x7f6a14103e30 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a08010910 con 0x7f6a14103e30 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a080189c0 con 0x7f6a14103e30 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1419f730 con 0x7f6a14103e30 2024-09-05T11:58:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.257+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141a7010 con 0x7f6a14103e30 2024-09-05T11:58:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.259+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a14105310 con 0x7f6a14103e30 2024-09-05T11:58:04.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.261+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a08010ac0 con 0x7f6a14103e30 2024-09-05T11:58:04.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.263+0000 7f6a11600640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 0x7f69fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:04.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.263+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a080cfc80 con 0x7f6a14103e30 2024-09-05T11:58:04.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.265+0000 7f6a13e00640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 0x7f69fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:04.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.268+0000 7f6a13e00640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 0x7f69fc0a50c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a00005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:04.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.269+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a0809c120 con 0x7f6a14103e30 2024-09-05T11:58:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:04 smithi089 ceph-mon[34499]: pgmap v406: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:04.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:04 smithi059 ceph-mon[30025]: pgmap v406: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.497+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a14104210 con 0x7f6a14103e30 2024-09-05T11:58:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.498+0000 7f6a11600640 1 -- 172.21.15.59:0/502770914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f6a080a0420 con 0x7f6a14103e30 2024-09-05T11:58:04.500 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:58:04.501 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:58:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.500+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 msgr2=0x7f69fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.500+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 0x7f69fc0a50c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a00000c00 tx=0x7f6a00005b50 comp rx=0 tx=0).stop 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.500+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 msgr2=0x7f6a1419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.500+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 0x7f6a1419a490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800e9f0 tx=0x7f6a0800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 shutdown_connections 2024-09-05T11:58:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69fc0a2c00 0x7f69fc0a50c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a14108f60 0x7f6a1419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a141047e0 0x7f6a1419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 --2- 172.21.15.59:0/502770914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a14103e30 0x7f6a1419a490 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800e9f0 tx=0x7f6a0800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:58:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 >> 172.21.15.59:0/502770914 conn(0x7f6a140ff910 msgr2=0x7f6a14100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 shutdown_connections 2024-09-05T11:58:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:04.501+0000 7f6a1a400640 1 -- 172.21.15.59:0/502770914 wait complete. 2024-09-05T11:58:04.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:04 smithi189 ceph-mon[34772]: pgmap v406: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-05T11:58:05.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:58:05.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:05 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/502770914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:05.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:05 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/502770914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:05.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:05 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/502770914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:05.636 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:06.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 -- 172.21.15.59:0/4288605192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 --2- 172.21.15.59:0/4288605192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01019c0 0x7f1da0101e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009930 tx=0x7f1d8c02a420 comp rx=0 tx=0).stop 2024-09-05T11:58:06.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 -- 172.21.15.59:0/4288605192 shutdown_connections 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 --2- 172.21.15.59:0/4288605192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da0102360 0x7f1da010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 --2- 172.21.15.59:0/4288605192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01019c0 0x7f1da0101e20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 --2- 172.21.15.59:0/4288605192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da01079c0 0x7f1da0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.234+0000 7f1da5400640 1 -- 172.21.15.59:0/4288605192 >> 172.21.15.59:0/4288605192 conn(0x7f1da00fd740 msgr2=0x7f1da00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.235+0000 7f1da5400640 1 -- 172.21.15.59:0/4288605192 shutdown_connections 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.235+0000 7f1da5400640 1 -- 172.21.15.59:0/4288605192 wait complete. 2024-09-05T11:58:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.236+0000 7f1da5400640 1 Processor -- start 2024-09-05T11:58:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.236+0000 7f1da5400640 1 -- start start 2024-09-05T11:58:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111e10 con 0x7f1da01079c0 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111c90 con 0x7f1da01019c0 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1da5400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111f90 con 0x7f1da0102360 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50784/0 (socket says 172.21.15.59:50784) 2024-09-05T11:58:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1d9fe00640 1 -- 172.21.15.59:0/1077643812 learned_addr learned my addr 172.21.15.59:0/1077643812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.237+0000 7f1d9f400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1da4a00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 -- 172.21.15.59:0/1077643812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102360 msgr2=0x7f1da0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 -- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 msgr2=0x7f1da019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 -- 172.21.15.59:0/1077643812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019d2a0 con 0x7f1da01019c0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1da4a00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.238+0000 7f1d9cc00640 1 -- 172.21.15.59:0/1077643812 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000cd80 con 0x7f1da01019c0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.239+0000 7f1d9cc00640 1 -- 172.21.15.59:0/1077643812 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d90005500 con 0x7f1da01019c0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.239+0000 7f1d9cc00640 1 -- 172.21.15.59:0/1077643812 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90010690 con 0x7f1da01019c0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.239+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019d590 con 0x7f1da01019c0 2024-09-05T11:58:06.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.239+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da0074420 con 0x7f1da01019c0 2024-09-05T11:58:06.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.243+0000 7f1da95ef740 1 Processor -- start 2024-09-05T11:58:06.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.243+0000 7f1d9cc00640 1 -- 172.21.15.59:0/1077643812 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d900046c0 con 0x7f1da01019c0 2024-09-05T11:58:06.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 -- start start 2024-09-05T11:58:06.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83bd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83be4020 0x55dd83be6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd83b3f3e0 con 0x55dd83bd9760 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd83ae6d80 con 0x55dd83bd8e40 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.244+0000 7f1da95ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd83b3b3a0 con 0x55dd83be4020 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83be4020 0x55dd83be6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60344/0 (socket says 172.21.15.59:60344) 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4041648662 learned_addr learned my addr 172.21.15.59:0/4041648662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:06.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1da4a00640 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83bd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9cc00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 0x7f1d840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4041648662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83be4020 msgr2=0x55dd83be6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83be4020 0x55dd83be6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4041648662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 msgr2=0x55dd83bd9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83bd9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.245+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4041648662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd83b05ae0 con 0x55dd83bd9760 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.246+0000 7f1d9cc00640 1 -- 172.21.15.59:0/1077643812 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1d900cf900 con 0x7f1da01019c0 2024-09-05T11:58:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.246+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d9001eb80 tx=0x7f1d90021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9966cf5373d21188 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.247+0000 7f1d95200640 1 -- 172.21.15.59:0/4041648662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9009b830 con 0x55dd83bd9760 2024-09-05T11:58:06.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.247+0000 7f1d95200640 1 -- 172.21.15.59:0/4041648662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9001ee40 con 0x55dd83bd9760 2024-09-05T11:58:06.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.247+0000 7f1d95200640 1 -- 172.21.15.59:0/4041648662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9009b650 con 0x55dd83bd9760 2024-09-05T11:58:06.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da4a00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 0x7f1d840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 -- 172.21.15.59:0/4041648662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 msgr2=0x55dd83be3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d9001eb80 tx=0x7f1d90021000 comp rx=0 tx=0).stop 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 -- 172.21.15.59:0/4041648662 shutdown_connections 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83be4020 0x55dd83be6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be3ae0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4041648662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83bd9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 -- 172.21.15.59:0/4041648662 >> 172.21.15.59:0/4041648662 conn(0x55dd83bd53d0 msgr2=0x55dd83bd57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.248+0000 7f1da95ef740 1 -- 172.21.15.59:0/4041648662 shutdown_connections 2024-09-05T11:58:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.249+0000 7f1da95ef740 1 -- 172.21.15.59:0/4041648662 wait complete. 2024-09-05T11:58:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.249+0000 7f1da4a00640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 0x7f1d840a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005fd0 tx=0x7f1d98004a40 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.249+0000 7f1da95ef740 1 -- start start 2024-09-05T11:58:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 Processor -- start 2024-09-05T11:58:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83be0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd83be6b90 con 0x55dd83bd9760 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd83afa0f0 con 0x55dd83be4020 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd83be6d10 con 0x55dd83bd8e40 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.250+0000 7f1da95ef740 10 client.0 _ll_register_callbacks cb 0x55dd83ae1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.251+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83be0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.251+0000 7f1da4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.251+0000 7f1da4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50810/0 (socket says 172.21.15.59:50810) 2024-09-05T11:58:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.251+0000 7f1da4a00640 1 -- 172.21.15.59:0/2491935070 learned_addr learned my addr 172.21.15.59:0/2491935070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.251+0000 7f1d9f400640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 -- 172.21.15.59:0/2491935070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 msgr2=0x55dd83be0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83be0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 -- 172.21.15.59:0/2491935070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 msgr2=0x55dd83be1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 -- 172.21.15.59:0/2491935070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd838b03c0 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1da4a00640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d980118f0 tx=0x7f1d98004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79daa82f9d46f31d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1d89600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.252+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98022070 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.253+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1d9800dce0 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.253+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd83b4c950 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.253+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1d9802c050 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.253+0000 7f1da95ef740 10 client.25409 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.253+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd83bfb490 con 0x55dd83be4020 2024-09-05T11:58:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dd83b4c950 con 0x55dd83be4020 2024-09-05T11:58:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1d98027440 con 0x55dd83be4020 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 10 client.25409 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 20 client.25409 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83bfc9a0 0x55dd83bfedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 4 client.25409 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.254+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd83b46e60 con 0x55dd83bfc9a0 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.255+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83bfc9a0 0x55dd83bfedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.255+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83bfc9a0 0x55dd83bfedf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2682c6568d4c434e server_cookie=151adecad9c8ede in_seq=0 out_seq=0 2024-09-05T11:58:06.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.255+0000 7f1d89600640 10 client.25409 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.256+0000 7f1d89600640 1 -- 172.21.15.59:0/2491935070 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd83b46e60 con 0x55dd83bfc9a0 2024-09-05T11:58:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.256+0000 7f1d89600640 10 client.25409 handle_command_reply: tid=0 2024-09-05T11:58:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.256+0000 7f1da95ef740 1 client.25409 shutdown 2024-09-05T11:58:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.256+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 msgr2=0x55dd83be19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.256+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d980118f0 tx=0x7f1d98004ab0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 shutdown_connections 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83be4020 0x55dd83be19d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83bfc9a0 0x55dd83bfedf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83bd9760 0x55dd83be1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 --2- 172.21.15.59:0/2491935070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83be0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 >> 172.21.15.59:0/2491935070 conn(0x55dd83bd53d0 msgr2=0x55dd83bdb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1d89600640 0 client.25409 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:06.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 shutdown_connections 2024-09-05T11:58:06.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 1 -- 172.21.15.59:0/2491935070 wait complete. 2024-09-05T11:58:06.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.257+0000 7f1da95ef740 20 client.25409 trim_cache size 0 max 16384 2024-09-05T11:58:06.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.284+0000 7f1da95ef740 1 Processor -- start 2024-09-05T11:58:06.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 -- start start 2024-09-05T11:58:06.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01aab30 con 0x55dd83be4020 2024-09-05T11:58:06.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ad3b0 con 0x55dd83bd8e40 2024-09-05T11:58:06.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1da95ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ab1a0 con 0x55dd83bd9760 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1d9f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50818/0 (socket says 172.21.15.59:50818) 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4129226115 learned_addr learned my addr 172.21.15.59:0/4129226115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.285+0000 7f1d9f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44166/0 (socket says 172.21.15.59:44166) 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1da4a00640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4129226115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 msgr2=0x55dd83c04c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4129226115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 msgr2=0x55dd83bde1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4129226115 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd83bff470 con 0x55dd83bd8e40 2024-09-05T11:58:06.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1d9f400640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.286+0000 7f1da4a00640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.287+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c6a0 tx=0x7f1d90004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738d22b01f8dd5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.287+0000 7f1d8be00640 1 -- 172.21.15.59:0/4129226115 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9009b830 con 0x55dd83bd8e40 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.287+0000 7f1d8be00640 1 -- 172.21.15.59:0/4129226115 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9009c3f0 con 0x55dd83bd8e40 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.287+0000 7f1d8be00640 1 -- 172.21.15.59:0/4129226115 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90097b80 con 0x55dd83bd8e40 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 -- 172.21.15.59:0/4129226115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 msgr2=0x55dd83c046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c6a0 tx=0x7f1d90004a40 comp rx=0 tx=0).stop 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 -- 172.21.15.59:0/4129226115 shutdown_connections 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd9760 0x55dd83c04c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4129226115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd8e40 0x55dd83c046c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 -- 172.21.15.59:0/4129226115 >> 172.21.15.59:0/4129226115 conn(0x55dd83bd53d0 msgr2=0x55dd83bdc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.288+0000 7f1da95ef740 1 -- 172.21.15.59:0/4129226115 shutdown_connections 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.289+0000 7f1da95ef740 1 -- 172.21.15.59:0/4129226115 wait complete. 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.289+0000 7f1da95ef740 1 -- start start 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.289+0000 7f1da95ef740 1 Processor -- start 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd9760 0x55dd83bddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ab870 con 0x55dd83be4020 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01acce0 con 0x55dd83bd9760 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01abf40 con 0x55dd83bd8e40 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da95ef740 10 client.0 _ll_register_callbacks cb 0x7f1d980115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44180/0 (socket says 172.21.15.59:44180) 2024-09-05T11:58:06.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4286114835 learned_addr learned my addr 172.21.15.59:0/4286114835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:06.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1da4a00640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.290+0000 7f1d9f400640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd9760 0x55dd83bddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd9760 msgr2=0x55dd83bddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd9760 0x55dd83bddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4286114835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 msgr2=0x55dd83bde1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 -- 172.21.15.59:0/4286114835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd83bdea00 con 0x55dd83bd8e40 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1da4a00640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d9fe00640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d900a4430 tx=0x7f1d9000c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5eca698afe4290 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d89600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9009b830 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1d90097b80 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.291+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9009c3f0 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1d9009bcf0 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1da95ef740 10 client.25360 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd83c675a0 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9009c3f0 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1d90098360 con 0x55dd83bd8e40 2024-09-05T11:58:06.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.292+0000 7f1da95ef740 10 client.25360 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1da95ef740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1da95ef740 20 client.25360 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83c67930 0x55dd83c69d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1da95ef740 4 client.25360 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:58:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd83ac3010 con 0x55dd83c67930 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.293+0000 7f1d9f400640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83c67930 0x55dd83c69d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1d89600640 10 client.25360 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1d9f400640 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83c67930 0x55dd83c69d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93f340523e7496c server_cookie=5f1518c5c7c7e35d in_seq=0 out_seq=0 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1d89600640 1 -- 172.21.15.59:0/4286114835 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd83ac3010 con 0x55dd83c67930 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1d89600640 10 client.25360 handle_command_reply: tid=0 2024-09-05T11:58:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1da95ef740 1 client.25360 shutdown 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 msgr2=0x55dd83bf4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.294+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d900a4430 tx=0x7f1d9000c7d0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 shutdown_connections 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd83c67930 0x55dd83c69d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd83be4020 0x55dd83bde1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd83bd9760 0x55dd83bddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 --2- 172.21.15.59:0/4286114835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd83bd8e40 0x55dd83bf4f30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 >> 172.21.15.59:0/4286114835 conn(0x55dd83bd53d0 msgr2=0x55dd83be27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1d89600640 0 client.25360 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 shutdown_connections 2024-09-05T11:58:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.295+0000 7f1da95ef740 1 -- 172.21.15.59:0/4286114835 wait complete. 2024-09-05T11:58:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.296+0000 7f1da95ef740 20 client.25360 trim_cache size 0 max 16384 2024-09-05T11:58:06.299 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:58:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 msgr2=0x7f1d840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 0x7f1d840a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005fd0 tx=0x7f1d98004a40 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 msgr2=0x7f1da01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edf0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 shutdown_connections 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1d840a2c00 0x7f1d840a50c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da01079c0 0x7f1da019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102360 0x7f1da0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 --2- 172.21.15.59:0/1077643812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1da01019c0 0x7f1da01982f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 >> 172.21.15.59:0/1077643812 conn(0x7f1da00fd740 msgr2=0x7f1da00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 shutdown_connections 2024-09-05T11:58:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:06.297+0000 7f1da5400640 1 -- 172.21.15.59:0/1077643812 wait complete. 2024-09-05T11:58:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:06 smithi089 ceph-mon[34499]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:06.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:06 smithi059 ceph-mon[30025]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:06.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:06 smithi189 ceph-mon[34772]: pgmap v407: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T11:58:08.067 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:58:08.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:58:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:08 smithi089 ceph-mon[34499]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:08.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:08 smithi059 ceph-mon[30025]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:08.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:08 smithi189 ceph-mon[34772]: pgmap v408: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:09.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.335+0000 7f9dba200640 1 -- 172.21.15.59:0/297471375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db4100720 msgr2=0x7f9db4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.335+0000 7f9dba200640 1 --2- 172.21.15.59:0/297471375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db4100720 0x7f9db4100b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009900 tx=0x7f9da802a410 comp rx=0 tx=0).stop 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 -- 172.21.15.59:0/297471375 shutdown_connections 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 --2- 172.21.15.59:0/297471375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db410e2d0 0x7f9db41106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 --2- 172.21.15.59:0/297471375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 --2- 172.21.15.59:0/297471375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db4100720 0x7f9db4100b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 -- 172.21.15.59:0/297471375 >> 172.21.15.59:0/297471375 conn(0x7f9db40fc610 msgr2=0x7f9db40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 -- 172.21.15.59:0/297471375 shutdown_connections 2024-09-05T11:58:09.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.336+0000 7f9dba200640 1 -- 172.21.15.59:0/297471375 wait complete. 2024-09-05T11:58:09.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.337+0000 7f9dba200640 1 Processor -- start 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.337+0000 7f9dba200640 1 -- start start 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 0x7f9db419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4114080 con 0x7f9db410e2d0 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4113f00 con 0x7f9db41010d0 2024-09-05T11:58:09.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.338+0000 7f9dba200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4114200 con 0x7f9db4100720 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44194/0 (socket says 172.21.15.59:44194) 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 -- 172.21.15.59:0/3424423234 learned_addr learned my addr 172.21.15.59:0/3424423234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db9800640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 0x7f9db419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 -- 172.21.15.59:0/3424423234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 msgr2=0x7f9db419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 -- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 msgr2=0x7f9db419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 0x7f9db419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db8e00640 1 -- 172.21.15.59:0/3424423234 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db419f490 con 0x7f9db4100720 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db9800640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 0x7f9db419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.339+0000 7f9db3e00640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9db8e00640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9da802a980 tx=0x7f9da8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9db1600640 1 -- 172.21.15.59:0/3424423234 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da803f070 con 0x7f9db4100720 2024-09-05T11:58:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9db1600640 1 -- 172.21.15.59:0/3424423234 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da8031c90 con 0x7f9db4100720 2024-09-05T11:58:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9db1600640 1 -- 172.21.15.59:0/3424423234 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8035030 con 0x7f9db4100720 2024-09-05T11:58:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db419f720 con 0x7f9db4100720 2024-09-05T11:58:09.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.340+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db419fbb0 con 0x7f9db4100720 2024-09-05T11:58:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.345+0000 7f9dbe23a740 1 Processor -- start 2024-09-05T11:58:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.345+0000 7f9dbe23a740 1 -- start start 2024-09-05T11:58:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.345+0000 7f9db1600640 1 -- 172.21.15.59:0/3424423234 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9da8042070 con 0x7f9db4100720 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 0x56134a184090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a18e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 0x56134a191280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56134a0ea290 con 0x56134a1845d0 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56134a091b40 con 0x56134a18ee90 2024-09-05T11:58:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56134a0e6250 con 0x56134a183cb0 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9db9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 0x56134a184090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9db9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 0x56134a184090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44210/0 (socket says 172.21.15.59:44210) 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.346+0000 7f9db9800640 1 -- 172.21.15.59:0/1047483328 learned_addr learned my addr 172.21.15.59:0/1047483328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 0x56134a191280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db8e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a18e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 -- 172.21.15.59:0/1047483328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 msgr2=0x56134a184090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 0x56134a184090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 -- 172.21.15.59:0/1047483328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 msgr2=0x56134a18e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a18e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 -- 172.21.15.59:0/1047483328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561349e5b6a0 con 0x56134a18ee90 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db8e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a18e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.347+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 0x56134a191280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009c00 tx=0x7f9da00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18dcbf6c25fd240 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.348+0000 7f9da6000640 1 -- 172.21.15.59:0/1047483328 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0012450 con 0x56134a18ee90 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.348+0000 7f9da6000640 1 -- 172.21.15.59:0/1047483328 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da0012ad0 con 0x56134a18ee90 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.348+0000 7f9da6000640 1 -- 172.21.15.59:0/1047483328 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000fb90 con 0x56134a18ee90 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.348+0000 7f9db1600640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 0x7f9d9c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.348+0000 7f9db1600640 1 -- 172.21.15.59:0/3424423234 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9da80efbe0 con 0x7f9db4100720 2024-09-05T11:58:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1047483328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 msgr2=0x56134a191280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 0x56134a191280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009c00 tx=0x7f9da00117b0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1047483328 shutdown_connections 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a18ee90 0x56134a191280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a18e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1047483328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a183cb0 0x56134a184090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1047483328 >> 172.21.15.59:0/1047483328 conn(0x56134a180240 msgr2=0x56134a180620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1047483328 shutdown_connections 2024-09-05T11:58:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1047483328 wait complete. 2024-09-05T11:58:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.349+0000 7f9db9800640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 0x7f9d9c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.350+0000 7f9dbe23a740 1 -- start start 2024-09-05T11:58:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.350+0000 7f9dbe23a740 1 Processor -- start 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9db9800640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 0x7f9d9c0a5180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00f400 tx=0x7f9dac0077a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a18e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 0x56134a18c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56134a191b00 con 0x56134a18ee90 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56134a0a4fa0 con 0x56134a183cb0 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56134a191c80 con 0x56134a1845d0 2024-09-05T11:58:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9dbe23a740 10 client.0 _ll_register_callbacks cb 0x561349df8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44226/0 (socket says 172.21.15.59:44226) 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 -- 172.21.15.59:0/1555844353 learned_addr learned my addr 172.21.15.59:0/1555844353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db9800640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 0x56134a18c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.351+0000 7f9db8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a18e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 -- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 msgr2=0x56134a18e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a18e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 -- 172.21.15.59:0/1555844353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 msgr2=0x56134a18c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 0x56134a18c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db3e00640 1 -- 172.21.15.59:0/1555844353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134a18c880 con 0x56134a1845d0 2024-09-05T11:58:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.352+0000 7f9db9800640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 0x56134a18c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.353+0000 7f9db3e00640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9da001c040 tx=0x7f9da000f4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfc70da54b88743f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.353+0000 7f9d9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.353+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0029020 con 0x56134a1845d0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.354+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9da0012660 con 0x56134a1845d0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.354+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da0012ad0 con 0x56134a1845d0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.354+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9da0034050 con 0x56134a1845d0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.354+0000 7f9dbe23a740 10 client.25369 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.354+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56134a1a61d0 con 0x56134a1845d0 2024-09-05T11:58:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.355+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da0012ad0 con 0x56134a1845d0 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9da00054d0 con 0x56134a1845d0 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 10 client.25369 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 10 client.25369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 20 client.25369 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a1a7580 0x56134a1a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 4 client.25369 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56134a0f7800 con 0x56134a1a7580 2024-09-05T11:58:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.356+0000 7f9db8e00640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a1a7580 0x56134a1a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.357+0000 7f9db8e00640 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a1a7580 0x56134a1a9970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf89bfcddafaad35 server_cookie=4458a4ab1de46722 in_seq=0 out_seq=0 2024-09-05T11:58:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.357+0000 7f9d9aa00640 10 client.25369 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.358+0000 7f9d9aa00640 1 -- 172.21.15.59:0/1555844353 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56134a0f7800 con 0x56134a1a7580 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.358+0000 7f9d9aa00640 10 client.25369 handle_command_reply: tid=0 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.358+0000 7f9dbe23a740 1 client.25369 shutdown 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 msgr2=0x56134a18bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9da001c040 tx=0x7f9da000f4b0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 shutdown_connections 2024-09-05T11:58:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a1a7580 0x56134a1a9970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a18ee90 0x56134a18c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a1845d0 0x56134a18bad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/1555844353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a18e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 >> 172.21.15.59:0/1555844353 conn(0x56134a180240 msgr2=0x56134a185f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.359+0000 7f9d9aa00640 0 client.25369 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.360+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 shutdown_connections 2024-09-05T11:58:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.360+0000 7f9dbe23a740 1 -- 172.21.15.59:0/1555844353 wait complete. 2024-09-05T11:58:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.360+0000 7f9dbe23a740 20 client.25369 trim_cache size 0 max 16384 2024-09-05T11:58:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 Processor -- start 2024-09-05T11:58:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 -- start start 2024-09-05T11:58:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a1aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a17ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41aabc0 con 0x56134a1845d0 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ad3c0 con 0x56134a183cb0 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.397+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ab210 con 0x56134a18ee90 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.398+0000 7f9db3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.398+0000 7f9db3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60414/0 (socket says 172.21.15.59:60414) 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.398+0000 7f9db3e00640 1 -- 172.21.15.59:0/2887615521 learned_addr learned my addr 172.21.15.59:0/2887615521 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.398+0000 7f9db8e00640 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a1aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.399+0000 7f9db3e00640 1 -- 172.21.15.59:0/2887615521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 msgr2=0x56134a17ea50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T11:58:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.399+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a17ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.399+0000 7f9db3e00640 1 -- 172.21.15.59:0/2887615521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 msgr2=0x56134a1aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.399+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a1aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.399+0000 7f9db3e00640 1 -- 172.21.15.59:0/2887615521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56134a191280 con 0x56134a1845d0 2024-09-05T11:58:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.400+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000c00 tx=0x7f9da00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c8e25462164cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.400+0000 7f9da5600640 1 -- 172.21.15.59:0/2887615521 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0029070 con 0x56134a1845d0 2024-09-05T11:58:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.400+0000 7f9da5600640 1 -- 172.21.15.59:0/2887615521 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da003d070 con 0x56134a1845d0 2024-09-05T11:58:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.400+0000 7f9da5600640 1 -- 172.21.15.59:0/2887615521 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000fc70 con 0x56134a1845d0 2024-09-05T11:58:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.401+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2887615521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 msgr2=0x56134a1af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.401+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000c00 tx=0x7f9da00117b0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.401+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2887615521 shutdown_connections 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.401+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a17ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.401+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a1845d0 0x56134a1af4e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.402+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2887615521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a183cb0 0x56134a1aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.402+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2887615521 >> 172.21.15.59:0/2887615521 conn(0x56134a180240 msgr2=0x56134a18db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.402+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2887615521 shutdown_connections 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.402+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2887615521 wait complete. 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.403+0000 7f9dbe23a740 1 -- start start 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.403+0000 7f9dbe23a740 1 Processor -- start 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a19b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 0x56134a19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ab880 con 0x56134a183cb0 2024-09-05T11:58:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41accf0 con 0x56134a19b5c0 2024-09-05T11:58:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41abf50 con 0x56134a18ee90 2024-09-05T11:58:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9dbe23a740 10 client.0 _ll_register_callbacks cb 0x7f9dac007640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9db8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9db8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60420/0 (socket says 172.21.15.59:60420) 2024-09-05T11:58:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9db8e00640 1 -- 172.21.15.59:0/2261987197 learned_addr learned my addr 172.21.15.59:0/2261987197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:09.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.404+0000 7f9db9800640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 0x56134a19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db8e00640 1 -- 172.21.15.59:0/2261987197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 msgr2=0x56134a19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db8e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db8e00640 1 -- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 msgr2=0x56134a19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db8e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 0x56134a19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db8e00640 1 -- 172.21.15.59:0/2261987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134a210fc0 con 0x56134a183cb0 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.405+0000 7f9db9800640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 0x56134a19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.406+0000 7f9db8e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005ab0 tx=0x7f9da8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db73512b71e1e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.406+0000 7f9d9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.406+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80ba2d0 con 0x56134a183cb0 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.406+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da003d070 con 0x56134a183cb0 2024-09-05T11:58:09.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.406+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9da80ba070 con 0x56134a183cb0 2024-09-05T11:58:09.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.407+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9da80ba2d0 con 0x56134a183cb0 2024-09-05T11:58:09.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.407+0000 7f9dbe23a740 10 client.15417 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:09.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.407+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56134a212320 con 0x56134a183cb0 2024-09-05T11:58:09.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.407+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da80f3030 con 0x56134a183cb0 2024-09-05T11:58:09.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.407+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9da80c47b0 con 0x56134a183cb0 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 10 client.15417 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 20 client.15417 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a212a40 0x56134a214e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 4 client.15417 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56134a0f1d10 con 0x56134a212a40 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.408+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a212a40 0x56134a214e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:09.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.409+0000 7f9db3e00640 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a212a40 0x56134a214e30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c12d6bfa160e38 server_cookie=c9176127177c6908 in_seq=0 out_seq=0 2024-09-05T11:58:09.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.409+0000 7f9d9aa00640 10 client.15417 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:09.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.410+0000 7f9d9aa00640 1 -- 172.21.15.59:0/2261987197 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56134a0f1d10 con 0x56134a212a40 2024-09-05T11:58:09.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.410+0000 7f9d9aa00640 10 client.15417 handle_command_reply: tid=0 2024-09-05T11:58:09.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.410+0000 7f9dbe23a740 1 client.15417 shutdown 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 msgr2=0x56134a19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005ab0 tx=0x7f9da8003f90 comp rx=0 tx=0).stop 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 shutdown_connections 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56134a19b5c0 0x56134a19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56134a18ee90 0x56134a19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56134a212a40 0x56134a214e30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 --2- 172.21.15.59:0/2261987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56134a183cb0 0x56134a19fdf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 >> 172.21.15.59:0/2261987197 conn(0x56134a180240 msgr2=0x56134a17ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.411+0000 7f9d9aa00640 0 client.15417 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "051a9c3a-6e50-4fc3-9c1e-30fe1095744c", 2024-09-05T11:58:09.415 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.412+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 shutdown_connections 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.412+0000 7f9dbe23a740 1 -- 172.21.15.59:0/2261987197 wait complete. 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.412+0000 7f9dbe23a740 20 client.15417 trim_cache size 0 max 16384 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.413+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 msgr2=0x7f9d9c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.413+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 0x7f9d9c0a5180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00f400 tx=0x7f9dac0077a0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.413+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 msgr2=0x7f9db419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.413+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9da802a980 tx=0x7f9da8003b90 comp rx=0 tx=0).stop 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 shutdown_connections 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f9d9c0a2cc0 0x7f9d9c0a5180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db410e2d0 0x7f9db419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db41010d0 0x7f9db419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 --2- 172.21.15.59:0/3424423234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9db4100720 0x7f9db419a4e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:09.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 >> 172.21.15.59:0/3424423234 conn(0x7f9db40fc610 msgr2=0x7f9db40fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:09.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.414+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 shutdown_connections 2024-09-05T11:58:09.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:09.415+0000 7f9dba200640 1 -- 172.21.15.59:0/3424423234 wait complete. 2024-09-05T11:58:10.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:10 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:10 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:58:10.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: pgmap v409: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:58:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:58:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:10 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:58:10.712 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:11.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 -- 172.21.15.59:0/3530649952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac101a00 msgr2=0x7f24ac101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 --2- 172.21.15.59:0/3530649952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac101a00 0x7f24ac101e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2498009900 tx=0x7f249802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 -- 172.21.15.59:0/3530649952 shutdown_connections 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 --2- 172.21.15.59:0/3530649952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 --2- 172.21.15.59:0/3530649952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac101a00 0x7f24ac101e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 --2- 172.21.15.59:0/3530649952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac107a00 0x7f24ac107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.357+0000 7f24b0c00640 1 -- 172.21.15.59:0/3530649952 >> 172.21.15.59:0/3530649952 conn(0x7f24ac0fd6e0 msgr2=0x7f24ac0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.358+0000 7f24b0c00640 1 -- 172.21.15.59:0/3530649952 shutdown_connections 2024-09-05T11:58:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.358+0000 7f24b0c00640 1 -- 172.21.15.59:0/3530649952 wait complete. 2024-09-05T11:58:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.359+0000 7f24b0c00640 1 Processor -- start 2024-09-05T11:58:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.359+0000 7f24b0c00640 1 -- start start 2024-09-05T11:58:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 0x7f24ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac107a00 0x7f24ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111e50 con 0x7f24ac101a00 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111cd0 con 0x7f24ac107a00 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24b0c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111fd0 con 0x7f24ac1023a0 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 0x7f24ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 0x7f24ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53504/0 (socket says 172.21.15.59:53504) 2024-09-05T11:58:11.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24a9e00640 1 -- 172.21.15.59:0/2635349112 learned_addr learned my addr 172.21.15.59:0/2635349112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.360+0000 7f24a9400640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24aa800640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac107a00 0x7f24ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24a9400640 1 -- 172.21.15.59:0/2635349112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac107a00 msgr2=0x7f24ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24a9400640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac107a00 0x7f24ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24a9400640 1 -- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 msgr2=0x7f24ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24a9400640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 0x7f24ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.361+0000 7f24a9400640 1 -- 172.21.15.59:0/2635349112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1aacc0 con 0x7f24ac1023a0 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.362+0000 7f24a9400640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac074a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f249802a910 tx=0x7f2498003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.363+0000 7f24a2a00640 1 -- 172.21.15.59:0/2635349112 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249803f070 con 0x7f24ac1023a0 2024-09-05T11:58:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.363+0000 7f24a2a00640 1 -- 172.21.15.59:0/2635349112 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498031c20 con 0x7f24ac1023a0 2024-09-05T11:58:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.363+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac1aae90 con 0x7f24ac1023a0 2024-09-05T11:58:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.363+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1ab340 con 0x7f24ac1023a0 2024-09-05T11:58:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.363+0000 7f24a2a00640 1 -- 172.21.15.59:0/2635349112 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498035030 con 0x7f24ac1023a0 2024-09-05T11:58:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 Processor -- start 2024-09-05T11:58:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 -- start start 2024-09-05T11:58:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 0x55ec277ec130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277f6f30 0x55ec277f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec277523e0 con 0x55ec277ebd50 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec276f9d80 con 0x55ec277ec670 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.367+0000 7f24b3824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec2774e3a0 con 0x55ec277f6f30 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46010/0 (socket says 172.21.15.59:46010) 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 -- 172.21.15.59:0/1574335418 learned_addr learned my addr 172.21.15.59:0/1574335418 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 -- 172.21.15.59:0/1574335418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277f6f30 msgr2=0x55ec277f9320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277f6f30 0x55ec277f9320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.368+0000 7f24a9e00640 1 -- 172.21.15.59:0/1574335418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 msgr2=0x55ec277ec130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:58:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.369+0000 7f24aa800640 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 0x55ec277ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.369+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 0x55ec277ec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.369+0000 7f24a9e00640 1 -- 172.21.15.59:0/1574335418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec27718ae0 con 0x55ec277ec670 2024-09-05T11:58:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.369+0000 7f24aa800640 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 0x55ec277ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.370+0000 7f24a2a00640 1 -- 172.21.15.59:0/2635349112 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2498042070 con 0x7f24ac1023a0 2024-09-05T11:58:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.370+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f249c00bcd0 tx=0x7f249c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa778a4a7254d515 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.370+0000 7f2497000640 1 -- 172.21.15.59:0/1574335418 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c015070 con 0x55ec277ec670 2024-09-05T11:58:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.371+0000 7f2497000640 1 -- 172.21.15.59:0/1574335418 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c011920 con 0x55ec277ec670 2024-09-05T11:58:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.371+0000 7f2497000640 1 -- 172.21.15.59:0/1574335418 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c015070 con 0x55ec277ec670 2024-09-05T11:58:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.371+0000 7f24b3824740 1 -- 172.21.15.59:0/1574335418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 msgr2=0x55ec277f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.371+0000 7f24b3824740 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f249c00bcd0 tx=0x7f249c00f730 comp rx=0 tx=0).stop 2024-09-05T11:58:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24a2a00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24a2a00640 1 -- 172.21.15.59:0/2635349112 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24980efa50 con 0x7f24ac1023a0 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 -- 172.21.15.59:0/1574335418 shutdown_connections 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277f6f30 0x55ec277f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f69f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 --2- 172.21.15.59:0/1574335418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ebd50 0x55ec277ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 -- 172.21.15.59:0/1574335418 >> 172.21.15.59:0/1574335418 conn(0x55ec277e82e0 msgr2=0x55ec277e86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24aa800640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.372+0000 7f24b3824740 1 -- 172.21.15.59:0/1574335418 shutdown_connections 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.373+0000 7f24b3824740 1 -- 172.21.15.59:0/1574335418 wait complete. 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.373+0000 7f24b3824740 1 -- start start 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24b3824740 1 Processor -- start 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24b3824740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 0x55ec277f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24b3824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ec670 0x55ec277f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24aa800640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 0x7f24900a5140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005fd0 tx=0x7f24a4009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24b3824740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.374+0000 7f24b3824740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ec277f9ba0 con 0x55ec277f6f30 2024-09-05T11:58:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 0x55ec277f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 0x55ec277f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46024/0 (socket says 172.21.15.59:46024) 2024-09-05T11:58:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24a9e00640 1 -- 172.21.15.59:0/1061128256 learned_addr learned my addr 172.21.15.59:0/1061128256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec2770d0f0 con 0x55ec277ebd50 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec277f9d20 con 0x55ec277ec670 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24b3824740 10 client.0 _ll_register_callbacks cb 0x55ec276fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24aa800640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.375+0000 7f24a9400640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ec670 0x55ec277f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.376+0000 7f24aa800640 1 -- 172.21.15.59:0/1061128256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ec670 msgr2=0x55ec277f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.376+0000 7f24aa800640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ec670 0x55ec277f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.376+0000 7f24aa800640 1 -- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 msgr2=0x55ec277f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.376+0000 7f24aa800640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 0x55ec277f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.376+0000 7f24aa800640 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec274c33c0 con 0x55ec277f6f30 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.377+0000 7f24aa800640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f48d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24a4013d10 tx=0x7f24a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a97f8d234436a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.377+0000 7f248b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.377+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40119f0 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.377+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24a4011b90 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.377+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249c011920 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.378+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f24a4031050 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.378+0000 7f24b3824740 10 client.15426 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.378+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ec278058e0 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.378+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c011920 con 0x55ec277f6f30 2024-09-05T11:58:11.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.378+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f24a4020400 con 0x55ec277f6f30 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 10 client.15426 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 10 client.15426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 20 client.15426 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27806df0 0x55ec27809230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 4 client.15426 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:11.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec2775f950 con 0x55ec27806df0 2024-09-05T11:58:11.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.379+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27806df0 0x55ec27809230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.380+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27806df0 0x55ec27809230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2552d86a2eec716 server_cookie=f881061fdb71f64a in_seq=0 out_seq=0 2024-09-05T11:58:11.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.380+0000 7f248b400640 10 client.15426 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:11.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f248b400640 1 -- 172.21.15.59:0/1061128256 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec2775f950 con 0x55ec27806df0 2024-09-05T11:58:11.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f248b400640 10 client.15426 handle_command_reply: tid=0 2024-09-05T11:58:11.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 client.15426 shutdown 2024-09-05T11:58:11.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 msgr2=0x55ec277f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f48d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24a4013d10 tx=0x7f24a4004a40 comp rx=0 tx=0).stop 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 shutdown_connections 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27806df0 0x55ec27809230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f48d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ec670 0x55ec277f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 --2- 172.21.15.59:0/1061128256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ebd50 0x55ec277f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.381+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 >> 172.21.15.59:0/1061128256 conn(0x55ec277e82e0 msgr2=0x55ec277ee010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.382+0000 7f248b400640 0 client.15426 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.382+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 shutdown_connections 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.382+0000 7f24b3824740 1 -- 172.21.15.59:0/1061128256 wait complete. 2024-09-05T11:58:11.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.382+0000 7f24b3824740 20 client.15426 trim_cache size 0 max 16384 2024-09-05T11:58:11.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.400+0000 7f24b3824740 1 Processor -- start 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 -- start start 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 0x55ec277f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 0x55ec2780eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f248c00dd30 con 0x55ec277ec670 2024-09-05T11:58:11.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0105b0 con 0x55ec277f6f30 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24b3824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f248c00e3a0 con 0x55ec277ebd50 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24a9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 0x55ec277f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24a9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 0x55ec277f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53546/0 (socket says 172.21.15.59:53546) 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24a9400640 1 -- 172.21.15.59:0/1021979836 learned_addr learned my addr 172.21.15.59:0/1021979836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.401+0000 7f24a9e00640 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 0x55ec2780eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 -- 172.21.15.59:0/1021979836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 msgr2=0x55ec277ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 -- 172.21.15.59:0/1021979836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 msgr2=0x55ec277f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 0x55ec277f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.402+0000 7f24aa800640 1 -- 172.21.15.59:0/1021979836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec277f2e00 con 0x55ec277f6f30 2024-09-05T11:58:11.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.403+0000 7f24aa800640 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 0x55ec2780eb00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24a4013db0 tx=0x7f24a40095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24ad8f556ab1b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.403+0000 7f2496600640 1 -- 172.21.15.59:0/1021979836 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4020400 con 0x55ec277f6f30 2024-09-05T11:58:11.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.403+0000 7f2496600640 1 -- 172.21.15.59:0/1021979836 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a4039070 con 0x55ec277f6f30 2024-09-05T11:58:11.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.403+0000 7f2496600640 1 -- 172.21.15.59:0/1021979836 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4011b60 con 0x55ec277f6f30 2024-09-05T11:58:11.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 -- 172.21.15.59:0/1021979836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 msgr2=0x55ec2780eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 0x55ec2780eb00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f24a4013db0 tx=0x7f24a40095e0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 -- 172.21.15.59:0/1021979836 shutdown_connections 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277f6f30 0x55ec2780eb00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277ec670 0x55ec277f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 --2- 172.21.15.59:0/1021979836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 -- 172.21.15.59:0/1021979836 >> 172.21.15.59:0/1021979836 conn(0x55ec277e82e0 msgr2=0x55ec277f56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 -- 172.21.15.59:0/1021979836 shutdown_connections 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.404+0000 7f24b3824740 1 -- 172.21.15.59:0/1021979836 wait complete. 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.405+0000 7f24b3824740 1 -- start start 2024-09-05T11:58:11.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.405+0000 7f24b3824740 1 Processor -- start 2024-09-05T11:58:11.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f248c00ea70 con 0x55ec277f6f30 2024-09-05T11:58:11.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f248c00fee0 con 0x55ec277ec670 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f248c00f140 con 0x55ec277ebd50 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24b3824740 10 client.0 _ll_register_callbacks cb 0x7f24a4003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24a9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.406+0000 7f24a9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46030/0 (socket says 172.21.15.59:46030) 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 -- 172.21.15.59:0/4275191671 learned_addr learned my addr 172.21.15.59:0/4275191671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:11.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9e00640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 -- 172.21.15.59:0/4275191671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 msgr2=0x55ec277ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 -- 172.21.15.59:0/4275191671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 msgr2=0x55ec277f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 -- 172.21.15.59:0/4275191671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec277f1300 con 0x55ec277ec670 2024-09-05T11:58:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24aa800640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.407+0000 7f24a9400640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8810 tx=0x7f24980bbd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9848dc634e0fe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.408+0000 7f248b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.408+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24980be070 con 0x55ec277ec670 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.408+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f249803b460 con 0x55ec277ec670 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.408+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a4039070 con 0x55ec277ec670 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.408+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f249803b660 con 0x55ec277ec670 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.409+0000 7f24b3824740 10 client.25448 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.409+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ec27871870 con 0x55ec277ec670 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.409+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a4039070 con 0x55ec277ec670 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f24980c4620 con 0x55ec277ec670 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 10 client.25448 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 20 client.25448 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27871fa0 0x55ec278743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 4 client.25448 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:58:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.410+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec27759e60 con 0x55ec27871fa0 2024-09-05T11:58:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.411+0000 7f24a9e00640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27871fa0 0x55ec278743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.411+0000 7f24a9e00640 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27871fa0 0x55ec278743c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ac698395d78df3 server_cookie=61344701d976042 in_seq=0 out_seq=0 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.411+0000 7f248b400640 10 client.25448 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.412+0000 7f248b400640 1 -- 172.21.15.59:0/4275191671 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ec27759e60 con 0x55ec27871fa0 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.412+0000 7f248b400640 10 client.25448 handle_command_reply: tid=0 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.412+0000 7f24b3824740 1 client.25448 shutdown 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.412+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 msgr2=0x55ec277f0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.412+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8810 tx=0x7f24980bbd00 comp rx=0 tx=0).stop 2024-09-05T11:58:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 shutdown_connections 2024-09-05T11:58:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ec27871fa0 0x55ec278743c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ec277f6f30 0x55ec277f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec277ec670 0x55ec277f0880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 --2- 172.21.15.59:0/4275191671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec277ebd50 0x55ec277ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 >> 172.21.15.59:0/4275191671 conn(0x55ec277e82e0 msgr2=0x55ec277edfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f248b400640 0 client.25448 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 shutdown_connections 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.413+0000 7f24b3824740 1 -- 172.21.15.59:0/4275191671 wait complete. 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.414+0000 7f24b3824740 20 client.25448 trim_cache size 0 max 16384 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:58:11.417 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.414+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 msgr2=0x7f24900a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.414+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 0x7f24900a5140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005fd0 tx=0x7f24a4009500 comp rx=0 tx=0).stop 2024-09-05T11:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 msgr2=0x7f24ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac074a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f249802a910 tx=0x7f2498003b90 comp rx=0 tx=0).stop 2024-09-05T11:58:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 shutdown_connections 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24ac107a00 0x7f24ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24ac1023a0 0x7f24ac074a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 --2- 172.21.15.59:0/2635349112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24ac101a00 0x7f24ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 >> 172.21.15.59:0/2635349112 conn(0x7f24ac0fd6e0 msgr2=0x7f24ac0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 shutdown_connections 2024-09-05T11:58:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:11.415+0000 7f24b0c00640 1 -- 172.21.15.59:0/2635349112 wait complete. 2024-09-05T11:58:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:12 smithi059 ceph-mon[30025]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-05T11:58:12.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:58:12.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:12.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:12 smithi089 ceph-mon[34499]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-05T11:58:12.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:12 smithi189 ceph-mon[34772]: pgmap v410: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-05T11:58:12.868 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:13.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 -- 172.21.15.59:0/3339316441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8106f10 msgr2=0x7ff7d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:13.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3339316441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8106f10 0x7ff7d810ea90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009ca0 tx=0x7ff7d002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 -- 172.21.15.59:0/3339316441 shutdown_connections 2024-09-05T11:58:13.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3339316441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8106f10 0x7ff7d810ea90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3339316441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8102790 0x7ff7d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3339316441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 0x7ff7d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 -- 172.21.15.59:0/3339316441 >> 172.21.15.59:0/3339316441 conn(0x7ff7d80fd8c0 msgr2=0x7ff7d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.482+0000 7ff7e0400640 1 -- 172.21.15.59:0/3339316441 shutdown_connections 2024-09-05T11:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.483+0000 7ff7e0400640 1 -- 172.21.15.59:0/3339316441 wait complete. 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.484+0000 7ff7e0400640 1 Processor -- start 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.484+0000 7ff7e0400640 1 -- start start 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.484+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 0x7ff7d819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.484+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.484+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8106f10 0x7ff7d819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7e0400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8111fc0 con 0x7ff7d8102790 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7e0400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8111e40 con 0x7ff7d8106f10 2024-09-05T11:58:13.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7e0400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8112140 con 0x7ff7d8101de0 2024-09-05T11:58:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7dd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7dd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53582/0 (socket says 172.21.15.59:53582) 2024-09-05T11:58:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7dd000640 1 -- 172.21.15.59:0/3724815022 learned_addr learned my addr 172.21.15.59:0/3724815022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7dda00640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 0x7ff7d819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.485+0000 7ff7de400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8106f10 0x7ff7d819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:13.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 -- 172.21.15.59:0/3724815022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 msgr2=0x7ff7d819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:13.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 0x7ff7d819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 -- 172.21.15.59:0/3724815022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8106f10 msgr2=0x7ff7d819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:13.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8106f10 0x7ff7d819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 -- 172.21.15.59:0/3724815022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d819f670 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7dd000640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400ea60 tx=0x7ff7c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400cde0 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c40035f0 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.486+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4010670 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.487+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d819f960 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.487+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d81a7240 con 0x7ff7d8102790 2024-09-05T11:58:13.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.488+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d81021c0 con 0x7ff7d8102790 2024-09-05T11:58:13.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.491+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7c40037a0 con 0x7ff7d8102790 2024-09-05T11:58:13.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.493+0000 7ff7d6000640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 0x7ff7c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:13.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.493+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff7c40cf770 con 0x7ff7d8102790 2024-09-05T11:58:13.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.494+0000 7ff7dda00640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 0x7ff7c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:13.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.495+0000 7ff7dda00640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 0x7ff7c00a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009bd0 tx=0x7ff7cc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.498+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7c409c260 con 0x7ff7d8102790 2024-09-05T11:58:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.795+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7d81a74e0 con 0x7ff7d8102790 2024-09-05T11:58:13.798 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:58:13.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.796+0000 7ff7d6000640 1 -- 172.21.15.59:0/3724815022 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7ff7c409bb60 con 0x7ff7d8102790 2024-09-05T11:58:13.799 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:58:13.799 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:58:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 msgr2=0x7ff7c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 0x7ff7c00a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009bd0 tx=0x7ff7cc005f90 comp rx=0 tx=0).stop 2024-09-05T11:58:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 msgr2=0x7ff7d819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400ea60 tx=0x7ff7c400ef20 comp rx=0 tx=0).stop 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 shutdown_connections 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff7c00a2c00 0x7ff7c00a50c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7d8106f10 0x7ff7d819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7d8102790 0x7ff7d819ac00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 --2- 172.21.15.59:0/3724815022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d8101de0 0x7ff7d819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.799+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 >> 172.21.15.59:0/3724815022 conn(0x7ff7d80fd8c0 msgr2=0x7ff7d80ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.800+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 shutdown_connections 2024-09-05T11:58:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:13.800+0000 7ff7e0400640 1 -- 172.21.15.59:0/3724815022 wait complete. 2024-09-05T11:58:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:14 smithi089 ceph-mon[34499]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:14 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3724815022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:14 smithi059 ceph-mon[30025]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:14 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3724815022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:14.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:14 smithi189 ceph-mon[34772]: pgmap v411: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:14.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:14 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3724815022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:14.637 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:58:15.092 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 -- 172.21.15.59:0/384079321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 msgr2=0x7f6a3010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 --2- 172.21.15.59:0/384079321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3010b9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c02a480 comp rx=0 tx=0).stop 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 -- 172.21.15.59:0/384079321 shutdown_connections 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 --2- 172.21.15.59:0/384079321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 0x7f6a3010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 --2- 172.21.15.59:0/384079321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3010b9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 --2- 172.21.15.59:0/384079321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a30102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 -- 172.21.15.59:0/384079321 >> 172.21.15.59:0/384079321 conn(0x7f6a300fc610 msgr2=0x7f6a300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 -- 172.21.15.59:0/384079321 shutdown_connections 2024-09-05T11:58:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.647+0000 7f6a35e00640 1 -- 172.21.15.59:0/384079321 wait complete. 2024-09-05T11:58:15.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.648+0000 7f6a35e00640 1 Processor -- start 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.648+0000 7f6a35e00640 1 -- start start 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 0x7f6a3019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30111dc0 con 0x7f6a3010bfc0 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30111c40 con 0x7f6a30102b90 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30111f40 con 0x7f6a301034b0 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46068/0 (socket says 172.21.15.59:46068) 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 -- 172.21.15.59:0/723807796 learned_addr learned my addr 172.21.15.59:0/723807796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a35400640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 0x7f6a3019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 -- 172.21.15.59:0/723807796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 msgr2=0x7f6a3019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 -- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 msgr2=0x7f6a3019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 0x7f6a3019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.649+0000 7f6a34a00640 1 -- 172.21.15.59:0/723807796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3019f470 con 0x7f6a30102b90 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a34a00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400ea60 tx=0x7f6a2400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a2d600640 1 -- 172.21.15.59:0/723807796 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400cde0 con 0x7f6a30102b90 2024-09-05T11:58:15.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3019f760 con 0x7f6a30102b90 2024-09-05T11:58:15.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a2d600640 1 -- 172.21.15.59:0/723807796 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a240035f0 con 0x7f6a30102b90 2024-09-05T11:58:15.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a2d600640 1 -- 172.21.15.59:0/723807796 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24010690 con 0x7f6a30102b90 2024-09-05T11:58:15.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.650+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a301a7040 con 0x7f6a30102b90 2024-09-05T11:58:15.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.653+0000 7f6a39fed740 1 Processor -- start 2024-09-05T11:58:15.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.653+0000 7f6a2d600640 1 -- 172.21.15.59:0/723807796 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a240037a0 con 0x7f6a30102b90 2024-09-05T11:58:15.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 -- start start 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb47b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 0x561afb485ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb486020 0x561afb488410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561afb3e13e0 con 0x561afb47b760 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561afb388d80 con 0x561afb486020 2024-09-05T11:58:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a39fed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561afb3dd3a0 con 0x561afb47ae40 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a34a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 0x561afb485ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a35400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb47b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a34a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 0x561afb485ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53612/0 (socket says 172.21.15.59:53612) 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.654+0000 7f6a34a00640 1 -- 172.21.15.59:0/794166897 learned_addr learned my addr 172.21.15.59:0/794166897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb486020 0x561afb488410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 -- 172.21.15.59:0/794166897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb486020 msgr2=0x561afb488410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb486020 0x561afb488410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 -- 172.21.15.59:0/794166897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 msgr2=0x561afb485ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 0x561afb485ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 -- 172.21.15.59:0/794166897 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afb3a7ae0 con 0x561afb47ae40 2024-09-05T11:58:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a35400640 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb47b220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009f00 tx=0x7f6a28011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4710a8d29135d3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.655+0000 7f6a2d600640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 0x7f6a180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.656+0000 7f6a21c00640 1 -- 172.21.15.59:0/794166897 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004d20 con 0x561afb47ae40 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.656+0000 7f6a2d600640 1 -- 172.21.15.59:0/723807796 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a240cf720 con 0x7f6a30102b90 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.656+0000 7f6a21c00640 1 -- 172.21.15.59:0/794166897 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a28004050 con 0x561afb47ae40 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.656+0000 7f6a21c00640 1 -- 172.21.15.59:0/794166897 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28005220 con 0x561afb47ae40 2024-09-05T11:58:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.656+0000 7f6a34a00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 0x7f6a180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- 172.21.15.59:0/794166897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 msgr2=0x561afb47b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a34a00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 0x7f6a180a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a240977f0 tx=0x7f6a240d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb47b220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009f00 tx=0x7f6a28011810 comp rx=0 tx=0).stop 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- 172.21.15.59:0/794166897 shutdown_connections 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb486020 0x561afb488410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47b760 0x561afb485ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 --2- 172.21.15.59:0/794166897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb47b220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- 172.21.15.59:0/794166897 >> 172.21.15.59:0/794166897 conn(0x561afb4773d0 msgr2=0x561afb4777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- 172.21.15.59:0/794166897 shutdown_connections 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- 172.21.15.59:0/794166897 wait complete. 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 -- start start 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.657+0000 7f6a39fed740 1 Processor -- start 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb482f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb4834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 0x561afb483a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561afb488bf0 con 0x561afb486020 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561afb39c0f0 con 0x561afb47b760 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561afb488d70 con 0x561afb47ae40 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a39fed740 10 client.0 _ll_register_callbacks cb 0x561afb383bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a35400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb482f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb4834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a35400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb482f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:60686/0 (socket says 172.21.15.59:60686) 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a35400640 1 -- 172.21.15.59:0/1611117262 learned_addr learned my addr 172.21.15.59:0/1611117262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.658+0000 7f6a35400640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 0x561afb483a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 -- 172.21.15.59:0/1611117262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 msgr2=0x561afb482f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb482f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 -- 172.21.15.59:0/1611117262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 msgr2=0x561afb483a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 0x561afb483a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 -- 172.21.15.59:0/1611117262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afb1523c0 con 0x561afb47b760 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a35400640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 0x561afb483a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb4834c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c006010 tx=0x7f6a1c0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6da87850e86f3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a16000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.659+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c035040 con 0x561afb47b760 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a28004050 con 0x561afb47b760 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a1c004260 con 0x561afb47b760 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c04e050 con 0x561afb47b760 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a39fed740 10 client.25466 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561afb494d00 con 0x561afb47b760 2024-09-05T11:58:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.660+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a28004050 con 0x561afb47b760 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6a1c004b10 con 0x561afb47b760 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 10 client.25466 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 20 client.25466 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb496110 0x561afb498520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 4 client.25466 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afb3ee950 con 0x561afb496110 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.661+0000 7f6a35400640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb496110 0x561afb498520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.662+0000 7f6a35400640 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb496110 0x561afb498520 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7534e2a9f18279d server_cookie=7953446d2e1794d9 in_seq=0 out_seq=0 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.662+0000 7f6a16000640 10 client.25466 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.662+0000 7f6a16000640 1 -- 172.21.15.59:0/1611117262 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561afb3ee950 con 0x561afb496110 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.662+0000 7f6a16000640 10 client.25466 handle_command_reply: tid=0 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 client.25466 shutdown 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 msgr2=0x561afb4834c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb4834c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c006010 tx=0x7f6a1c0317e0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 shutdown_connections 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb496110 0x561afb498520 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb486020 0x561afb483a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb4834c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 --2- 172.21.15.59:0/1611117262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb47ae40 0x561afb482f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a16000640 0 client.25466 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 >> 172.21.15.59:0/1611117262 conn(0x561afb4773d0 msgr2=0x561afb47d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 shutdown_connections 2024-09-05T11:58:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 1 -- 172.21.15.59:0/1611117262 wait complete. 2024-09-05T11:58:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.663+0000 7f6a39fed740 20 client.25466 trim_cache size 0 max 16384 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.677+0000 7f6a39fed740 1 Processor -- start 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.677+0000 7f6a39fed740 1 -- start start 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb483850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb49e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301aef50 con 0x561afb47ae40 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301b17d0 con 0x561afb47b760 2024-09-05T11:58:15.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a39fed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301af5c0 con 0x561afb486020 2024-09-05T11:58:15.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb483850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46094/0 (socket says 172.21.15.59:46094) 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 -- 172.21.15.59:0/625855219 learned_addr learned my addr 172.21.15.59:0/625855219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a35400640 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb49e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 -- 172.21.15.59:0/625855219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 msgr2=0x561afb49e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb49e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 -- 172.21.15.59:0/625855219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 msgr2=0x561afb483850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb483850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a2fe00640 1 -- 172.21.15.59:0/625855219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afb4858f0 con 0x561afb47b760 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.678+0000 7f6a35400640 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb49e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.679+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004ee0 tx=0x7f6a1c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe34a17c6a084f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.679+0000 7f6a21200640 1 -- 172.21.15.59:0/625855219 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c057070 con 0x561afb47b760 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.679+0000 7f6a21200640 1 -- 172.21.15.59:0/625855219 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c041070 con 0x561afb47b760 2024-09-05T11:58:15.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.679+0000 7f6a21200640 1 -- 172.21.15.59:0/625855219 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c035030 con 0x561afb47b760 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 -- 172.21.15.59:0/625855219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 msgr2=0x561afb49ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004ee0 tx=0x7f6a1c000bc0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 -- 172.21.15.59:0/625855219 shutdown_connections 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb49e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb49ddf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 --2- 172.21.15.59:0/625855219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb483850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 -- 172.21.15.59:0/625855219 >> 172.21.15.59:0/625855219 conn(0x561afb4773d0 msgr2=0x561afb485000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 -- 172.21.15.59:0/625855219 shutdown_connections 2024-09-05T11:58:15.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.680+0000 7f6a39fed740 1 -- 172.21.15.59:0/625855219 wait complete. 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 -- start start 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 Processor -- start 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb489ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb48a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301afc90 con 0x561afb47ae40 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301b1100 con 0x561afb47b760 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a301b0360 con 0x561afb486020 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.681+0000 7f6a39fed740 10 client.0 _ll_register_callbacks cb 0x7f6a24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a2fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb489ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53646/0 (socket says 172.21.15.59:53646) 2024-09-05T11:58:15.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 -- 172.21.15.59:0/259081178 learned_addr learned my addr 172.21.15.59:0/259081178 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb48a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 -- 172.21.15.59:0/259081178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 msgr2=0x561afb48a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb48a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 -- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 msgr2=0x561afb489ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb489ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a35400640 1 -- 172.21.15.59:0/259081178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afb48a890 con 0x561afb47ae40 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.682+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb489ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a35400640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6a28011d10 tx=0x7f6a280096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba2291e77eba15 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a16000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004460 con 0x561afb47ae40 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1c041070 con 0x561afb47ae40 2024-09-05T11:58:15.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a28004600 con 0x561afb47ae40 2024-09-05T11:58:15.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a28004800 con 0x561afb47ae40 2024-09-05T11:58:15.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a39fed740 10 client.15456 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:15.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561afb500e60 con 0x561afb47ae40 2024-09-05T11:58:15.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.683+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c041070 con 0x561afb47ae40 2024-09-05T11:58:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6a280195c0 con 0x561afb47ae40 2024-09-05T11:58:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 10 client.15456 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 20 client.15456 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb5013f0 0x561afb5037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 4 client.15456 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561afb4729a0 con 0x561afb5013f0 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.684+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb5013f0 0x561afb5037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a2fe00640 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb5013f0 0x561afb5037e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed59bcb187832568 server_cookie=914ccf214e3464c5 in_seq=0 out_seq=0 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a16000640 10 client.15456 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a16000640 1 -- 172.21.15.59:0/259081178 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561afb4729a0 con 0x561afb5013f0 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a16000640 10 client.15456 handle_command_reply: tid=0 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 client.15456 shutdown 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 msgr2=0x561afb48e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6a28011d10 tx=0x7f6a280096f0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 shutdown_connections 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561afb486020 0x561afb48a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561afb47b760 0x561afb489ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.685+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561afb5013f0 0x561afb5037e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a39fed740 1 --2- 172.21.15.59:0/259081178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561afb47ae40 0x561afb48e7b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 >> 172.21.15.59:0/259081178 conn(0x561afb4773d0 msgr2=0x561afb475c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a16000640 0 client.15456 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 shutdown_connections 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a39fed740 1 -- 172.21.15.59:0/259081178 wait complete. 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.686+0000 7f6a39fed740 20 client.15456 trim_cache size 0 max 16384 2024-09-05T11:58:15.692 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 msgr2=0x7f6a180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 0x7f6a180a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a240977f0 tx=0x7f6a240d5530 comp rx=0 tx=0).stop 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 msgr2=0x7f6a3019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400ea60 tx=0x7f6a2400ef20 comp rx=0 tx=0).stop 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 shutdown_connections 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6a180a2c00 0x7f6a180a50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a3010bfc0 0x7f6a3019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a301034b0 0x7f6a3019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 --2- 172.21.15.59:0/723807796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a30102b90 0x7f6a3019a4c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 >> 172.21.15.59:0/723807796 conn(0x7f6a300fc610 msgr2=0x7f6a3010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 shutdown_connections 2024-09-05T11:58:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:15.687+0000 7f6a35e00640 1 -- 172.21.15.59:0/723807796 wait complete. 2024-09-05T11:58:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:16 smithi089 ceph-mon[34499]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-05T11:58:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:16 smithi059 ceph-mon[30025]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-05T11:58:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:16 smithi189 ceph-mon[34772]: pgmap v412: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-05T11:58:17.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:58:17.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:58:18.146 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:18.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:18 smithi089 ceph-mon[34499]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:18 smithi059 ceph-mon[30025]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:18.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:18 smithi189 ceph-mon[34772]: pgmap v413: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:20 smithi089 ceph-mon[34499]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:20 smithi059 ceph-mon[30025]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:20 smithi189 ceph-mon[34772]: pgmap v414: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 -- 172.21.15.59:0/739702179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 msgr2=0x7f57a010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 --2- 172.21.15.59:0/739702179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a010dd40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f578c009900 tx=0x7f578c02a480 comp rx=0 tx=0).stop 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 -- 172.21.15.59:0/739702179 shutdown_connections 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 --2- 172.21.15.59:0/739702179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a010e280 0x7f57a0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 --2- 172.21.15.59:0/739702179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a010dd40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 --2- 172.21.15.59:0/739702179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a0074510 0x7f57a00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 -- 172.21.15.59:0/739702179 >> 172.21.15.59:0/739702179 conn(0x7f57a00fd740 msgr2=0x7f57a00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 -- 172.21.15.59:0/739702179 shutdown_connections 2024-09-05T11:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.742+0000 7f57a6a00640 1 -- 172.21.15.59:0/739702179 wait complete. 2024-09-05T11:58:20.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 Processor -- start 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 -- start start 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 0x7f57a019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a010e280 0x7f57a019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a0114000 con 0x7f57a0074e30 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57a0113e80 con 0x7f57a0074510 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.743+0000 7f57a6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57a0114180 con 0x7f57a010e280 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 0x7f57a019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53678/0 (socket says 172.21.15.59:53678) 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a4c00640 1 -- 172.21.15.59:0/4203878149 learned_addr learned my addr 172.21.15.59:0/4203878149 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:20.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a6000640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a010e280 0x7f57a019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 -- 172.21.15.59:0/4203878149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a010e280 msgr2=0x7f57a019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a010e280 0x7f57a019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 -- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 msgr2=0x7f57a019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.744+0000 7f57a5600640 1 -- 172.21.15.59:0/4203878149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a019f430 con 0x7f57a0074510 2024-09-05T11:58:20.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f57a5600640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 0x7f57a019a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f579400e9f0 tx=0x7f579400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f579e000640 1 -- 172.21.15.59:0/4203878149 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400ccc0 con 0x7f57a0074510 2024-09-05T11:58:20.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f579e000640 1 -- 172.21.15.59:0/4203878149 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5794010910 con 0x7f57a0074510 2024-09-05T11:58:20.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a019f720 con 0x7f57a0074510 2024-09-05T11:58:20.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f579e000640 1 -- 172.21.15.59:0/4203878149 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794018960 con 0x7f57a0074510 2024-09-05T11:58:20.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.745+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a01a7000 con 0x7f57a0074510 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.748+0000 7f57aa9d4740 1 Processor -- start 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.748+0000 7f579e000640 1 -- 172.21.15.59:0/4203878149 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5794010ac0 con 0x7f57a0074510 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.748+0000 7f57aa9d4740 1 -- start start 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb11950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb14280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bdfea6d290 con 0x55bdfeb06cb0 2024-09-05T11:58:20.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bdfea14b40 con 0x55bdfeb075d0 2024-09-05T11:58:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdfea69250 con 0x55bdfeb11e90 2024-09-05T11:58:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57a5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb11950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53684/0 (socket says 172.21.15.59:53684) 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.749+0000 7f57a6000640 1 -- 172.21.15.59:0/1851575071 learned_addr learned my addr 172.21.15.59:0/1851575071 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb14280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 -- 172.21.15.59:0/1851575071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 msgr2=0x55bdfeb14280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb14280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 -- 172.21.15.59:0/1851575071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 msgr2=0x55bdfeb11950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb11950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 -- 172.21.15.59:0/1851575071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfe7de6a0 con 0x55bdfeb06cb0 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a5600640 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb11950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:20.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f57a6000640 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5798009f30 tx=0x7f57980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eebe8fb086c27b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.750+0000 7f579e000640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.751+0000 7f579e000640 1 -- 172.21.15.59:0/4203878149 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f57940cf840 con 0x7f57a0074510 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.751+0000 7f5792600640 1 -- 172.21.15.59:0/1851575071 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798004d20 con 0x55bdfeb06cb0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.751+0000 7f5792600640 1 -- 172.21.15.59:0/1851575071 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798004050 con 0x55bdfeb06cb0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.751+0000 7f5792600640 1 -- 172.21.15.59:0/1851575071 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798005260 con 0x55bdfeb06cb0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.751+0000 7f57a5600640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1851575071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 msgr2=0x55bdfeb07090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57a5600640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f579409bc70 tx=0x7f57940d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5798009f30 tx=0x7f57980118d0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1851575071 shutdown_connections 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb14280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb11950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1851575071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb07090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1851575071 >> 172.21.15.59:0/1851575071 conn(0x55bdfeb03240 msgr2=0x55bdfeb03620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1851575071 shutdown_connections 2024-09-05T11:58:20.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.752+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1851575071 wait complete. 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.753+0000 7f57aa9d4740 1 -- start start 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.753+0000 7f57aa9d4740 1 Processor -- start 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bdfeb14b00 con 0x55bdfeb06cb0 2024-09-05T11:58:20.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bdfea27fa0 con 0x55bdfeb075d0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdfeb14c80 con 0x55bdfeb11e90 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57aa9d4740 10 client.0 _ll_register_callbacks cb 0x55bdfe77bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53700/0 (socket says 172.21.15.59:53700) 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57a6000640 1 -- 172.21.15.59:0/1197468217 learned_addr learned my addr 172.21.15.59:0/1197468217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.754+0000 7f57a6000640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 -- 172.21.15.59:0/1197468217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 msgr2=0x55bdfeb0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb0efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 -- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 msgr2=0x55bdfeb0ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb0ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 -- 172.21.15.59:0/1197468217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfeb0f7a0 con 0x55bdfeb06cb0 2024-09-05T11:58:20.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57a6000640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f579801b040 tx=0x7f5798004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b98439b849e991 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f5786a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579802b020 con 0x55bdfeb06cb0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5798005040 con 0x55bdfeb06cb0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.755+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5798004050 con 0x55bdfeb06cb0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.756+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5798036050 con 0x55bdfeb06cb0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.756+0000 7f57aa9d4740 10 client.15468 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.756+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bdfeb209e0 con 0x55bdfeb06cb0 2024-09-05T11:58:20.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.756+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798004050 con 0x55bdfeb06cb0 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.756+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5798027630 con 0x55bdfeb06cb0 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 10 client.15468 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 20 client.15468 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb21e40 0x55bdfeb24230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 4 client.15468 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdfea7a800 con 0x55bdfeb21e40 2024-09-05T11:58:20.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.757+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb21e40 0x55bdfeb24230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.758+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb21e40 0x55bdfeb24230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea92b30be00aa838 server_cookie=3892e18b83e05227 in_seq=0 out_seq=0 2024-09-05T11:58:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.758+0000 7f5786a00640 10 client.15468 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f5786a00640 1 -- 172.21.15.59:0/1197468217 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdfea7a800 con 0x55bdfeb21e40 2024-09-05T11:58:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f5786a00640 10 client.15468 handle_command_reply: tid=0 2024-09-05T11:58:20.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 client.15468 shutdown 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 msgr2=0x55bdfeb117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f579801b040 tx=0x7f5798004760 comp rx=0 tx=0).stop 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 shutdown_connections 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb21e40 0x55bdfeb24230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1197468217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb117b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 >> 172.21.15.59:0/1197468217 conn(0x55bdfeb03240 msgr2=0x55bdfeb08f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f5786a00640 0 client.15468 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 shutdown_connections 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1197468217 wait complete. 2024-09-05T11:58:20.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.759+0000 7f57aa9d4740 20 client.15468 trim_cache size 0 max 16384 2024-09-05T11:58:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.778+0000 7f57aa9d4740 1 Processor -- start 2024-09-05T11:58:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.778+0000 7f57aa9d4740 1 -- start start 2024-09-05T11:58:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb29e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01aef60 con 0x55bdfeb06cb0 2024-09-05T11:58:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01b17e0 con 0x55bdfeb075d0 2024-09-05T11:58:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01af5d0 con 0x55bdfeb11e90 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53710/0 (socket says 172.21.15.59:53710) 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46154/0 (socket says 172.21.15.59:46154) 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a6000640 1 -- 172.21.15.59:0/1412573939 learned_addr learned my addr 172.21.15.59:0/1412573939 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.779+0000 7f57a6000640 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb29e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 -- 172.21.15.59:0/1412573939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 msgr2=0x55bdfeb29e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb29e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 -- 172.21.15.59:0/1412573939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 msgr2=0x55bdfeb29950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 -- 172.21.15.59:0/1412573939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdfeb03d80 con 0x55bdfeb06cb0 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a4c00640 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f57a6000640 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5798005d70 tx=0x7f5798009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf72c07a0fad247b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f5791c00640 1 -- 172.21.15.59:0/1412573939 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579802b040 con 0x55bdfeb06cb0 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f5791c00640 1 -- 172.21.15.59:0/1412573939 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798027c20 con 0x55bdfeb06cb0 2024-09-05T11:58:20.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.780+0000 7f5791c00640 1 -- 172.21.15.59:0/1412573939 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798005040 con 0x55bdfeb06cb0 2024-09-05T11:58:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1412573939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 msgr2=0x55bdfeb0da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5798005d70 tx=0x7f5798009900 comp rx=0 tx=0).stop 2024-09-05T11:58:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1412573939 shutdown_connections 2024-09-05T11:58:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb11e90 0x55bdfeb29e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb075d0 0x55bdfeb29950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/1412573939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb0da40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1412573939 >> 172.21.15.59:0/1412573939 conn(0x55bdfeb03240 msgr2=0x55bdfeb0d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1412573939 shutdown_connections 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.781+0000 7f57aa9d4740 1 -- 172.21.15.59:0/1412573939 wait complete. 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 -- start start 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 Processor -- start 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb1a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 0x55bdfeb15d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01afca0 con 0x55bdfeb06cb0 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01b1110 con 0x55bdfeb11e90 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.782+0000 7f57aa9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57a01b0370 con 0x55bdfeb075d0 2024-09-05T11:58:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57aa9d4740 10 client.0 _ll_register_callbacks cb 0x7f57940d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:60746/0 (socket says 172.21.15.59:60746) 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 -- 172.21.15.59:0/3738233684 learned_addr learned my addr 172.21.15.59:0/3738233684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb1a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a6000640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 0x55bdfeb15d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 -- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 msgr2=0x55bdfeb15d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 0x55bdfeb15d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 -- 172.21.15.59:0/3738233684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 msgr2=0x55bdfeb1a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb1a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a4c00640 1 -- 172.21.15.59:0/3738233684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdfeb24720 con 0x55bdfeb075d0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.783+0000 7f57a6000640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 0x55bdfeb15d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f57a4c00640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f578c02aa70 tx=0x7f578c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1a9bce296f2567 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f5786a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:20.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c035040 con 0x55bdfeb075d0 2024-09-05T11:58:20.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f578c005220 con 0x55bdfeb075d0 2024-09-05T11:58:20.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5798027c20 con 0x55bdfeb075d0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f578c055050 con 0x55bdfeb075d0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f57aa9d4740 10 client.25423 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.784+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bdfeb8c650 con 0x55bdfeb075d0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798027c20 con 0x55bdfeb075d0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f578c031740 con 0x55bdfeb075d0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 10 client.25423 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 20 client.25423 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb8cde0 0x55bdfeb8f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 4 client.25423 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.785+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bdfea74d10 con 0x55bdfeb8cde0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.786+0000 7f57a6000640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb8cde0 0x55bdfeb8f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:20.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.786+0000 7f57a6000640 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb8cde0 0x55bdfeb8f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7476893253b39f server_cookie=ba3a6da03aff9e1c in_seq=0 out_seq=0 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.786+0000 7f5786a00640 10 client.25423 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f5786a00640 1 -- 172.21.15.59:0/3738233684 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bdfea74d10 con 0x55bdfeb8cde0 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f5786a00640 10 client.25423 handle_command_reply: tid=0 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 client.25423 shutdown 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 msgr2=0x55bdfeb157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f578c02aa70 tx=0x7f578c005c00 comp rx=0 tx=0).stop 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 shutdown_connections 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdfeb11e90 0x55bdfeb15d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdfeb075d0 0x55bdfeb157d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdfeb8cde0 0x55bdfeb8f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 --2- 172.21.15.59:0/3738233684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdfeb06cb0 0x55bdfeb1a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f5786a00640 0 client.25423 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.787+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 >> 172.21.15.59:0/3738233684 conn(0x55bdfeb03240 msgr2=0x55bdfeb0db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.788+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 shutdown_connections 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.788+0000 7f57aa9d4740 1 -- 172.21.15.59:0/3738233684 wait complete. 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.788+0000 7f57aa9d4740 20 client.25423 trim_cache size 0 max 16384 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "b7150b6f-8878-487f-8924-84b73cf49456", 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:58:20.792 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 msgr2=0x7f57880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f579409bc70 tx=0x7f57940d5530 comp rx=0 tx=0).stop 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 msgr2=0x7f57a019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 0x7f57a019a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f579400e9f0 tx=0x7f579400eeb0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 shutdown_connections 2024-09-05T11:58:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a010e280 0x7f57a019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a0074e30 0x7f57a019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 --2- 172.21.15.59:0/4203878149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a0074510 0x7f57a019a480 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 >> 172.21.15.59:0/4203878149 conn(0x7f57a00fd740 msgr2=0x7f57a00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 shutdown_connections 2024-09-05T11:58:20.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:20.789+0000 7f57a6a00640 1 -- 172.21.15.59:0/4203878149 wait complete. 2024-09-05T11:58:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:21 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:58:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:21 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:21 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:58:21.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:21 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:21 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:58:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:21 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:21 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:58:21.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:21 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:58:21.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:21 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:58:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:21 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:21 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:58:21.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:21 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:58:21.959 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:22.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:22 smithi089 ceph-mon[34499]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:22.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:22 smithi059 ceph-mon[30025]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:22.575 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 61% (7454/12088) Updating files: 62% (7495/12088) Updating files: 63% (7616/12088) Updating files: 64% (7737/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 98% (11896/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-05T11:58:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 -- 172.21.15.59:0/954294763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860077040 msgr2=0x7f58601124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 --2- 172.21.15.59:0/954294763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860077040 0x7f58601124f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f584c009900 tx=0x7f584c02a3f0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 -- 172.21.15.59:0/954294763 shutdown_connections 2024-09-05T11:58:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 --2- 172.21.15.59:0/954294763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860112a30 0x7f5860114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 --2- 172.21.15.59:0/954294763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860077040 0x7f58601124f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 --2- 172.21.15.59:0/954294763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860076720 0x7f5860076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 -- 172.21.15.59:0/954294763 >> 172.21.15.59:0/954294763 conn(0x7f58600ffa90 msgr2=0x7f5860101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 -- 172.21.15.59:0/954294763 shutdown_connections 2024-09-05T11:58:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.579+0000 7f5866800640 1 -- 172.21.15.59:0/954294763 wait complete. 2024-09-05T11:58:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.580+0000 7f5866800640 1 Processor -- start 2024-09-05T11:58:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.580+0000 7f5866800640 1 -- start start 2024-09-05T11:58:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860077040 0x7f58601a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860112a30 0x7f586019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5860118610 con 0x7f5860076720 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5860118490 con 0x7f5860077040 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f5866800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5860118790 con 0x7f5860112a30 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40078/0 (socket says 172.21.15.59:40078) 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f585f400640 1 -- 172.21.15.59:0/2239350184 learned_addr learned my addr 172.21.15.59:0/2239350184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f585ea00640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860077040 0x7f58601a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.581+0000 7f585fe00640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860112a30 0x7f586019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 -- 172.21.15.59:0/2239350184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860112a30 msgr2=0x7f586019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860112a30 0x7f586019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 -- 172.21.15.59:0/2239350184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860077040 msgr2=0x7f58601a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860077040 0x7f58601a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 -- 172.21.15.59:0/2239350184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586019f010 con 0x7f5860076720 2024-09-05T11:58:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f585f400640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f585000e9f0 tx=0x7f585000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.582+0000 7f5857e00640 1 -- 172.21.15.59:0/2239350184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585000cd80 con 0x7f5860076720 2024-09-05T11:58:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.583+0000 7f5857e00640 1 -- 172.21.15.59:0/2239350184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5850005500 con 0x7f5860076720 2024-09-05T11:58:22.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.583+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586019f300 con 0x7f5860076720 2024-09-05T11:58:22.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.583+0000 7f5857e00640 1 -- 172.21.15.59:0/2239350184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850010690 con 0x7f5860076720 2024-09-05T11:58:22.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.583+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601ab5f0 con 0x7f5860076720 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 Processor -- start 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 -- start start 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5857e00640 1 -- 172.21.15.59:0/2239350184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58500046c0 con 0x7f5860076720 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 0x55fabae34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae354b0 0x55fabae3f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fabad9b3e0 con 0x55fabae354b0 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fabad42d80 con 0x55fabae3fd70 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.586+0000 7f5869227740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fabad973a0 con 0x55fabae34b90 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 0x55fabae34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42010/0 (socket says 172.21.15.59:42010) 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 -- 172.21.15.59:0/1960207059 learned_addr learned my addr 172.21.15.59:0/1960207059 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585f400640 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae354b0 0x55fabae3f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 -- 172.21.15.59:0/1960207059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 msgr2=0x55fabae34f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 0x55fabae34f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 -- 172.21.15.59:0/1960207059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae354b0 msgr2=0x55fabae3f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae354b0 0x55fabae3f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585ea00640 1 -- 172.21.15.59:0/1960207059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fabad61ae0 con 0x55fabae3fd70 2024-09-05T11:58:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.587+0000 7f585fe00640 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 0x55fabae34f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f585ea00640 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f584c009900 tx=0x7f584c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c05c2c98c7bf1a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:22 smithi189 ceph-mon[34772]: pgmap v415: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f5857e00640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f5854c00640 1 -- 172.21.15.59:0/1960207059 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c035040 con 0x55fabae3fd70 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f5854c00640 1 -- 172.21.15.59:0/1960207059 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c005520 con 0x55fabae3fd70 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f5854c00640 1 -- 172.21.15.59:0/1960207059 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c003850 con 0x55fabae3fd70 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f5857e00640 1 -- 172.21.15.59:0/2239350184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58500cf8a0 con 0x7f5860076720 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.588+0000 7f585fe00640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f585fe00640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 0x7f58440a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f586006b460 tx=0x7f58580096a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 -- 172.21.15.59:0/1960207059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 msgr2=0x55fabae42160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f584c009900 tx=0x7f584c004040 comp rx=0 tx=0).stop 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 -- 172.21.15.59:0/1960207059 shutdown_connections 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae42160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae354b0 0x55fabae3f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 --2- 172.21.15.59:0/1960207059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae34b90 0x55fabae34f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 -- 172.21.15.59:0/1960207059 >> 172.21.15.59:0/1960207059 conn(0x55fabae31120 msgr2=0x55fabae31500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 -- 172.21.15.59:0/1960207059 shutdown_connections 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.589+0000 7f5869227740 1 -- 172.21.15.59:0/1960207059 wait complete. 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.590+0000 7f5869227740 1 -- start start 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.590+0000 7f5869227740 1 Processor -- start 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fabae42980 con 0x55fabae34b90 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fabad560f0 con 0x55fabae3fd70 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fabae42b00 con 0x55fabae354b0 2024-09-05T11:58:22.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f5869227740 10 client.0 _ll_register_callbacks cb 0x55fabad479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58360/0 (socket says 172.21.15.59:58360) 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 -- 172.21.15.59:0/258818339 learned_addr learned my addr 172.21.15.59:0/258818339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585fe00640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 -- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 msgr2=0x55fabae3d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae3d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 -- 172.21.15.59:0/258818339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 msgr2=0x55fabae3cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae3cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.591+0000 7f585ea00640 1 -- 172.21.15.59:0/258818339 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fabab0c3c0 con 0x55fabae354b0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.592+0000 7f585ea00640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f584c004980 tx=0x7f584c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93322346b48dea0e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.592+0000 7f5849600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.592+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c035050 con 0x55fabae354b0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.592+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584c005520 con 0x55fabae354b0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.592+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f584c004e60 con 0x55fabae354b0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f584c059050 con 0x55fabae354b0 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 10 client.25435 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fabae42980 con 0x55fabae354b0 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c005520 con 0x55fabae354b0 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f584c003ad0 con 0x55fabae354b0 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 10 client.25435 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 10 client.25435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 20 client.25435 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabae4f830 0x55fabae51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 4 client.25435 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.593+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fabada8950 con 0x55fabae4f830 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.594+0000 7f585f400640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabae4f830 0x55fabae51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.594+0000 7f585f400640 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabae4f830 0x55fabae51c20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddaaac3a6991ff0 server_cookie=7a8003ac6ae41b43 in_seq=0 out_seq=0 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.594+0000 7f5849600640 10 client.25435 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.595+0000 7f5849600640 1 -- 172.21.15.59:0/258818339 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fabada8950 con 0x55fabae4f830 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.595+0000 7f5849600640 10 client.25435 handle_command_reply: tid=0 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.595+0000 7f5869227740 1 client.25435 shutdown 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.595+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 msgr2=0x55fabae3d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.595+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f584c004980 tx=0x7f584c000bc0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 shutdown_connections 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae3d1e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabae4f830 0x55fabae51c20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 --2- 172.21.15.59:0/258818339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae3cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 >> 172.21.15.59:0/258818339 conn(0x55fabae31120 msgr2=0x55fabae36e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5849600640 0 client.25435 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 shutdown_connections 2024-09-05T11:58:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 1 -- 172.21.15.59:0/258818339 wait complete. 2024-09-05T11:58:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.596+0000 7f5869227740 20 client.25435 trim_cache size 0 max 16384 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 Processor -- start 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 -- start start 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b3460 con 0x55fabae34b90 2024-09-05T11:58:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b5ce0 con 0x55fabae3fd70 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f5869227740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b3ad0 con 0x55fabae354b0 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40120/0 (socket says 172.21.15.59:40120) 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f585f400640 1 -- 172.21.15.59:0/3451455365 learned_addr learned my addr 172.21.15.59:0/3451455365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f585fe00640 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.616+0000 7f585ea00640 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.617+0000 7f585fe00640 1 -- 172.21.15.59:0/3451455365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 msgr2=0x55fabae573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.617+0000 7f585fe00640 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.617+0000 7f585fe00640 1 -- 172.21.15.59:0/3451455365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 msgr2=0x55fabae375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.617+0000 7f585fe00640 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.617+0000 7f585fe00640 1 -- 172.21.15.59:0/3451455365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fabae3b1f0 con 0x55fabae3fd70 2024-09-05T11:58:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.618+0000 7f585fe00640 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae578f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f586007aa10 tx=0x7f5858019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd05bba532b10f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.618+0000 7f584be00640 1 -- 172.21.15.59:0/3451455365 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858010dd0 con 0x55fabae3fd70 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.619+0000 7f584be00640 1 -- 172.21.15.59:0/3451455365 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585801ee50 con 0x55fabae3fd70 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.619+0000 7f584be00640 1 -- 172.21.15.59:0/3451455365 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585801c3f0 con 0x55fabae3fd70 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.619+0000 7f5869227740 1 -- 172.21.15.59:0/3451455365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 msgr2=0x55fabae578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.619+0000 7f5869227740 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae578f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f586007aa10 tx=0x7f5858019000 comp rx=0 tx=0).stop 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 -- 172.21.15.59:0/3451455365 shutdown_connections 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae3fd70 0x55fabae578f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae354b0 0x55fabae573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 --2- 172.21.15.59:0/3451455365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 -- 172.21.15.59:0/3451455365 >> 172.21.15.59:0/3451455365 conn(0x55fabae31120 msgr2=0x55fabae3ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 -- 172.21.15.59:0/3451455365 shutdown_connections 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 -- 172.21.15.59:0/3451455365 wait complete. 2024-09-05T11:58:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 -- start start 2024-09-05T11:58:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.620+0000 7f5869227740 1 Processor -- start 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 0x55fabaeb2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b41a0 con 0x55fabae34b90 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b5610 con 0x55fabae354b0 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58601b4870 con 0x55fabae3fd70 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f5869227740 10 client.0 _ll_register_callbacks cb 0x7f584c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f585f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f585ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f585fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 0x55fabaeb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f585ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42030/0 (socket says 172.21.15.59:42030) 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.621+0000 7f585ea00640 1 -- 172.21.15.59:0/2292993784 learned_addr learned my addr 172.21.15.59:0/2292993784 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 -- 172.21.15.59:0/2292993784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 msgr2=0x55fabaeb2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 0x55fabaeb2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 -- 172.21.15.59:0/2292993784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 msgr2=0x55fabae446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 -- 172.21.15.59:0/2292993784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fabaeb2b70 con 0x55fabae354b0 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585fe00640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 0x55fabaeb2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f585ea00640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f584c031550 tx=0x7f584c031470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254ec13d95a6fdd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f5849600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c035040 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585801ee50 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.622+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f584c031a20 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f584c009520 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 10 client.25508 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fabaebae90 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585801ee50 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f584c004c90 con 0x55fabae354b0 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 10 client.25508 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 20 client.25508 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabaebb470 0x55fabaebd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 4 client.25508 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.623+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fabada2e60 con 0x55fabaebb470 2024-09-05T11:58:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f585f400640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabaebb470 0x55fabaebd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f585f400640 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabaebb470 0x55fabaebd890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26ac3ee1525761 server_cookie=f7a57e8445982ff6 in_seq=0 out_seq=0 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f5849600640 10 client.25508 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f5849600640 1 -- 172.21.15.59:0/2292993784 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fabada2e60 con 0x55fabaebb470 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f5849600640 10 client.25508 handle_command_reply: tid=0 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.624+0000 7f5869227740 1 client.25508 shutdown 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 msgr2=0x55fabae44bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f584c031550 tx=0x7f584c031470 comp rx=0 tx=0).stop 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 shutdown_connections 2024-09-05T11:58:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fabae3fd70 0x55fabaeb2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fabae354b0 0x55fabae44bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fabaebb470 0x55fabaebd890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 --2- 172.21.15.59:0/2292993784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fabae34b90 0x55fabae446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 >> 172.21.15.59:0/2292993784 conn(0x55fabae31120 msgr2=0x55fabae3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5849600640 0 client.25508 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 shutdown_connections 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 1 -- 172.21.15.59:0/2292993784 wait complete. 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.625+0000 7f5869227740 20 client.25508 trim_cache size 0 max 16384 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:22.630 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: "b7150b6f-8878-487f-8924-84b73cf49456": { 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "b7150b6f-8878-487f-8924-84b73cf49456", 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:22.631 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 msgr2=0x7f58440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 0x7f58440a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f586006b460 tx=0x7f58580096a0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 msgr2=0x7f58601a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f585000e9f0 tx=0x7f585000edf0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 shutdown_connections 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5860112a30 0x7f586019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5860077040 0x7f58601a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 --2- 172.21.15.59:0/2239350184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5860076720 0x7f58601a4720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 >> 172.21.15.59:0/2239350184 conn(0x7f58600ffa90 msgr2=0x7f5860101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 shutdown_connections 2024-09-05T11:58:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:22.626+0000 7f5866800640 1 -- 172.21.15.59:0/2239350184 wait complete. 2024-09-05T11:58:23.476 INFO:tasks.cephfs.filesystem:scrub status for tag:b7150b6f-8878-487f-8924-84b73cf49456 - {'path': '/', 'tag': 'b7150b6f-8878-487f-8924-84b73cf49456', 'options': 'recursive,force'} 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr:Note: switching to 'e856d6b2462509a377ef39149a05475ed006ebac'. 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by switching back to a branch. 2024-09-05T11:58:23.746 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: git switch -c 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr:Or undo this operation with: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: git switch - 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-09-05T11:58:23.747 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at e856d6b2462 Merge branch 'wip-67307-squid' of https://github.com/cbodley/ceph into wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T11:58:23.759 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-05T11:58:23.802 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-05T11:58:23.807 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T11:58:23.807 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-05T11:58:24.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:24 smithi089 ceph-mon[34499]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T11:58:24.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:24 smithi059 ceph-mon[30025]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T11:58:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:24 smithi189 ceph-mon[34772]: pgmap v416: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T11:58:24.620 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-05T11:58:24.714 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-05T11:58:24.780 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T11:58:24.783 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T11:58:24.783 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-05T11:58:24.849 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T11:58:24.857 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T11:58:24.857 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-05T11:58:24.933 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-05T11:58:24.934 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-05T11:58:24.935 DEBUG:teuthology.orchestra.run.smithi059:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e856d6b2462509a377ef39149a05475ed006ebac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-05T11:58:25.023 INFO:tasks.workunit.client.0.smithi059.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-05T11:58:25.024 INFO:tasks.workunit.client.0.smithi059.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-05T11:58:25.025 INFO:tasks.workunit.client.0.smithi059.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-05T11:58:25.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:25 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:58:25.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:25 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:58:25.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:25 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:58:26.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:26 smithi189 ceph-mon[34772]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:26 smithi089 ceph-mon[34499]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:26 smithi059 ceph-mon[30025]: pgmap v417: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T11:58:28.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:28 smithi189 ceph-mon[34772]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T11:58:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:28 smithi089 ceph-mon[34499]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T11:58:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:28 smithi059 ceph-mon[30025]: pgmap v418: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T11:58:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:30 smithi089 ceph-mon[34499]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-05T11:58:30.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:30 smithi059 ceph-mon[30025]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-05T11:58:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:30 smithi189 ceph-mon[34772]: pgmap v419: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-05T11:58:32.939 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:32 smithi089 ceph-mon[34499]: pgmap v420: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 4 op/s 2024-09-05T11:58:32.964 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:32 smithi059 ceph-mon[30025]: pgmap v420: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 4 op/s 2024-09-05T11:58:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:32 smithi189 ceph-mon[34772]: pgmap v420: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 4 op/s 2024-09-05T11:58:34.856 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T11:58:34.856 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-05T11:58:34.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:34.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:34 smithi059 ceph-mon[30025]: pgmap v421: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:34 smithi089 ceph-mon[34499]: pgmap v421: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:34 smithi189 ceph-mon[34772]: pgmap v421: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:35.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 -- 172.21.15.59:0/881107111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28102790 msgr2=0x7f4d281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:35.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 --2- 172.21.15.59:0/881107111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28102790 0x7f4d281067e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009900 tx=0x7f4d1002a410 comp rx=0 tx=0).stop 2024-09-05T11:58:35.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 -- 172.21.15.59:0/881107111 shutdown_connections 2024-09-05T11:58:35.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 --2- 172.21.15.59:0/881107111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28106f10 0x7f4d2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 --2- 172.21.15.59:0/881107111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28102790 0x7f4d281067e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 --2- 172.21.15.59:0/881107111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d28101de0 0x7f4d281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 -- 172.21.15.59:0/881107111 >> 172.21.15.59:0/881107111 conn(0x7f4d280fd8c0 msgr2=0x7f4d280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 -- 172.21.15.59:0/881107111 shutdown_connections 2024-09-05T11:58:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.833+0000 7f4d2d200640 1 -- 172.21.15.59:0/881107111 wait complete. 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.834+0000 7f4d2d200640 1 Processor -- start 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.834+0000 7f4d2d200640 1 -- start start 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28102790 0x7f4d2819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d28106f10 0x7f4d2819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111f80 con 0x7f4d28101de0 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28111e00 con 0x7f4d28102790 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d2d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28112100 con 0x7f4d28106f10 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37710/0 (socket says 172.21.15.59:37710) 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 -- 172.21.15.59:0/4116759226 learned_addr learned my addr 172.21.15.59:0/4116759226 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d1d400640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28102790 0x7f4d2819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 -- 172.21.15.59:0/4116759226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d28106f10 msgr2=0x7f4d2819ef60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d28106f10 0x7f4d2819ef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 -- 172.21.15.59:0/4116759226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28102790 msgr2=0x7f4d2819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28102790 0x7f4d2819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.835+0000 7f4d25e00640 1 -- 172.21.15.59:0/4116759226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2819f640 con 0x7f4d28101de0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d25e00640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d950 tx=0x7f4d1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18017030 con 0x7f4d28101de0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2819f930 con 0x7f4d28101de0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18004020 con 0x7f4d28101de0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a7210 con 0x7f4d28101de0 2024-09-05T11:58:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.836+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d180047a0 con 0x7f4d28101de0 2024-09-05T11:58:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.838+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d281021c0 con 0x7f4d28101de0 2024-09-05T11:58:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.839+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d180041d0 con 0x7f4d28101de0 2024-09-05T11:58:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.840+0000 7f4d1f400640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 0x7f4d0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.841+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4d180ced90 con 0x7f4d28101de0 2024-09-05T11:58:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.841+0000 7f4d1d400640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 0x7f4d0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:35.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.841+0000 7f4d1d400640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 0x7f4d0c0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000c00 tx=0x7f4d10005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:35.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:35.845+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d1809b880 con 0x7f4d28101de0 2024-09-05T11:58:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.069+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d281a7440 con 0x7f4d28101de0 2024-09-05T11:58:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.070+0000 7f4d1f400640 1 -- 172.21.15.59:0/4116759226 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f4d1809b180 con 0x7f4d28101de0 2024-09-05T11:58:36.072 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:58:36.072 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:58:36.073 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:58:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 msgr2=0x7f4d0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 0x7f4d0c0a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000c00 tx=0x7f4d10005b50 comp rx=0 tx=0).stop 2024-09-05T11:58:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 msgr2=0x7f4d2819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d950 tx=0x7f4d1800de10 comp rx=0 tx=0).stop 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 shutdown_connections 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d0c0a2b80 0x7f4d0c0a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d28106f10 0x7f4d2819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d28102790 0x7f4d2819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 --2- 172.21.15.59:0/4116759226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28101de0 0x7f4d2819a690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.074+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 >> 172.21.15.59:0/4116759226 conn(0x7f4d280fd8c0 msgr2=0x7f4d280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.075+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 shutdown_connections 2024-09-05T11:58:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:36.075+0000 7f4d2d200640 1 -- 172.21.15.59:0/4116759226 wait complete. 2024-09-05T11:58:36.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:36 smithi059 ceph-mon[30025]: pgmap v422: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:36.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:36 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4116759226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:36.917 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-05T11:58:36.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-05T11:58:36.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:36 smithi089 ceph-mon[34499]: pgmap v422: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:36.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:36 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4116759226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:36 smithi189 ceph-mon[34772]: pgmap v422: 129 pgs: 129 active+clean; 710 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-09-05T11:58:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:36 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4116759226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:58:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.709 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:37 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 -- 172.21.15.59:0/3159609635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98103ff0 msgr2=0x7fec981043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 --2- 172.21.15.59:0/3159609635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98103ff0 0x7fec981043d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fec88009900 tx=0x7fec8802a410 comp rx=0 tx=0).stop 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 -- 172.21.15.59:0/3159609635 shutdown_connections 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 --2- 172.21.15.59:0/3159609635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98109120 0x7fec98110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 --2- 172.21.15.59:0/3159609635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec981089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 --2- 172.21.15.59:0/3159609635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98103ff0 0x7fec981043d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 -- 172.21.15.59:0/3159609635 >> 172.21.15.59:0/3159609635 conn(0x7fec980ffab0 msgr2=0x7fec98101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 -- 172.21.15.59:0/3159609635 shutdown_connections 2024-09-05T11:58:37.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.903+0000 7fec9ec00640 1 -- 172.21.15.59:0/3159609635 wait complete. 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.904+0000 7fec9ec00640 1 Processor -- start 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 -- start start 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec9819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98109120 0x7fec9819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec98114120 con 0x7fec98109120 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fec98113fa0 con 0x7fec981049a0 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.905+0000 7fec9ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec981142a0 con 0x7fec98103ff0 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:42660/0 (socket says 172.21.15.59:42660) 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97e00640 1 -- 172.21.15.59:0/2884540649 learned_addr learned my addr 172.21.15.59:0/2884540649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec9819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec9cc00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98109120 0x7fec9819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 -- 172.21.15.59:0/2884540649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 msgr2=0x7fec9819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 -- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98109120 msgr2=0x7fec9819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98109120 0x7fec9819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 -- 172.21.15.59:0/2884540649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819f6b0 con 0x7fec981049a0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97e00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.906+0000 7fec97400640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec9819ac40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d6d0 tx=0x7fec8400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.907+0000 7fec94c00640 1 -- 172.21.15.59:0/2884540649 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84017070 con 0x7fec981049a0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.907+0000 7fec94c00640 1 -- 172.21.15.59:0/2884540649 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec84009820 con 0x7fec981049a0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.907+0000 7fec94c00640 1 -- 172.21.15.59:0/2884540649 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84004d00 con 0x7fec981049a0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.907+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819f9a0 con 0x7fec981049a0 2024-09-05T11:58:37.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.907+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a7280 con 0x7fec981049a0 2024-09-05T11:58:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.910+0000 7feca1807740 1 Processor -- start 2024-09-05T11:58:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.910+0000 7feca1807740 1 -- start start 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 0x56227fcac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fcae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56227fc07530 con 0x56227fcac590 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56227fbae860 con 0x56227fca13b0 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7feca1807740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56227fc034f0 con 0x56227fca1cd0 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 0x56227fcac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 0x56227fcac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:42676/0 (socket says 172.21.15.59:42676) 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec97e00640 1 -- 172.21.15.59:0/812368656 learned_addr learned my addr 172.21.15.59:0/812368656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 -- 172.21.15.59:0/812368656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 msgr2=0x56227fcac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 0x56227fcac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 -- 172.21.15.59:0/812368656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 msgr2=0x56227fcae980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T11:58:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fcae980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.911+0000 7fec9cc00640 1 -- 172.21.15.59:0/812368656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227f978b90 con 0x56227fca13b0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.912+0000 7fec9cc00640 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca1790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec90009da0 tx=0x7fec90010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ea56a9e8197b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.912+0000 7fec8d200640 1 -- 172.21.15.59:0/812368656 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90016070 con 0x56227fca13b0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.912+0000 7fec8d200640 1 -- 172.21.15.59:0/812368656 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90010e50 con 0x56227fca13b0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.912+0000 7fec8d200640 1 -- 172.21.15.59:0/812368656 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900127d0 con 0x56227fca13b0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.912+0000 7fec94c00640 1 -- 172.21.15.59:0/2884540649 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec840099d0 con 0x7fec981049a0 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- 172.21.15.59:0/812368656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 msgr2=0x56227fca1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca1790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec90009da0 tx=0x7fec90010740 comp rx=0 tx=0).stop 2024-09-05T11:58:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- 172.21.15.59:0/812368656 shutdown_connections 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fcae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca1cd0 0x56227fcac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 --2- 172.21.15.59:0/812368656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca1790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- 172.21.15.59:0/812368656 >> 172.21.15.59:0/812368656 conn(0x56227fc9d940 msgr2=0x56227fc9dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- 172.21.15.59:0/812368656 shutdown_connections 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- 172.21.15.59:0/812368656 wait complete. 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.913+0000 7feca1807740 1 -- start start 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.914+0000 7fec94c00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 0x7fec7c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.914+0000 7feca1807740 1 Processor -- start 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.914+0000 7fec94c00640 1 -- 172.21.15.59:0/2884540649 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec840cf020 con 0x7fec981049a0 2024-09-05T11:58:37.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.914+0000 7fec9cc00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 0x7fec7c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fca9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 0x56227fcaa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56227fcaf100 con 0x56227fca1cd0 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56227fbc1e60 con 0x56227fca13b0 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56227fcaf280 con 0x56227fcac590 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7feca1807740 10 client.0 _ll_register_callbacks cb 0x56227fbc6cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53238/0 (socket says 172.21.15.59:53238) 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fca9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 -- 172.21.15.59:0/1174277282 learned_addr learned my addr 172.21.15.59:0/1174277282 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec9cc00640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 0x56227fcaa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec9cc00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 0x7fec7c0a4f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec900059d0 tx=0x7fec90005a30 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 -- 172.21.15.59:0/1174277282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 msgr2=0x56227fcaa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 0x56227fcaa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 -- 172.21.15.59:0/1174277282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 msgr2=0x56227fca9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fca9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.915+0000 7fec97e00640 1 -- 172.21.15.59:0/1174277282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227fbcd990 con 0x56227fca13b0 2024-09-05T11:58:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7fec9cc00640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 0x56227fcaa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7fec97e00640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec88031990 tx=0x7fec880319c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d9dcff35115eff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7fec81600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88035040 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec90010e50 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.916+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec880042c0 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fec880044c0 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 10 client.25526 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56227fcc3730 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90010e50 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fec880035f0 con 0x56227fca13b0 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 10 client.25526 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:37.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 10 client.25526 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 20 client.25526 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fcc4b40 0x56227fcc6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 4 client.25526 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.917+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56227fbaa310 con 0x56227fcc4b40 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.918+0000 7fec97400640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fcc4b40 0x56227fcc6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.918+0000 7fec97400640 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fcc4b40 0x56227fcc6f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a4b55a926d1afc server_cookie=cc1276e1702f2380 in_seq=0 out_seq=0 2024-09-05T11:58:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.918+0000 7fec81600640 10 client.25526 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7fec81600640 1 -- 172.21.15.59:0/1174277282 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56227fbaa310 con 0x56227fcc4b40 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7fec81600640 10 client.25526 handle_command_reply: tid=0 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 client.25526 shutdown 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 msgr2=0x56227fca9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fec88031990 tx=0x7fec880319c0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 shutdown_connections 2024-09-05T11:58:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fcac590 0x56227fcaa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fcc4b40 0x56227fcc6f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fca9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 --2- 172.21.15.59:0/1174277282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca13b0 0x56227fca9660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7fec81600640 0 client.25526 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.919+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 >> 172.21.15.59:0/1174277282 conn(0x56227fc9d940 msgr2=0x56227fc9cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.920+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 shutdown_connections 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.920+0000 7feca1807740 1 -- 172.21.15.59:0/1174277282 wait complete. 2024-09-05T11:58:37.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.920+0000 7feca1807740 20 client.25526 trim_cache size 0 max 16384 2024-09-05T11:58:37.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:58:37.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:37 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.939+0000 7feca1807740 1 Processor -- start 2024-09-05T11:58:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.939+0000 7feca1807740 1 -- start start 2024-09-05T11:58:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.939+0000 7feca1807740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fccc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7feca1807740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7feca1807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fca68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7feca1807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec981aad10 con 0x56227fcac590 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7feca1807740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fec981ad590 con 0x56227fca1cd0 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7feca1807740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec981ab380 con 0x56227fca13b0 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53242/0 (socket says 172.21.15.59:53242) 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 -- 172.21.15.59:0/3174594928 learned_addr learned my addr 172.21.15.59:0/3174594928 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97e00640 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fccc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec9cc00640 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fca68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 -- 172.21.15.59:0/3174594928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 msgr2=0x56227fccc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fccc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 -- 172.21.15.59:0/3174594928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 msgr2=0x56227fca68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fca68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 -- 172.21.15.59:0/3174594928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227fca8630 con 0x56227fca1cd0 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.940+0000 7fec97400640 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec840ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a984d9a3fbc8907b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.941+0000 7fec83e00640 1 -- 172.21.15.59:0/3174594928 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840b4070 con 0x56227fca1cd0 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.941+0000 7fec83e00640 1 -- 172.21.15.59:0/3174594928 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec840a40e0 con 0x56227fca1cd0 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.941+0000 7fec83e00640 1 -- 172.21.15.59:0/3174594928 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84020040 con 0x56227fca1cd0 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- 172.21.15.59:0/3174594928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 msgr2=0x56227fcccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec840ce040 comp rx=0 tx=0).stop 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- 172.21.15.59:0/3174594928 shutdown_connections 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fcac590 0x56227fca68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fca1cd0 0x56227fcccd00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 --2- 172.21.15.59:0/3174594928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fccc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- 172.21.15.59:0/3174594928 >> 172.21.15.59:0/3174594928 conn(0x56227fc9d940 msgr2=0x56227fc9cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- 172.21.15.59:0/3174594928 shutdown_connections 2024-09-05T11:58:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- 172.21.15.59:0/3174594928 wait complete. 2024-09-05T11:58:37.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.942+0000 7feca1807740 1 -- start start 2024-09-05T11:58:37.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 Processor -- start 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fcbd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fcbd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec981aba50 con 0x56227fca1cd0 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fec981acec0 con 0x56227fcac590 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec981ac120 con 0x56227fca13b0 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7feca1807740 10 client.0 _ll_register_callbacks cb 0x7fec88009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7fec97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fcbd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.943+0000 7fec9cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec9cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53256/0 (socket says 172.21.15.59:53256) 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec9cc00640 1 -- 172.21.15.59:0/1882326075 learned_addr learned my addr 172.21.15.59:0/1882326075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fcbd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 -- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 msgr2=0x56227fcb96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 -- 172.21.15.59:0/1882326075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 msgr2=0x56227fcbd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fcbd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 -- 172.21.15.59:0/1882326075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227fcc72a0 con 0x56227fca13b0 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec9cc00640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec97e00640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fcbd140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec88031550 tx=0x7fec88005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5db81032496f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.944+0000 7fec81600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88035070 con 0x56227fca13b0 2024-09-05T11:58:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec88031a30 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec840a40e0 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fec88031c30 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 10 client.25462 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56227fd30260 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec840a40e0 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fec88005120 con 0x56227fca13b0 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 10 client.25462 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 10 client.25462 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 20 client.25462 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fd30a40 0x56227fd32e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.945+0000 7feca1807740 4 client.25462 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-05T11:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.946+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x56227fc14c60 con 0x56227fd30a40 2024-09-05T11:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.946+0000 7fec97400640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fd30a40 0x56227fd32e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.946+0000 7fec97400640 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fd30a40 0x56227fd32e60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d373cfaf6674b8 server_cookie=3d36b959b2ab641b in_seq=0 out_seq=0 2024-09-05T11:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.946+0000 7fec81600640 10 client.25462 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7fec81600640 1 -- 172.21.15.59:0/1882326075 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56227fc14c60 con 0x56227fd30a40 2024-09-05T11:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7fec81600640 10 client.25462 handle_command_reply: tid=0 2024-09-05T11:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 client.25462 shutdown 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 msgr2=0x56227fcbd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fcbd140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec88031550 tx=0x7fec88005c00 comp rx=0 tx=0).stop 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 shutdown_connections 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56227fcac590 0x56227fcb96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56227fd30a40 0x56227fd32e60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56227fca1cd0 0x56227fcbd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 --2- 172.21.15.59:0/1882326075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56227fca13b0 0x56227fcbd140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 >> 172.21.15.59:0/1882326075 conn(0x56227fc9d940 msgr2=0x56227fca9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7fec81600640 0 client.25462 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 shutdown_connections 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 1 -- 172.21.15.59:0/1882326075 wait complete. 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.956+0000 7feca1807740 20 client.25462 trim_cache size 0 max 16384 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:58:37.959 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 6, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "f662b9f6": { 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 6, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 137.7, 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:58:37.960 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:37.961 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 msgr2=0x7fec7c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 0x7fec7c0a4f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec900059d0 tx=0x7fec90005a30 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 msgr2=0x7fec9819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec9819ac40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec8400d6d0 tx=0x7fec8400db90 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 shutdown_connections 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fec7c0a2a90 0x7fec7c0a4f50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec98109120 0x7fec9819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec981049a0 0x7fec9819ac40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 --2- 172.21.15.59:0/2884540649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec98103ff0 0x7fec9819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:37.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.957+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 >> 172.21.15.59:0/2884540649 conn(0x7fec980ffab0 msgr2=0x7fec98100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:37.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.958+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 shutdown_connections 2024-09-05T11:58:37.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:37.958+0000 7fec9ec00640 1 -- 172.21.15.59:0/2884540649 wait complete. 2024-09-05T11:58:38.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:58:38.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:38.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:38.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:38.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:37 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:38 smithi059 ceph-mon[30025]: pgmap v423: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T11:58:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:58:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:58:38.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:38.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f662b9f6', quiesce took 4.0 seconds. Will release after: 13.7 2024-09-05T11:58:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:38 smithi089 ceph-mon[34499]: pgmap v423: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T11:58:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:58:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:58:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:38 smithi189 ceph-mon[34772]: pgmap v423: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T11:58:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:58:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:58:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:58:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:40 smithi089 ceph-mon[34499]: pgmap v424: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:40 smithi059 ceph-mon[30025]: pgmap v424: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:40 smithi189 ceph-mon[34772]: pgmap v424: 129 pgs: 129 active+clean; 770 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:42.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:42 smithi089 ceph-mon[34499]: pgmap v425: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-05T11:58:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:42 smithi059 ceph-mon[30025]: pgmap v425: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-05T11:58:43.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:42 smithi189 ceph-mon[34772]: pgmap v425: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-05T11:58:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:44 smithi089 ceph-mon[34499]: pgmap v426: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T11:58:44.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:44 smithi059 ceph-mon[30025]: pgmap v426: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T11:58:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:44 smithi189 ceph-mon[34772]: pgmap v426: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T11:58:46.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:46 smithi059 ceph-mon[30025]: pgmap v427: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-09-05T11:58:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:46 smithi089 ceph-mon[34499]: pgmap v427: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-09-05T11:58:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:46 smithi189 ceph-mon[34772]: pgmap v427: 129 pgs: 129 active+clean; 803 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.5 KiB/s wr, 5 op/s 2024-09-05T11:58:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:48 smithi089 ceph-mon[34499]: pgmap v428: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:48 smithi059 ceph-mon[30025]: pgmap v428: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:48 smithi189 ceph-mon[34772]: pgmap v428: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:58:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:50 smithi089 ceph-mon[34499]: pgmap v429: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T11:58:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:50 smithi059 ceph-mon[30025]: pgmap v429: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T11:58:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:50 smithi189 ceph-mon[34772]: pgmap v429: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T11:58:52.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:52 smithi089 ceph-mon[34499]: pgmap v430: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T11:58:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:52 smithi059 ceph-mon[30025]: pgmap v430: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T11:58:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:52 smithi189 ceph-mon[34772]: pgmap v430: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T11:58:53.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 -- 172.21.15.59:0/2929802866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4102de0 msgr2=0x7f24d41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2929802866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4102de0 0x7f24d41031c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009900 tx=0x7f24c402a480 comp rx=0 tx=0).stop 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 -- 172.21.15.59:0/2929802866 shutdown_connections 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2929802866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d410c180 0x7f24d410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2929802866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d4103700 0x7f24d410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2929802866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4102de0 0x7f24d41031c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 -- 172.21.15.59:0/2929802866 >> 172.21.15.59:0/2929802866 conn(0x7f24d40fc7a0 msgr2=0x7f24d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 -- 172.21.15.59:0/2929802866 shutdown_connections 2024-09-05T11:58:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.493+0000 7f24d8c00640 1 -- 172.21.15.59:0/2929802866 wait complete. 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.494+0000 7f24d8c00640 1 Processor -- start 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.494+0000 7f24d8c00640 1 -- start start 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 0x7f24d419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 0x7f24d419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4111f80 con 0x7f24d410c180 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4111e00 con 0x7f24d4102de0 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4112100 con 0x7f24d4103700 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:57412/0 (socket says 172.21.15.59:57412) 2024-09-05T11:58:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d1e00640 1 -- 172.21.15.59:0/2191831595 learned_addr learned my addr 172.21.15.59:0/2191831595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d2800640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 0x7f24d419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.495+0000 7f24d1400640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 0x7f24d419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 -- 172.21.15.59:0/2191831595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 msgr2=0x7f24d419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 -- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 msgr2=0x7f24d419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 0x7f24d419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 -- 172.21.15.59:0/2191831595 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419f5e0 con 0x7f24d4103700 2024-09-05T11:58:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1e00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d2800640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d1400640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 0x7f24d419ab70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9f0 tx=0x7f24c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000cd80 con 0x7f24d4103700 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c00035f0 con 0x7f24d4103700 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419f8d0 con 0x7f24d4103700 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0010670 con 0x7f24d4103700 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.496+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a71b0 con 0x7f24d4103700 2024-09-05T11:58:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.497+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d41031c0 con 0x7f24d4103700 2024-09-05T11:58:53.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.500+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24c00037a0 con 0x7f24d4103700 2024-09-05T11:58:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.501+0000 7f24caa00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 0x7f24b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.502+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24c00cf820 con 0x7f24d4103700 2024-09-05T11:58:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.502+0000 7f24d1e00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 0x7f24b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:53.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.503+0000 7f24d1e00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 0x7f24b80a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24c40096c0 tx=0x7f24c40046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:53.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.503+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24c009c3a0 con 0x7f24d4103700 2024-09-05T11:58:53.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.731+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24d419b990 con 0x7f24d4103700 2024-09-05T11:58:53.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.732+0000 7f24caa00640 1 -- 172.21.15.59:0/2191831595 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f24d419b990 con 0x7f24d4103700 2024-09-05T11:58:53.734 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:58:53.734 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:58:53.735 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:58:53.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 msgr2=0x7f24b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:53.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 0x7f24b80a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24c40096c0 tx=0x7f24c40046c0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 msgr2=0x7f24d419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 0x7f24d419ab70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9f0 tx=0x7f24c000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 shutdown_connections 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24b80a2b80 0x7f24b80a5040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d410c180 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24d4103700 0x7f24d419ab70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 --2- 172.21.15.59:0/2191831595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102de0 0x7f24d419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 >> 172.21.15.59:0/2191831595 conn(0x7f24d40fc7a0 msgr2=0x7f24d410d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.735+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 shutdown_connections 2024-09-05T11:58:53.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:53.736+0000 7f24d8c00640 1 -- 172.21.15.59:0/2191831595 wait complete. 2024-09-05T11:58:54.577 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id f662b9f6 --release --await' 2024-09-05T11:58:54.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id f662b9f6 --release --await 2024-09-05T11:58:54.604 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:54.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:54 smithi059 ceph-mon[30025]: pgmap v431: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:54.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:54 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2191831595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:54 smithi089 ceph-mon[34499]: pgmap v431: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:54 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2191831595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:54 smithi189 ceph-mon[34772]: pgmap v431: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:54 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2191831595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:58:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 -- 172.21.15.59:0/2774530973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1538109140 msgr2=0x7f1538110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 --2- 172.21.15.59:0/2774530973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1538109140 0x7f1538110cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1530009ca0 tx=0x7f153002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 -- 172.21.15.59:0/2774530973 shutdown_connections 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 --2- 172.21.15.59:0/2774530973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1538109140 0x7f1538110cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 --2- 172.21.15.59:0/2774530973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15381049c0 0x7f1538108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 --2- 172.21.15.59:0/2774530973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 0x7f15381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 -- 172.21.15.59:0/2774530973 >> 172.21.15.59:0/2774530973 conn(0x7f15380ffa50 msgr2=0x7f1538101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 -- 172.21.15.59:0/2774530973 shutdown_connections 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.643+0000 7f153d200640 1 -- 172.21.15.59:0/2774530973 wait complete. 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 Processor -- start 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 -- start start 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 0x7f153819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15381049c0 0x7f153819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1538114150 con 0x7f1538104010 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1538113fd0 con 0x7f1538109140 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.644+0000 7f153d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15381142d0 con 0x7f15381049c0 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:57426/0 (socket says 172.21.15.59:57426) 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 -- 172.21.15.59:0/606778793 learned_addr learned my addr 172.21.15.59:0/606778793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1535400640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15381049c0 0x7f153819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1535e00640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 0x7f153819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 -- 172.21.15.59:0/606778793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15381049c0 msgr2=0x7f153819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15381049c0 0x7f153819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 -- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 msgr2=0x7f153819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 0x7f153819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 -- 172.21.15.59:0/606778793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819f5a0 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.645+0000 7f1536800640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1530031a20 tx=0x7f1530031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.646+0000 7f152ea00640 1 -- 172.21.15.59:0/606778793 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530003b70 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.646+0000 7f152ea00640 1 -- 172.21.15.59:0/606778793 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1530003d10 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.646+0000 7f152ea00640 1 -- 172.21.15.59:0/606778793 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153002fc40 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.646+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819f830 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.646+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381a7170 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.649+0000 7f152ea00640 1 -- 172.21.15.59:0/606778793 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1530004050 con 0x7f1538109140 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.649+0000 7f153fe50740 1 Processor -- start 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.649+0000 7f153fe50740 1 -- start start 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f8b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78f8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd78ee73e0 con 0x55dd78f8bd70 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd78e8ed80 con 0x55dd78f814b0 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f153fe50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd78ee33a0 con 0x55dd78f80b90 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:32840/0 (socket says 172.21.15.59:32840) 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 -- 172.21.15.59:0/2076198409 learned_addr learned my addr 172.21.15.59:0/2076198409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78f8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535e00640 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f8b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 -- 172.21.15.59:0/2076198409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 msgr2=0x55dd78f8b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f8b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 -- 172.21.15.59:0/2076198409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 msgr2=0x55dd78f8e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78f8e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.650+0000 7f1535400640 1 -- 172.21.15.59:0/2076198409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd78eadae0 con 0x55dd78f80b90 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f1535400640 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1524009fd0 tx=0x7f15240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e63fbe7d2ceac572 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f152ea00640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 0x7f151c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f1523400640 1 -- 172.21.15.59:0/2076198409 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524004050 con 0x55dd78f80b90 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f1523400640 1 -- 172.21.15.59:0/2076198409 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1524005500 con 0x55dd78f80b90 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f1523400640 1 -- 172.21.15.59:0/2076198409 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524004c80 con 0x55dd78f80b90 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f152ea00640 1 -- 172.21.15.59:0/606778793 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f15300f0490 con 0x7f1538109140 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.651+0000 7f1535e00640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 0x7f151c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- 172.21.15.59:0/2076198409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 msgr2=0x55dd78f80f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1524009fd0 tx=0x7f15240046c0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f1535e00640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 0x7f151c0a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1528009da0 tx=0x7f1528005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- 172.21.15.59:0/2076198409 shutdown_connections 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78f8e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f8b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 --2- 172.21.15.59:0/2076198409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f80b90 0x55dd78f80f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- 172.21.15.59:0/2076198409 >> 172.21.15.59:0/2076198409 conn(0x55dd78f7d120 msgr2=0x55dd78f7d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- 172.21.15.59:0/2076198409 shutdown_connections 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- 172.21.15.59:0/2076198409 wait complete. 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.652+0000 7f153fe50740 1 -- start start 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 Processor -- start 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f8bd70 0x55dd78f89720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dd78f8e9e0 con 0x55dd78f80b90 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd78ea20f0 con 0x55dd78f814b0 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd78f8eb60 con 0x55dd78f8bd70 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f153fe50740 10 client.0 _ll_register_callbacks cb 0x55dd78e939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f1535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.653+0000 7f1535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57836/0 (socket says 172.21.15.59:57836) 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 -- 172.21.15.59:0/1043392708 learned_addr learned my addr 172.21.15.59:0/1043392708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1536800640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f8bd70 0x55dd78f89720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535400640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 -- 172.21.15.59:0/1043392708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f8bd70 msgr2=0x55dd78f89720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f8bd70 0x55dd78f89720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 -- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 msgr2=0x55dd78f891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 -- 172.21.15.59:0/1043392708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd78c583c0 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1535e00640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f152801ea30 tx=0x7f1528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909b6f20d6ce626a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.654+0000 7f1517e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152802e060 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1528018030 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1524005500 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1528035050 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f153fe50740 10 client.15531 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd78f9a770 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1524005500 con 0x55dd78f80b90 2024-09-05T11:58:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.655+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f152800f660 con 0x55dd78f80b90 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 10 client.15531 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 20 client.15531 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd78f9bcd0 0x55dd78f9e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 4 client.15531 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd78ef4950 con 0x55dd78f9bcd0 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f1535400640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd78f9bcd0 0x55dd78f9e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f1535400640 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd78f9bcd0 0x55dd78f9e150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f2fd7a97f6443 server_cookie=f63a24066d418c15 in_seq=0 out_seq=0 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.656+0000 7f1517e00640 10 client.15531 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.657+0000 7f1517e00640 1 -- 172.21.15.59:0/1043392708 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd78ef4950 con 0x55dd78f9bcd0 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.657+0000 7f1517e00640 10 client.15531 handle_command_reply: tid=0 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.657+0000 7f153fe50740 1 client.15531 shutdown 2024-09-05T11:58:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.657+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 msgr2=0x55dd78f88ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.657+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f152801ea30 tx=0x7f1528004730 comp rx=0 tx=0).stop 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 shutdown_connections 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f8bd70 0x55dd78f89720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f814b0 0x55dd78f891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd78f9bcd0 0x55dd78f9e150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 --2- 172.21.15.59:0/1043392708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f88ca0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f1517e00640 0 client.15531 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 >> 172.21.15.59:0/1043392708 conn(0x55dd78f7d120 msgr2=0x55dd78f82e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 shutdown_connections 2024-09-05T11:58:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 1 -- 172.21.15.59:0/1043392708 wait complete. 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.658+0000 7f153fe50740 20 client.15531 trim_cache size 0 max 16384 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 Processor -- start 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 -- start start 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78fa3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78fa3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15381af080 con 0x55dd78f8bd70 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b1900 con 0x55dd78f80b90 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.676+0000 7f153fe50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15381af6f0 con 0x55dd78f814b0 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78fa3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:57464/0 (socket says 172.21.15.59:57464) 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 -- 172.21.15.59:0/2179805364 learned_addr learned my addr 172.21.15.59:0/2179805364 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 -- 172.21.15.59:0/2179805364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 msgr2=0x55dd78fa3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78fa3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 -- 172.21.15.59:0/2179805364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 msgr2=0x55dd78fa3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78fa3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 -- 172.21.15.59:0/2179805364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd78f7cbc0 con 0x55dd78f80b90 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.677+0000 7f1535e00640 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f152800fa80 tx=0x7f152800fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e98a845563d8bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f1522a00640 1 -- 172.21.15.59:0/2179805364 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152800f3c0 con 0x55dd78f80b90 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f1522a00640 1 -- 172.21.15.59:0/2179805364 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152803e070 con 0x55dd78f80b90 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f1522a00640 1 -- 172.21.15.59:0/2179805364 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528039430 con 0x55dd78f80b90 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 -- 172.21.15.59:0/2179805364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 msgr2=0x55dd78f835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f152800fa80 tx=0x7f152800fe00 comp rx=0 tx=0).stop 2024-09-05T11:58:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 -- 172.21.15.59:0/2179805364 shutdown_connections 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f8bd70 0x55dd78fa3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78fa3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 --2- 172.21.15.59:0/2179805364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f80b90 0x55dd78f835f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 -- 172.21.15.59:0/2179805364 >> 172.21.15.59:0/2179805364 conn(0x55dd78f7d120 msgr2=0x55dd78f89480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.678+0000 7f153fe50740 1 -- 172.21.15.59:0/2179805364 shutdown_connections 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.679+0000 7f153fe50740 1 -- 172.21.15.59:0/2179805364 wait complete. 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.679+0000 7f153fe50740 1 -- start start 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.679+0000 7f153fe50740 1 Processor -- start 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.679+0000 7f153fe50740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f8fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.679+0000 7f153fe50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f153fe50740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 0x55dd790059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f153fe50740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15381afdc0 con 0x55dd78f80b90 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f153fe50740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b1230 con 0x55dd78f8bd70 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f153fe50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15381b0490 con 0x55dd78f814b0 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f153fe50740 10 client.0 _ll_register_callbacks cb 0x7f152801e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f8fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:32862/0 (socket says 172.21.15.59:32862) 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535400640 1 -- 172.21.15.59:0/3933392171 learned_addr learned my addr 172.21.15.59:0/3933392171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1536800640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 0x55dd790059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 -- 172.21.15.59:0/3933392171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 msgr2=0x55dd78f903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 -- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 msgr2=0x55dd790059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 0x55dd790059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535e00640 1 -- 172.21.15.59:0/3933392171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd78f90c30 con 0x55dd78f80b90 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1535400640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.680+0000 7f1536800640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 0x55dd790059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f1535e00640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f8fe60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f152800a010 tx=0x7f152800f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb72f52302af992d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f1517e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528046020 con 0x55dd78f80b90 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1528029b20 con 0x55dd78f80b90 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f152803e070 con 0x55dd78f80b90 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.681+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f152800f7e0 con 0x55dd78f80b90 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 10 client.15537 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd79007010 con 0x55dd78f80b90 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152803e070 con 0x55dd78f80b90 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f15280046a0 con 0x55dd78f80b90 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 10 client.15537 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 20 client.15537 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd79007670 0x55dd79009a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 4 client.15537 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.682+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd78eeee60 con 0x55dd79007670 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f1535400640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd79007670 0x55dd79009a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f1535400640 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd79007670 0x55dd79009a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a6fd3009d57c42 server_cookie=32c56881318aefaa in_seq=0 out_seq=0 2024-09-05T11:58:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f1517e00640 10 client.15537 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f1517e00640 1 -- 172.21.15.59:0/3933392171 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd78eeee60 con 0x55dd79007670 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f1517e00640 10 client.15537 handle_command_reply: tid=0 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.683+0000 7f153fe50740 1 client.15537 shutdown 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 msgr2=0x55dd78f8fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f8fe60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f152800a010 tx=0x7f152800f3e0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 shutdown_connections 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd78f8bd70 0x55dd790059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd78f814b0 0x55dd78f903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dd79007670 0x55dd79009a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 --2- 172.21.15.59:0/3933392171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dd78f80b90 0x55dd78f8fe60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 >> 172.21.15.59:0/3933392171 conn(0x55dd78f7d120 msgr2=0x55dd78f7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f1517e00640 0 client.15537 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:58:56.795 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 shutdown_connections 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 1 -- 172.21.15.59:0/3933392171 wait complete. 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.684+0000 7f153fe50740 20 client.15537 trim_cache size 0 max 16384 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 msgr2=0x7f151c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 0x7f151c0a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1528009da0 tx=0x7f1528005f90 comp rx=0 tx=0).stop 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 msgr2=0x7f153819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1530031a20 tx=0x7f1530031a50 comp rx=0 tx=0).stop 2024-09-05T11:58:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 shutdown_connections 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f151c0a2c00 0x7f151c0a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1538109140 0x7f153819eec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15381049c0 0x7f153819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 --2- 172.21.15.59:0/606778793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1538104010 0x7f153819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 >> 172.21.15.59:0/606778793 conn(0x7f15380ffa50 msgr2=0x7f1538100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 shutdown_connections 2024-09-05T11:58:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:55.685+0000 7f153d200640 1 -- 172.21.15.59:0/606778793 wait complete. 2024-09-05T11:58:56.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:56 smithi189 ceph-mon[34772]: pgmap v432: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:56 smithi089 ceph-mon[34499]: pgmap v432: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:56.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:56 smithi059 ceph-mon[30025]: pgmap v432: 129 pgs: 129 active+clean; 814 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:58:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:57.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:58:57.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:58.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 -- 172.21.15.59:0/1394459213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d10103e30 msgr2=0x7f3d10104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 --2- 172.21.15.59:0/1394459213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d10103e30 0x7f3d10104210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d0002a410 comp rx=0 tx=0).stop 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 -- 172.21.15.59:0/1394459213 shutdown_connections 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 --2- 172.21.15.59:0/1394459213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d10110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 --2- 172.21.15.59:0/1394459213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d101047e0 0x7f3d10108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 --2- 172.21.15.59:0/1394459213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d10103e30 0x7f3d10104210 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 -- 172.21.15.59:0/1394459213 >> 172.21.15.59:0/1394459213 conn(0x7f3d100ff8d0 msgr2=0x7f3d10101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.092+0000 7f3d15200640 1 -- 172.21.15.59:0/1394459213 shutdown_connections 2024-09-05T11:58:58.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.093+0000 7f3d15200640 1 -- 172.21.15.59:0/1394459213 wait complete. 2024-09-05T11:58:58.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 Processor -- start 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 -- start start 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 0x7f3d1019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d1019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10114010 con 0x7f3d101047e0 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10113e90 con 0x7f3d10103e30 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d15200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10114190 con 0x7f3d10108f60 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.094+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57878/0 (socket says 172.21.15.59:57878) 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 -- 172.21.15.59:0/2895286766 learned_addr learned my addr 172.21.15.59:0/2895286766 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d0f400640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 0x7f3d1019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d1019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 -- 172.21.15.59:0/2895286766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 msgr2=0x7f3d1019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d1019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 -- 172.21.15.59:0/2895286766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 msgr2=0x7f3d1019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 0x7f3d1019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 -- 172.21.15.59:0/2895286766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1019f450 con 0x7f3d101047e0 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d1019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d0f400640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 0x7f3d1019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:58.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.095+0000 7f3d07e00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9f0 tx=0x7f3cf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.096+0000 7f3d0cc00640 1 -- 172.21.15.59:0/2895286766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800cd80 con 0x7f3d101047e0 2024-09-05T11:58:58.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.096+0000 7f3d0cc00640 1 -- 172.21.15.59:0/2895286766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8010910 con 0x7f3d101047e0 2024-09-05T11:58:58.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.096+0000 7f3d0cc00640 1 -- 172.21.15.59:0/2895286766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8018990 con 0x7f3d101047e0 2024-09-05T11:58:58.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.096+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1019f740 con 0x7f3d101047e0 2024-09-05T11:58:58.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.096+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a7020 con 0x7f3d101047e0 2024-09-05T11:58:58.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 Processor -- start 2024-09-05T11:58:58.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d0cc00640 1 -- 172.21.15.59:0/2895286766 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3cf8010ac0 con 0x7f3d101047e0 2024-09-05T11:58:58.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 -- start start 2024-09-05T11:58:58.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c78900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c85af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c53bde650 con 0x561c53c78520 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561c53b85f10 con 0x561c53c83700 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.099+0000 7f3d192b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c53bda610 con 0x561c53c78e40 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c78900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:32880/0 (socket says 172.21.15.59:32880) 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 -- 172.21.15.59:0/2256449431 learned_addr learned my addr 172.21.15.59:0/2256449431 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d07e00640 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c85af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 -- 172.21.15.59:0/2256449431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 msgr2=0x561c53c85af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c85af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 -- 172.21.15.59:0/2256449431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 msgr2=0x561c53c78900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c78900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.100+0000 7f3d0f400640 1 -- 172.21.15.59:0/2256449431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c538ed580 con 0x561c53c78e40 2024-09-05T11:58:58.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d0f400640 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d000043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4937fe20d165355 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d0cc00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 0x7f3cf40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d0cc00640 1 -- 172.21.15.59:0/2895286766 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3cf80cf7c0 con 0x7f3d101047e0 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d04c00640 1 -- 172.21.15.59:0/2256449431 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00035040 con 0x561c53c78e40 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 0x7f3cf40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d04c00640 1 -- 172.21.15.59:0/2256449431 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00004760 con 0x561c53c78e40 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.101+0000 7f3d04c00640 1 -- 172.21.15.59:0/2256449431 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00003ab0 con 0x561c53c78e40 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 0x7f3cf40a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005fd0 tx=0x7f3d08009820 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- 172.21.15.59:0/2256449431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 msgr2=0x561c53c831c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d000043f0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- 172.21.15.59:0/2256449431 shutdown_connections 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c85af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c831c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 --2- 172.21.15.59:0/2256449431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c78900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- 172.21.15.59:0/2256449431 >> 172.21.15.59:0/2256449431 conn(0x561c53c74ab0 msgr2=0x561c53c74e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- 172.21.15.59:0/2256449431 shutdown_connections 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- 172.21.15.59:0/2256449431 wait complete. 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.102+0000 7f3d192b4740 1 -- start start 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 Processor -- start 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c80bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c53c86270 con 0x561c53c78520 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561c53b99350 con 0x561c53c83700 2024-09-05T11:58:58.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c53c863f0 con 0x561c53c78e40 2024-09-05T11:58:58.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d192b4740 10 client.0 _ll_register_callbacks cb 0x561c53bbc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:58.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.103+0000 7f3d0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57900/0 (socket says 172.21.15.59:57900) 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c80bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 -- 172.21.15.59:0/3699303848 learned_addr learned my addr 172.21.15.59:0/3699303848 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 -- 172.21.15.59:0/3699303848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 msgr2=0x561c53c80bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c80bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 -- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 msgr2=0x561c53c810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 -- 172.21.15.59:0/3699303848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c539b7960 con 0x561c53c78520 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d07e00640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3d0f400640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004b40 tx=0x7f3d000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd729c506933656 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.104+0000 7f3cfd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00052060 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3d00005000 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d00004760 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3d00035050 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3d192b4740 10 client.15549 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c53c91f00 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.105+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00004760 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3d000315b0 con 0x561c53c78520 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 10 client.15549 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 10 client.15549 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 20 client.15549 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:58.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53c93250 0x561c53c95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 4 client.15549 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:58:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c53bebbc0 con 0x561c53c93250 2024-09-05T11:58:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.106+0000 7f3d07e00640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53c93250 0x561c53c95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.107+0000 7f3d07e00640 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53c93250 0x561c53c95640 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d4b45727c4fe31 server_cookie=6d7d75ded99a3d8 in_seq=0 out_seq=0 2024-09-05T11:58:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.107+0000 7f3cfd600640 10 client.15549 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3cfd600640 1 -- 172.21.15.59:0/3699303848 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c53bebbc0 con 0x561c53c93250 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3cfd600640 10 client.15549 handle_command_reply: tid=0 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 client.15549 shutdown 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 msgr2=0x561c53c80670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004b40 tx=0x7f3d000045e0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 shutdown_connections 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78e40 0x561c53c80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53c93250 0x561c53c95640 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 --2- 172.21.15.59:0/3699303848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78520 0x561c53c80670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3cfd600640 0 client.15549 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 >> 172.21.15.59:0/3699303848 conn(0x561c53c74ab0 msgr2=0x561c53c73cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 shutdown_connections 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.108+0000 7f3d192b4740 1 -- 172.21.15.59:0/3699303848 wait complete. 2024-09-05T11:58:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.109+0000 7f3d192b4740 20 client.15549 trim_cache size 0 max 16384 2024-09-05T11:58:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.126+0000 7f3d192b4740 1 Processor -- start 2024-09-05T11:58:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 -- start start 2024-09-05T11:58:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101aef80 con 0x561c53c78e40 2024-09-05T11:58:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b1800 con 0x561c53c83700 2024-09-05T11:58:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d192b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101af5f0 con 0x561c53c78520 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57908/0 (socket says 172.21.15.59:57908) 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.127+0000 7f3d07e00640 1 -- 172.21.15.59:0/4124293127 learned_addr learned my addr 172.21.15.59:0/4124293127 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d0f400640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 -- 172.21.15.59:0/4124293127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 msgr2=0x561c53c73b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c73b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 -- 172.21.15.59:0/4124293127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 msgr2=0x561c53c98ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c98ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 -- 172.21.15.59:0/4124293127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c53c862c0 con 0x561c53c78e40 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d0f400640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c73b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3d07e00640 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8098580 tx=0x7f3cf80ccd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc155365e814a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.128+0000 7f3cffe00640 1 -- 172.21.15.59:0/4124293127 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80a4820 con 0x561c53c78e40 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3cffe00640 1 -- 172.21.15.59:0/4124293127 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8099eb0 con 0x561c53c78e40 2024-09-05T11:58:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3cffe00640 1 -- 172.21.15.59:0/4124293127 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf809b000 con 0x561c53c78e40 2024-09-05T11:58:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 -- 172.21.15.59:0/4124293127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 msgr2=0x561c53c98660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8098580 tx=0x7f3cf80ccd20 comp rx=0 tx=0).stop 2024-09-05T11:58:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 -- 172.21.15.59:0/4124293127 shutdown_connections 2024-09-05T11:58:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c98ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c78e40 0x561c53c98660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 --2- 172.21.15.59:0/4124293127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c73b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.129+0000 7f3d192b4740 1 -- 172.21.15.59:0/4124293127 >> 172.21.15.59:0/4124293127 conn(0x561c53c74ab0 msgr2=0x561c53c7a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.130+0000 7f3d192b4740 1 -- 172.21.15.59:0/4124293127 shutdown_connections 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.130+0000 7f3d192b4740 1 -- 172.21.15.59:0/4124293127 wait complete. 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.130+0000 7f3d192b4740 1 -- start start 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.130+0000 7f3d192b4740 1 Processor -- start 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c96b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101afcc0 con 0x561c53c97070 2024-09-05T11:58:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b1130 con 0x561c53c83700 2024-09-05T11:58:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b0390 con 0x561c53c78520 2024-09-05T11:58:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d192b4740 10 client.0 _ll_register_callbacks cb 0x7f3d080096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57910/0 (socket says 172.21.15.59:57910) 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d0fe00640 1 -- 172.21.15.59:0/1689597464 learned_addr learned my addr 172.21.15.59:0/1689597464 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d07e00640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c96b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.131+0000 7f3d0f400640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 -- 172.21.15.59:0/1689597464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 msgr2=0x561c53c965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 -- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 msgr2=0x561c53c96b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c96b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 -- 172.21.15.59:0/1689597464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c53cfd220 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d07e00640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c96b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0f400640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3d0fe00640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3d08010a80 tx=0x7f3d08014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74b5093263c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3cfd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004b20 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.132+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3d08004cc0 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf8099eb0 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3d08036050 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 10 client.15555 fetch_fsmap learned FSMap version 21 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c53cfe510 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8099eb0 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3d080055b0 con 0x561c53c97070 2024-09-05T11:58:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 10 client.15555 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 10 client.15555 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 20 client.15555 populate_metadata read hostname 'smithi059' 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53cfece0 0x561c53d01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 4 client.15555 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f662b9f6", "release": true, "await": true}] 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.133+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f662b9f6", "release": true, "await": true}) -- 0x561c53be60d0 con 0x561c53cfece0 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.134+0000 7f3d0f400640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53cfece0 0x561c53d01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.134+0000 7f3d0f400640 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53cfece0 0x561c53d01100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1003510b536ea1e7 server_cookie=3b62407371627c19 in_seq=0 out_seq=0 2024-09-05T11:58:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.134+0000 7f3cfd600640 10 client.15555 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3cfd600640 1 -- 172.21.15.59:0/1689597464 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561c53be60d0 con 0x561c53cfece0 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3cfd600640 10 client.15555 handle_command_reply: tid=0 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 client.15555 shutdown 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 msgr2=0x561c53cfcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3d08010a80 tx=0x7f3d08014040 comp rx=0 tx=0).stop 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 shutdown_connections 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c53cfece0 0x561c53d01100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c53c97070 0x561c53cfcb90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c53c83700 0x561c53c96b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 --2- 172.21.15.59:0/1689597464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c53c78520 0x561c53c965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3cfd600640 0 client.15555 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.136+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 >> 172.21.15.59:0/1689597464 conn(0x561c53c74ab0 msgr2=0x561c53c7ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.137+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 shutdown_connections 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.137+0000 7f3d192b4740 1 -- 172.21.15.59:0/1689597464 wait complete. 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.137+0000 7f3d192b4740 20 client.15555 trim_cache size 0 max 16384 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:58:58.140 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 8, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "f662b9f6": { 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 8, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:58:58.141 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 137.7, 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:58.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 msgr2=0x7f3cf40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 0x7f3cf40a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005fd0 tx=0x7f3d08009820 comp rx=0 tx=0).stop 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 msgr2=0x7f3d1019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9f0 tx=0x7f3cf800eeb0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 shutdown_connections 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cf40a2c00 0x7f3cf40a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10108f60 0x7f3d1019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d101047e0 0x7f3d1019a9e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 --2- 172.21.15.59:0/2895286766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d10103e30 0x7f3d1019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 >> 172.21.15.59:0/2895286766 conn(0x7f3d100ff8d0 msgr2=0x7f3d10100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 shutdown_connections 2024-09-05T11:58:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:58.138+0000 7f3d15200640 1 -- 172.21.15.59:0/2895286766 wait complete. 2024-09-05T11:58:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:58:58 smithi089 ceph-mon[34499]: pgmap v433: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-05T11:58:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:58:58 smithi059 ceph-mon[30025]: pgmap v433: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-05T11:58:59.064 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f662b9f6', total seconds elapsed: 24.2 2024-09-05T11:58:59.064 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-05T11:58:59.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:58:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:58:58 smithi189 ceph-mon[34772]: pgmap v433: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-05T11:58:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 -- 172.21.15.59:0/607956751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f38108f60 msgr2=0x7f3f38110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 --2- 172.21.15.59:0/607956751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f38108f60 0x7f3f38110ae0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009ca0 tx=0x7f3f3002a8c0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 -- 172.21.15.59:0/607956751 shutdown_connections 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 --2- 172.21.15.59:0/607956751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f38108f60 0x7f3f38110ae0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 --2- 172.21.15.59:0/607956751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f381047e0 0x7f3f38108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 --2- 172.21.15.59:0/607956751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38103e30 0x7f3f38104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.667+0000 7f3f3d400640 1 -- 172.21.15.59:0/607956751 >> 172.21.15.59:0/607956751 conn(0x7f3f380ff8f0 msgr2=0x7f3f38101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.668+0000 7f3f3d400640 1 -- 172.21.15.59:0/607956751 shutdown_connections 2024-09-05T11:58:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.668+0000 7f3f3d400640 1 -- 172.21.15.59:0/607956751 wait complete. 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 Processor -- start 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 -- start start 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 0x7f3f3819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38108f60 0x7f3f3819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38113fc0 con 0x7f3f381047e0 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38113e40 con 0x7f3f38103e30 2024-09-05T11:58:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.669+0000 7f3f3d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38114140 con 0x7f3f38108f60 2024-09-05T11:58:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:57536/0 (socket says 172.21.15.59:57536) 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 -- 172.21.15.59:0/2499557952 learned_addr learned my addr 172.21.15.59:0/2499557952 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f3ca00640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38108f60 0x7f3f3819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 0x7f3f3819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 -- 172.21.15.59:0/2499557952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38108f60 msgr2=0x7f3f3819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38108f60 0x7f3f3819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 -- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 msgr2=0x7f3f3819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 0x7f3f3819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37e00640 1 -- 172.21.15.59:0/2499557952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3819f3d0 con 0x7f3f38103e30 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.670+0000 7f3f37400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 0x7f3f3819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T11:58:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f37e00640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800e9f0 tx=0x7f3f2800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800cd80 con 0x7f3f38103e30 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3819f6c0 con 0x7f3f38103e30 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f28005500 con 0x7f3f38103e30 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28010690 con 0x7f3f38103e30 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.671+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381a6fa0 con 0x7f3f38103e30 2024-09-05T11:58:59.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.672+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f38104210 con 0x7f3f38103e30 2024-09-05T11:58:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.674+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f280046c0 con 0x7f3f38103e30 2024-09-05T11:58:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.676+0000 7f3f34c00640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 0x7f3f1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:58:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.676+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3f280cf940 con 0x7f3f38103e30 2024-09-05T11:58:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.676+0000 7f3f37400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 0x7f3f1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:58:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.677+0000 7f3f37400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 0x7f3f1c0a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f3810a960 tx=0x7f3f24005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:58:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.679+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f2809c430 con 0x7f3f38103e30 2024-09-05T11:58:59.734 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:58:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.919+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f3810c1d0 con 0x7f3f38103e30 2024-09-05T11:58:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.920+0000 7f3f34c00640 1 -- 172.21.15.59:0/2499557952 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f3f2809bd30 con 0x7f3f38103e30 2024-09-05T11:58:59.922 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:58:59.922 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:58:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:58:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 msgr2=0x7f3f1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 0x7f3f1c0a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f3810a960 tx=0x7f3f24005f90 comp rx=0 tx=0).stop 2024-09-05T11:58:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 msgr2=0x7f3f3819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800e9f0 tx=0x7f3f2800edf0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 shutdown_connections 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3f1c0a2b80 0x7f3f1c0a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f38108f60 0x7f3f3819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f381047e0 0x7f3f3819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 --2- 172.21.15.59:0/2499557952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f38103e30 0x7f3f3819a420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 >> 172.21.15.59:0/2499557952 conn(0x7f3f380ff8f0 msgr2=0x7f3f38100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 shutdown_connections 2024-09-05T11:58:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:58:59.923+0000 7f3f3d400640 1 -- 172.21.15.59:0/2499557952 wait complete. 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 -- 172.21.15.59:0/3273398453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 msgr2=0x7f4ed4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 --2- 172.21.15.59:0/3273398453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed4108830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009900 tx=0x7f4ec002a400 comp rx=0 tx=0).stop 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 -- 172.21.15.59:0/3273398453 shutdown_connections 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 --2- 172.21.15.59:0/3273398453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4108f60 0x7f4ed4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 --2- 172.21.15.59:0/3273398453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed4108830 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 --2- 172.21.15.59:0/3273398453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4103e30 0x7f4ed4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 -- 172.21.15.59:0/3273398453 >> 172.21.15.59:0/3273398453 conn(0x7f4ed40ff8d0 msgr2=0x7f4ed4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 -- 172.21.15.59:0/3273398453 shutdown_connections 2024-09-05T11:59:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.463+0000 7f4edb000640 1 -- 172.21.15.59:0/3273398453 wait complete. 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.464+0000 7f4edb000640 1 Processor -- start 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.464+0000 7f4edb000640 1 -- start start 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 0x7f4ed419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4114010 con 0x7f4ed41047e0 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4113e90 con 0x7f4ed4108f60 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4edb000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4114190 con 0x7f4ed4103e30 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4ed9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4ed9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4ed9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:32940/0 (socket says 172.21.15.59:32940) 2024-09-05T11:59:00.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4ed9c00640 1 -- 172.21.15.59:0/2519672243 learned_addr learned my addr 172.21.15.59:0/2519672243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:00.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.465+0000 7f4eda600640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 0x7f4ed419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 -- 172.21.15.59:0/2519672243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 msgr2=0x7f4ed419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 -- 172.21.15.59:0/2519672243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 msgr2=0x7f4ed419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 0x7f4ed419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 -- 172.21.15.59:0/2519672243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed419f420 con 0x7f4ed41047e0 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9c00640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4eda600640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 0x7f4ed419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed9200640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed419a9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002a920 tx=0x7f4ec0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.466+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec003f070 con 0x7f4ed41047e0 2024-09-05T11:59:00.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.467+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec0031c90 con 0x7f4ed41047e0 2024-09-05T11:59:00.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.467+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0035030 con 0x7f4ed41047e0 2024-09-05T11:59:00.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.467+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed419f6b0 con 0x7f4ed41047e0 2024-09-05T11:59:00.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.467+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed419fb40 con 0x7f4ed41047e0 2024-09-05T11:59:00.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.468+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4104210 con 0x7f4ed41047e0 2024-09-05T11:59:00.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.470+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ec0042070 con 0x7f4ed41047e0 2024-09-05T11:59:00.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.471+0000 7f4ed2000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 0x7f4ebc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:00.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.471+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4ec00ef990 con 0x7f4ed41047e0 2024-09-05T11:59:00.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.471+0000 7f4ed9c00640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 0x7f4ebc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:00.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.472+0000 7f4ed9c00640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 0x7f4ebc0a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8011550 tx=0x7f4ec8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:00.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.475+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ec00bbe30 con 0x7f4ed41047e0 2024-09-05T11:59:00.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.699+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed402d7c0 con 0x7f4ed41047e0 2024-09-05T11:59:00.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.700+0000 7f4ed2000640 1 -- 172.21.15.59:0/2519672243 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f4ec00c0130 con 0x7f4ed41047e0 2024-09-05T11:59:00.702 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:59:00.702 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:59:00.702 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:59:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.702+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 msgr2=0x7f4ebc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.702+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 0x7f4ebc0a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8011550 tx=0x7f4ec8005f90 comp rx=0 tx=0).stop 2024-09-05T11:59:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.702+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 msgr2=0x7f4ed419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.702+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed419a9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002a920 tx=0x7f4ec0003b20 comp rx=0 tx=0).stop 2024-09-05T11:59:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 shutdown_connections 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4ebc0a2c00 0x7f4ebc0a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed4108f60 0x7f4ed419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed41047e0 0x7f4ed419a9b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 --2- 172.21.15.59:0/2519672243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed4103e30 0x7f4ed419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 >> 172.21.15.59:0/2519672243 conn(0x7f4ed40ff8d0 msgr2=0x7f4ed4100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 shutdown_connections 2024-09-05T11:59:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:00.703+0000 7f4edb000640 1 -- 172.21.15.59:0/2519672243 wait complete. 2024-09-05T11:59:00.718 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:00 smithi059 ceph-mon[30025]: pgmap v434: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 1 op/s 2024-09-05T11:59:00.719 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:00 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2499557952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:00 smithi089 ceph-mon[34499]: pgmap v434: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 1 op/s 2024-09-05T11:59:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:00 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2499557952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:00.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:59:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:00 smithi189 ceph-mon[34772]: pgmap v434: 129 pgs: 129 active+clean; 821 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 1 op/s 2024-09-05T11:59:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:00 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2499557952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:01 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2519672243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:01.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:01 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2519672243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:02.027 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-05T11:59:02.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-05T11:59:02.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:01 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2519672243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:02.089 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:02.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 -- 172.21.15.59:0/164827947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4109140 msgr2=0x7fbad4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 --2- 172.21.15.59:0/164827947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4109140 0x7fbad4110cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009ca0 tx=0x7fbacc02a8c0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 -- 172.21.15.59:0/164827947 shutdown_connections 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 --2- 172.21.15.59:0/164827947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad4109140 0x7fbad4110cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 --2- 172.21.15.59:0/164827947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad41049c0 0x7fbad4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 --2- 172.21.15.59:0/164827947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4104010 0x7fbad41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 -- 172.21.15.59:0/164827947 >> 172.21.15.59:0/164827947 conn(0x7fbad40ffa50 msgr2=0x7fbad4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.656+0000 7fbada000640 1 -- 172.21.15.59:0/164827947 shutdown_connections 2024-09-05T11:59:02.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.657+0000 7fbada000640 1 -- 172.21.15.59:0/164827947 wait complete. 2024-09-05T11:59:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.657+0000 7fbada000640 1 Processor -- start 2024-09-05T11:59:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 -- start start 2024-09-05T11:59:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41049c0 0x7fbad419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 0x7fbad419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41141c0 con 0x7fbad41049c0 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4114040 con 0x7fbad4104010 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbada000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4114340 con 0x7fbad4109140 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41049c0 0x7fbad419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbad3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbad3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58462/0 (socket says 172.21.15.59:58462) 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbad3400640 1 -- 172.21.15.59:0/3148316112 learned_addr learned my addr 172.21.15.59:0/3148316112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.658+0000 7fbad3e00640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 0x7fbad419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 -- 172.21.15.59:0/3148316112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 msgr2=0x7fbad419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 0x7fbad419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 -- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41049c0 msgr2=0x7fbad419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41049c0 0x7fbad419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 -- 172.21.15.59:0/3148316112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad419f5d0 con 0x7fbad4104010 2024-09-05T11:59:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3e00640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 0x7fbad419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:02.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.659+0000 7fbad3400640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9f0 tx=0x7fbac400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.662+0000 7fbacbe00640 1 -- 172.21.15.59:0/3148316112 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400cd80 con 0x7fbad4104010 2024-09-05T11:59:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.662+0000 7fbacbe00640 1 -- 172.21.15.59:0/3148316112 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4005500 con 0x7fbad4104010 2024-09-05T11:59:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.662+0000 7fbacbe00640 1 -- 172.21.15.59:0/3148316112 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4010690 con 0x7fbad4104010 2024-09-05T11:59:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.662+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad419f8c0 con 0x7fbad4104010 2024-09-05T11:59:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.662+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a71a0 con 0x7fbad4104010 2024-09-05T11:59:02.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.665+0000 7fbacbe00640 1 -- 172.21.15.59:0/3148316112 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbac40046c0 con 0x7fbad4104010 2024-09-05T11:59:02.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.666+0000 7fbadcbfb740 1 Processor -- start 2024-09-05T11:59:02.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.666+0000 7fbadcbfb740 1 -- start start 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.666+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26763910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26763e50 0x558f26766240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558f266bf3e0 con 0x558f26763e50 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558f26666d80 con 0x558f26758c70 2024-09-05T11:59:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f266bb3a0 con 0x558f26759590 2024-09-05T11:59:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26763910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26763910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41554/0 (socket says 172.21.15.59:41554) 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3400640 1 -- 172.21.15.59:0/4156634351 learned_addr learned my addr 172.21.15.59:0/4156634351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58466/0 (socket says 172.21.15.59:58466) 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad2a00640 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26763e50 0x558f26766240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 -- 172.21.15.59:0/4156634351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 msgr2=0x558f26763910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26763910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 -- 172.21.15.59:0/4156634351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26763e50 msgr2=0x558f26766240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26763e50 0x558f26766240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.667+0000 7fbad3e00640 1 -- 172.21.15.59:0/4156634351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f26685ae0 con 0x558f26758c70 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbacbe00640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 0x7fbab80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611bc9f8ca64372f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbacbe00640 1 -- 172.21.15.59:0/3148316112 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbac40cf8e0 con 0x7fbad4104010 2024-09-05T11:59:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbad3400640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 0x7fbab80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbabfe00640 1 -- 172.21.15.59:0/4156634351 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc031db0 con 0x558f26758c70 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbabfe00640 1 -- 172.21.15.59:0/4156634351 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc036040 con 0x558f26758c70 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.668+0000 7fbabfe00640 1 -- 172.21.15.59:0/4156634351 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0042a0 con 0x558f26758c70 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.669+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4156634351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 msgr2=0x558f26759050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.669+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbacc006010 tx=0x7fbacc009800 comp rx=0 tx=0).stop 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4156634351 shutdown_connections 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26763e50 0x558f26766240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26763910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4156634351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26759050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4156634351 >> 172.21.15.59:0/4156634351 conn(0x558f26755200 msgr2=0x558f267555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbad3400640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 0x7fbab80a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbac40ccf70 tx=0x7fbac40d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4156634351 shutdown_connections 2024-09-05T11:59:02.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.670+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4156634351 wait complete. 2024-09-05T11:59:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 -- start start 2024-09-05T11:59:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 Processor -- start 2024-09-05T11:59:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f26760dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f26761840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558f267669c0 con 0x558f26758c70 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558f2667a0f0 con 0x558f26759590 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f26766b40 con 0x558f26763e50 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.671+0000 7fbadcbfb740 10 client.0 _ll_register_callbacks cb 0x558f26661bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58474/0 (socket says 172.21.15.59:58474) 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 -- 172.21.15.59:0/1648154761 learned_addr learned my addr 172.21.15.59:0/1648154761 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f26760dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f26761840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 -- 172.21.15.59:0/1648154761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 msgr2=0x558f26761840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f26761840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 -- 172.21.15.59:0/1648154761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 msgr2=0x558f26760dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f26760dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad2a00640 1 -- 172.21.15.59:0/1648154761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f264303c0 con 0x558f26759590 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.672+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f26760dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009e00 tx=0x7fbac000e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ade5a042c68b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbabcc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0025030 con 0x558f26759590 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbac0005020 con 0x558f26759590 2024-09-05T11:59:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbacc036040 con 0x558f26759590 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbac002f050 con 0x558f26759590 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbadcbfb740 10 client.25598 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.673+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f26772ae0 con 0x558f26759590 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc036040 con 0x558f26759590 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbac0021610 con 0x558f26759590 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 10 client.25598 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 20 client.25598 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f26773fa0 0x558f267763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 4 client.25598 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f266cc950 con 0x558f26773fa0 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.674+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f26773fa0 0x558f267763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.675+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f26773fa0 0x558f267763d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51fa9044ea793e7 server_cookie=89bd4fbcb088dad1 in_seq=0 out_seq=0 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.675+0000 7fbabcc00640 10 client.25598 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.675+0000 7fbabcc00640 1 -- 172.21.15.59:0/1648154761 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f266cc950 con 0x558f26773fa0 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.675+0000 7fbabcc00640 10 client.25598 handle_command_reply: tid=0 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.675+0000 7fbadcbfb740 1 client.25598 shutdown 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 msgr2=0x558f26761300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009e00 tx=0x7fbac000e780 comp rx=0 tx=0).stop 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 shutdown_connections 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f26761840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26759590 0x558f26761300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f26773fa0 0x558f267763d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1648154761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f26760dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 >> 172.21.15.59:0/1648154761 conn(0x558f26755200 msgr2=0x558f2675aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbabcc00640 0 client.25598 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:02.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 shutdown_connections 2024-09-05T11:59:02.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1648154761 wait complete. 2024-09-05T11:59:02.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.676+0000 7fbadcbfb740 20 client.25598 trim_cache size 0 max 16384 2024-09-05T11:59:02.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 Processor -- start 2024-09-05T11:59:02.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 -- start start 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26754270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26763e50 0x558f2677bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41aad60 con 0x558f26758c70 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ad5e0 con 0x558f26763e50 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.690+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ab3d0 con 0x558f26759590 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50476/0 (socket says 172.21.15.59:50476) 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad3e00640 1 -- 172.21.15.59:0/1506400393 learned_addr learned my addr 172.21.15.59:0/1506400393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26754270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26763e50 0x558f2677bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 -- 172.21.15.59:0/1506400393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26763e50 msgr2=0x558f2677bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26763e50 0x558f2677bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 -- 172.21.15.59:0/1506400393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 msgr2=0x558f2675b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad2a00640 1 -- 172.21.15.59:0/1506400393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f267593c0 con 0x558f26759590 2024-09-05T11:59:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.691+0000 7fbad3e00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.692+0000 7fbad2a00640 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26754270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009250 tx=0x7fbac00093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ccf6e9af9975a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.692+0000 7fbabf400640 1 -- 172.21.15.59:0/1506400393 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0025040 con 0x558f26759590 2024-09-05T11:59:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.692+0000 7fbabf400640 1 -- 172.21.15.59:0/1506400393 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac0037030 con 0x558f26759590 2024-09-05T11:59:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.692+0000 7fbabf400640 1 -- 172.21.15.59:0/1506400393 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0003970 con 0x558f26759590 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1506400393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 msgr2=0x558f26754270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26754270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009250 tx=0x7fbac00093f0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1506400393 shutdown_connections 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26763e50 0x558f2677bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26759590 0x558f26754270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/1506400393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26758c70 0x558f2675b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1506400393 >> 172.21.15.59:0/1506400393 conn(0x558f26755200 msgr2=0x558f26753b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1506400393 shutdown_connections 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- 172.21.15.59:0/1506400393 wait complete. 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.693+0000 7fbadcbfb740 1 -- start start 2024-09-05T11:59:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 Processor -- start 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26767ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f267ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41abaa0 con 0x558f26759590 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41acf10 con 0x558f26758c70 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41ac170 con 0x558f26763e50 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.694+0000 7fbadcbfb740 10 client.0 _ll_register_callbacks cb 0x7fbac0017bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50490/0 (socket says 172.21.15.59:50490) 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 -- 172.21.15.59:0/4237865379 learned_addr learned my addr 172.21.15.59:0/4237865379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26767ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f267ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 -- 172.21.15.59:0/4237865379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 msgr2=0x558f267ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f267ddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 -- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 msgr2=0x558f26767ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26767ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 -- 172.21.15.59:0/4237865379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f26768d20 con 0x558f26759590 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26767ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbad2a00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbac0021c70 tx=0x7fbac0009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a9990fb65c893f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.695+0000 7fbabcc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0021810 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbac0005040 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac0037030 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbac004b050 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbadcbfb740 10 client.15582 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.696+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f267df320 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac0042070 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbac0003cc0 con 0x558f26759590 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 10 client.15582 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 20 client.15582 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f267df9c0 0x558f267e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 4 client.15582 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbac0037030 con 0x558f267df9c0 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.697+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f267df9c0 0x558f267e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbad3e00640 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f267df9c0 0x558f267e1db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a0fc05c9835b75 server_cookie=1d613c37bfec6ffa in_seq=0 out_seq=0 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbabcc00640 10 client.15582 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbabcc00640 1 -- 172.21.15.59:0/4237865379 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbac0037030 con 0x558f267df9c0 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbabcc00640 10 client.15582 handle_command_reply: tid=0 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 client.15582 shutdown 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 msgr2=0x558f26768420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbac0021c70 tx=0x7fbac0009d30 comp rx=0 tx=0).stop 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 shutdown_connections 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f26763e50 0x558f267ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558f267df9c0 0x558f267e1db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558f26759590 0x558f26768420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 --2- 172.21.15.59:0/4237865379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558f26758c70 0x558f26767ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 >> 172.21.15.59:0/4237865379 conn(0x558f26755200 msgr2=0x558f267615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.698+0000 7fbabcc00640 0 client.15582 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.699+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 shutdown_connections 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.699+0000 7fbadcbfb740 1 -- 172.21.15.59:0/4237865379 wait complete. 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.699+0000 7fbadcbfb740 20 client.15582 trim_cache size 0 max 16384 2024-09-05T11:59:02.707 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.699+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 msgr2=0x7fbab80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 0x7fbab80a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbac40ccf70 tx=0x7fbac40d5530 comp rx=0 tx=0).stop 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 msgr2=0x7fbad419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9f0 tx=0x7fbac400edf0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 shutdown_connections 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbab80a2c00 0x7fbab80a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad4109140 0x7fbad419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbad41049c0 0x7fbad419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 --2- 172.21.15.59:0/3148316112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbad4104010 0x7fbad419a620 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9f0 tx=0x7fbac400edf0 comp rx=0 tx=0).stop 2024-09-05T11:59:02.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 >> 172.21.15.59:0/3148316112 conn(0x7fbad40ffa50 msgr2=0x7fbad4100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:02.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 shutdown_connections 2024-09-05T11:59:02.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:02.700+0000 7fbada000640 1 -- 172.21.15.59:0/3148316112 wait complete. 2024-09-05T11:59:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:02 smithi089 ceph-mon[34499]: pgmap v435: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T11:59:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:02 smithi059 ceph-mon[30025]: pgmap v435: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T11:59:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:02 smithi189 ceph-mon[34772]: pgmap v435: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T11:59:03.471 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.072+0000 7fb5c2e00640 1 -- 172.21.15.59:0/1184206379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc1049c0 msgr2=0x7fb5bc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.072+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/1184206379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc108a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009900 tx=0x7fb5a402a3f0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 -- 172.21.15.59:0/1184206379 shutdown_connections 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/1184206379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc109140 0x7fb5bc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/1184206379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc108a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/1184206379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5bc104010 0x7fb5bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 -- 172.21.15.59:0/1184206379 >> 172.21.15.59:0/1184206379 conn(0x7fb5bc0ffa50 msgr2=0x7fb5bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 -- 172.21.15.59:0/1184206379 shutdown_connections 2024-09-05T11:59:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.073+0000 7fb5c2e00640 1 -- 172.21.15.59:0/1184206379 wait complete. 2024-09-05T11:59:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 Processor -- start 2024-09-05T11:59:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 -- start start 2024-09-05T11:59:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5bc104010 0x7fb5bc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc114160 con 0x7fb5bc109140 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc113fe0 con 0x7fb5bc1049c0 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.074+0000 7fb5c2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1142e0 con 0x7fb5bc104010 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50510/0 (socket says 172.21.15.59:50510) 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 -- 172.21.15.59:0/2182008105 learned_addr learned my addr 172.21.15.59:0/2182008105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5b3400640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 -- 172.21.15.59:0/2182008105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5bc104010 msgr2=0x7fb5bc19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5bc104010 0x7fb5bc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 -- 172.21.15.59:0/2182008105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc1049c0 msgr2=0x7fb5bc19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.075+0000 7fb5c0e00640 1 -- 172.21.15.59:0/2182008105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc19f690 con 0x7fb5bc109140 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400eaa0 tx=0x7fb5b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5b9600640 1 -- 172.21.15.59:0/2182008105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400cdb0 con 0x7fb5bc109140 2024-09-05T11:59:04.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5b9600640 1 -- 172.21.15.59:0/2182008105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b4005020 con 0x7fb5bc109140 2024-09-05T11:59:04.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc19f980 con 0x7fb5bc109140 2024-09-05T11:59:04.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5b9600640 1 -- 172.21.15.59:0/2182008105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40116b0 con 0x7fb5bc109140 2024-09-05T11:59:04.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.076+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a7260 con 0x7fb5bc109140 2024-09-05T11:59:04.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.079+0000 7fb5b9600640 1 -- 172.21.15.59:0/2182008105 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb5b4005560 con 0x7fb5bc109140 2024-09-05T11:59:04.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.079+0000 7fb5c5814740 1 Processor -- start 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.079+0000 7fb5c5814740 1 -- start start 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b020487590 0x55b020489980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0203e2530 con 0x55b02047c3b0 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b020389860 con 0x55b02047ccd0 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5c5814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b0203de4f0 con 0x55b020487590 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58516/0 (socket says 172.21.15.59:58516) 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50518/0 (socket says 172.21.15.59:50518) 2024-09-05T11:59:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5bbe00640 1 -- 172.21.15.59:0/2264310716 learned_addr learned my addr 172.21.15.59:0/2264310716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.080+0000 7fb5b3400640 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b020487590 0x55b020489980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 -- 172.21.15.59:0/2264310716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b020487590 msgr2=0x55b020489980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b020487590 0x55b020489980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 -- 172.21.15.59:0/2264310716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 msgr2=0x55b02047c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 -- 172.21.15.59:0/2264310716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b020153b90 con 0x55b02047ccd0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b9600640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 0x7fb5a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c40 tx=0x7fb5ac00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4ec7e5d3134ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b9600640 1 -- 172.21.15.59:0/2182008105 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb5b40cf690 con 0x7fb5bc109140 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b3400640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 0x7fb5a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b1600640 1 -- 172.21.15.59:0/2264310716 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac005020 con 0x55b02047ccd0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b1600640 1 -- 172.21.15.59:0/2264310716 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0047a0 con 0x55b02047ccd0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.081+0000 7fb5b1600640 1 -- 172.21.15.59:0/2264310716 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac010760 con 0x55b02047ccd0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 -- 172.21.15.59:0/2264310716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 msgr2=0x55b020487050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5b3400640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 0x7fb5a00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009230 tx=0x7fb5a40314e0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c40 tx=0x7fb5ac00e720 comp rx=0 tx=0).stop 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 -- 172.21.15.59:0/2264310716 shutdown_connections 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b020487590 0x55b020489980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020487050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 --2- 172.21.15.59:0/2264310716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 -- 172.21.15.59:0/2264310716 >> 172.21.15.59:0/2264310716 conn(0x55b020478940 msgr2=0x55b020478d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.082+0000 7fb5c5814740 1 -- 172.21.15.59:0/2264310716 shutdown_connections 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.083+0000 7fb5c5814740 1 -- 172.21.15.59:0/2264310716 wait complete. 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.083+0000 7fb5c5814740 1 -- start start 2024-09-05T11:59:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.083+0000 7fb5c5814740 1 Processor -- start 2024-09-05T11:59:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b020484600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020484b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b02048a010 con 0x55b020487590 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b02039ce60 con 0x55b02047ccd0 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b02048a190 con 0x55b02047c3b0 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c5814740 10 client.0 _ll_register_callbacks cb 0x55b0203a1cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b020484600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50528/0 (socket says 172.21.15.59:50528) 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c0e00640 1 -- 172.21.15.59:0/1195704200 learned_addr learned my addr 172.21.15.59:0/1195704200 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5b3400640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020484b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 msgr2=0x55b020484b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020484b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1195704200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 msgr2=0x55b020485080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1195704200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0203a8990 con 0x55b02047c3b0 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.084+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:04.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.085+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b020484600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c10 tx=0x7fb5ac009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f0df8310015e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.085+0000 7fb5aa000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.085+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac025020 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.085+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac0047a0 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.085+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb5ac004b60 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb5ac030050 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5c5814740 10 client.25537 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b020495e30 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0047a0 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb5ac0035f0 con 0x55b02047c3b0 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5c5814740 10 client.25537 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.086+0000 7fb5c5814740 10 client.25537 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:59:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.087+0000 7fb5c5814740 20 client.25537 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.087+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b0204971e0 0x55b0204995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.087+0000 7fb5c5814740 4 client.25537 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.087+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b020385310 con 0x55b0204971e0 2024-09-05T11:59:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.087+0000 7fb5b3400640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b0204971e0 0x55b0204995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.088+0000 7fb5b3400640 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b0204971e0 0x55b0204995d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd27f1801e2c599 server_cookie=4e7897bce482eab9 in_seq=0 out_seq=0 2024-09-05T11:59:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.088+0000 7fb5aa000640 10 client.25537 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.088+0000 7fb5aa000640 1 -- 172.21.15.59:0/1195704200 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b020385310 con 0x55b0204971e0 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.088+0000 7fb5aa000640 10 client.25537 handle_command_reply: tid=0 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.088+0000 7fb5c5814740 1 client.25537 shutdown 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 msgr2=0x55b020484600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b020484600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009c10 tx=0x7fb5ac009820 comp rx=0 tx=0).stop 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 shutdown_connections 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b0204971e0 0x55b0204995d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b020487590 0x55b020485080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b02047ccd0 0x55b020484b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1195704200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b020484600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5aa000640 0 client.25537 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 >> 172.21.15.59:0/1195704200 conn(0x55b020478940 msgr2=0x55b020477c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 shutdown_connections 2024-09-05T11:59:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 1 -- 172.21.15.59:0/1195704200 wait complete. 2024-09-05T11:59:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.089+0000 7fb5c5814740 20 client.25537 trim_cache size 0 max 16384 2024-09-05T11:59:04.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 Processor -- start 2024-09-05T11:59:04.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 -- start start 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047ccd0 0x55b02049eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02049f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1af190 con 0x55b02047c3b0 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1b1a10 con 0x55b020487590 2024-09-05T11:59:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5c5814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1af800 con 0x55b02047ccd0 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50536/0 (socket says 172.21.15.59:50536) 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1674487075 learned_addr learned my addr 172.21.15.59:0/1674487075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.107+0000 7fb5b3400640 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047ccd0 0x55b02049eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02049f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1674487075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047ccd0 msgr2=0x55b02049eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047ccd0 0x55b02049eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1674487075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 msgr2=0x55b02049f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02049f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.108+0000 7fb5bbe00640 1 -- 172.21.15.59:0/1674487075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02047cb00 con 0x55b02047c3b0 2024-09-05T11:59:04.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.109+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac003b60 tx=0x7fb5ac005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33348dcb31c09580 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.109+0000 7fb5b0c00640 1 -- 172.21.15.59:0/1674487075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac039070 con 0x55b02047c3b0 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.109+0000 7fb5b0c00640 1 -- 172.21.15.59:0/1674487075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0047a0 con 0x55b02047c3b0 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.109+0000 7fb5b0c00640 1 -- 172.21.15.59:0/1674487075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac025070 con 0x55b02047c3b0 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- 172.21.15.59:0/1674487075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 msgr2=0x55b02047ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac003b60 tx=0x7fb5ac005a70 comp rx=0 tx=0).stop 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- 172.21.15.59:0/1674487075 shutdown_connections 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02049f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047ccd0 0x55b02049eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 --2- 172.21.15.59:0/1674487075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047c3b0 0x55b02047ee10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- 172.21.15.59:0/1674487075 >> 172.21.15.59:0/1674487075 conn(0x55b020478940 msgr2=0x55b020484de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- 172.21.15.59:0/1674487075 shutdown_connections 2024-09-05T11:59:04.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- 172.21.15.59:0/1674487075 wait complete. 2024-09-05T11:59:04.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.110+0000 7fb5c5814740 1 -- start start 2024-09-05T11:59:04.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 Processor -- start 2024-09-05T11:59:04.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b02048b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02048be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1afed0 con 0x55b02047ccd0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1b1340 con 0x55b020487590 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1b05a0 con 0x55b02047c3b0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5c5814740 10 client.0 _ll_register_callbacks cb 0x7fb5a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50538/0 (socket says 172.21.15.59:50538) 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b02048b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.111+0000 7fb5b3400640 1 -- 172.21.15.59:0/608085419 learned_addr learned my addr 172.21.15.59:0/608085419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5c0e00640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02048be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 -- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 msgr2=0x55b02048be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02048be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 -- 172.21.15.59:0/608085419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 msgr2=0x55b02048b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 -- 172.21.15.59:0/608085419 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b020500ce0 con 0x55b02047c3b0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5b3400640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5bbe00640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b02048b3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac003b60 tx=0x7fb5ac003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fccbc5f9594066f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.112+0000 7fb5aa000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac039070 con 0x55b02047c3b0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac0047a0 con 0x55b02047c3b0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb5ac005380 con 0x55b02047c3b0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb5ac045050 con 0x55b02047c3b0 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5c5814740 10 client.25543 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b020501fa0 con 0x55b02047c3b0 2024-09-05T11:59:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.113+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac0047a0 con 0x55b02047c3b0 2024-09-05T11:59:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb5ac0043c0 con 0x55b02047c3b0 2024-09-05T11:59:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 10 client.25543 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 10 client.25543 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 20 client.25543 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b020502630 0x55b020504a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 4 client.25543 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55b0203efc60 con 0x55b020502630 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.114+0000 7fb5b3400640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b020502630 0x55b020504a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.115+0000 7fb5b3400640 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b020502630 0x55b020504a20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caefa3934a323a server_cookie=1f010115f9d8f3d0 in_seq=0 out_seq=0 2024-09-05T11:59:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.115+0000 7fb5aa000640 10 client.25543 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.122+0000 7fb5aa000640 1 -- 172.21.15.59:0/608085419 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b0203efc60 con 0x55b020502630 2024-09-05T11:59:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.122+0000 7fb5aa000640 10 client.25543 handle_command_reply: tid=0 2024-09-05T11:59:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.122+0000 7fb5c5814740 1 client.25543 shutdown 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 msgr2=0x55b02048b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b02048b3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac003b60 tx=0x7fb5ac003800 comp rx=0 tx=0).stop 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 shutdown_connections 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b020487590 0x55b02048be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b020502630 0x55b020504a20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b02047ccd0 0x55b02048b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 --2- 172.21.15.59:0/608085419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b02047c3b0 0x55b02048b3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5aa000640 0 client.25543 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 >> 172.21.15.59:0/608085419 conn(0x55b020478940 msgr2=0x55b020482da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 shutdown_connections 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 1 -- 172.21.15.59:0/608085419 wait complete. 2024-09-05T11:59:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.123+0000 7fb5c5814740 20 client.25543 trim_cache size 0 max 16384 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 10, 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "b020b871": { 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 10, 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:59:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 137.7, 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:04.128 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 msgr2=0x7fb5a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 0x7fb5a00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009230 tx=0x7fb5a40314e0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 msgr2=0x7fb5bc19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400eaa0 tx=0x7fb5b400ef60 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 shutdown_connections 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb5a00a2b80 0x7fb5a00a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5bc109140 0x7fb5bc19efb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5bc1049c0 0x7fb5bc19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 --2- 172.21.15.59:0/2182008105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5bc104010 0x7fb5bc19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 >> 172.21.15.59:0/2182008105 conn(0x7fb5bc0ffa50 msgr2=0x7fb5bc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 shutdown_connections 2024-09-05T11:59:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:04.124+0000 7fb5c2e00640 1 -- 172.21.15.59:0/2182008105 wait complete. 2024-09-05T11:59:04.464 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:59:04.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:59:04.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:04 smithi059 ceph-mon[30025]: pgmap v436: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:04.860 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b020b871', quiesce took 5.8 seconds. Will release after: 11.899999999999999 2024-09-05T11:59:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:04 smithi089 ceph-mon[34499]: pgmap v436: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:04 smithi189 ceph-mon[34772]: pgmap v436: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 -- 172.21.15.59:0/3440910189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 msgr2=0x7f5cd0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/3440910189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd0110ca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ca0 tx=0x7f5cc802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 -- 172.21.15.59:0/3440910189 shutdown_connections 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/3440910189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd0110ca0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/3440910189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd01049a0 0x7f5cd01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/3440910189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd0103ff0 0x7f5cd01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 -- 172.21.15.59:0/3440910189 >> 172.21.15.59:0/3440910189 conn(0x7f5cd00ffa90 msgr2=0x7f5cd0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 -- 172.21.15.59:0/3440910189 shutdown_connections 2024-09-05T11:59:05.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.490+0000 7f5cd4e00640 1 -- 172.21.15.59:0/3440910189 wait complete. 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 Processor -- start 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 -- start start 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd01049a0 0x7f5cd019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0114150 con 0x7f5cd0109120 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0113fd0 con 0x7f5cd01049a0 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.491+0000 7f5cd4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01142d0 con 0x7f5cd0103ff0 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5cc7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd01049a0 0x7f5cd019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41658/0 (socket says 172.21.15.59:41658) 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 -- 172.21.15.59:0/1057221307 learned_addr learned my addr 172.21.15.59:0/1057221307 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 -- 172.21.15.59:0/1057221307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd01049a0 msgr2=0x7f5cd019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd01049a0 0x7f5cd019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 -- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 msgr2=0x7f5cd019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.492+0000 7f5ccde00640 1 -- 172.21.15.59:0/1057221307 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd019f560 con 0x7f5cd0103ff0 2024-09-05T11:59:05.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5ccde00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9f0 tx=0x7f5cc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5cc6a00640 1 -- 172.21.15.59:0/1057221307 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000cd80 con 0x7f5cd0103ff0 2024-09-05T11:59:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5cc6a00640 1 -- 172.21.15.59:0/1057221307 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc0010910 con 0x7f5cd0103ff0 2024-09-05T11:59:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd019f850 con 0x7f5cd0103ff0 2024-09-05T11:59:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5cc6a00640 1 -- 172.21.15.59:0/1057221307 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0018990 con 0x7f5cd0103ff0 2024-09-05T11:59:05.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.493+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd01a7130 con 0x7f5cd0103ff0 2024-09-05T11:59:05.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cd7a71740 1 Processor -- start 2024-09-05T11:59:05.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cd7a71740 1 -- start start 2024-09-05T11:59:05.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cc6a00640 1 -- 172.21.15.59:0/1057221307 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cc0010ac0 con 0x7f5cd0103ff0 2024-09-05T11:59:05.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9ff3e0 0x562ebda09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.496+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ebd965290 con 0x562ebd9ff3e0 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562ebd90cb40 con 0x562ebd9feac0 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ebd961250 con 0x562ebda09ca0 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58562/0 (socket says 172.21.15.59:58562) 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 -- 172.21.15.59:0/4011790579 learned_addr learned my addr 172.21.15.59:0/4011790579 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5ccde00640 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9ff3e0 0x562ebda09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 -- 172.21.15.59:0/4011790579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 msgr2=0x562ebda0c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda0c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 -- 172.21.15.59:0/4011790579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9ff3e0 msgr2=0x562ebda09760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9ff3e0 0x562ebda09760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.497+0000 7f5cce800640 1 -- 172.21.15.59:0/4011790579 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebd6d66a0 con 0x562ebd9feac0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cce800640 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ca0 tx=0x7f5cc8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72fcd6129edafe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cc6a00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 0x7f5cb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cbf400640 1 -- 172.21.15.59:0/4011790579 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8031db0 con 0x562ebd9feac0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cbf400640 1 -- 172.21.15.59:0/4011790579 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8036040 con 0x562ebd9feac0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cbf400640 1 -- 172.21.15.59:0/4011790579 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80042d0 con 0x562ebd9feac0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5cc6a00640 1 -- 172.21.15.59:0/1057221307 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5cc00cf7c0 con 0x7f5cd0103ff0 2024-09-05T11:59:05.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.498+0000 7f5ccde00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 0x7f5cb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 -- 172.21.15.59:0/4011790579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 msgr2=0x562ebd9feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ca0 tx=0x7f5cc8005f10 comp rx=0 tx=0).stop 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 -- 172.21.15.59:0/4011790579 shutdown_connections 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda0c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9ff3e0 0x562ebda09760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/4011790579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9feac0 0x562ebd9feea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 -- 172.21.15.59:0/4011790579 >> 172.21.15.59:0/4011790579 conn(0x562ebd9fb050 msgr2=0x562ebd9fb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 -- 172.21.15.59:0/4011790579 shutdown_connections 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5ccde00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 0x7f5cb40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cc009b960 tx=0x7f5cc0000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.499+0000 7f5cd7a71740 1 -- 172.21.15.59:0/4011790579 wait complete. 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 -- start start 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 Processor -- start 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda06e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ebda0c730 con 0x562ebd9feac0 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562ebd91ffa0 con 0x562ebd9ff3e0 2024-09-05T11:59:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.500+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ebda0c8b0 con 0x562ebda09ca0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cd7a71740 10 client.0 _ll_register_callbacks cb 0x562ebd673ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50584/0 (socket says 172.21.15.59:50584) 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 -- 172.21.15.59:0/3038987530 learned_addr learned my addr 172.21.15.59:0/3038987530 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda06e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 -- 172.21.15.59:0/3038987530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 msgr2=0x562ebda06e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda06e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 -- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 msgr2=0x562ebda068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.501+0000 7f5cce800640 1 -- 172.21.15.59:0/3038987530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebda07640 con 0x562ebd9feac0 2024-09-05T11:59:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cce800640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006010 tx=0x7f5cc8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c6bc0771ad8e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8056060 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc8036040 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5cc8003b70 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5cc805d050 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cd7a71740 10 client.15612 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562ebda0c730 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.502+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8036040 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5cc8050070 con 0x562ebd9feac0 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 10 client.15612 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 20 client.15612 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda197f0 0x562ebda1bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 4 client.15612 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ebd972800 con 0x562ebda197f0 2024-09-05T11:59:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.503+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda197f0 0x562ebda1bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.504+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda197f0 0x562ebda1bbe0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a35ae742c03a7c server_cookie=761cb652d59aee64 in_seq=0 out_seq=0 2024-09-05T11:59:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.504+0000 7f5cb3e00640 10 client.15612 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.504+0000 7f5cb3e00640 1 -- 172.21.15.59:0/3038987530 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ebd972800 con 0x562ebda197f0 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.504+0000 7f5cb3e00640 10 client.15612 handle_command_reply: tid=0 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 client.15612 shutdown 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 msgr2=0x562ebda09600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006010 tx=0x7f5cc8031a20 comp rx=0 tx=0).stop 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 shutdown_connections 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda06e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda197f0 0x562ebda1bbe0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3038987530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda09600 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 >> 172.21.15.59:0/3038987530 conn(0x562ebd9fb050 msgr2=0x562ebd9fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cb3e00640 0 client.15612 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 shutdown_connections 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3038987530 wait complete. 2024-09-05T11:59:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.505+0000 7f5cd7a71740 20 client.15612 trim_cache size 0 max 16384 2024-09-05T11:59:05.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.523+0000 7f5cd7a71740 1 Processor -- start 2024-09-05T11:59:05.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 -- start start 2024-09-05T11:59:05.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda21a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda03b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01af000 con 0x562ebd9feac0 2024-09-05T11:59:05.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01b1880 con 0x562ebd9ff3e0 2024-09-05T11:59:05.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01af670 con 0x562ebda09ca0 2024-09-05T11:59:05.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50596/0 (socket says 172.21.15.59:50596) 2024-09-05T11:59:05.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cce800640 1 -- 172.21.15.59:0/3714968305 learned_addr learned my addr 172.21.15.59:0/3714968305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.524+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda21a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 -- 172.21.15.59:0/3714968305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 msgr2=0x562ebda03b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda03b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 -- 172.21.15.59:0/3714968305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 msgr2=0x562ebda21a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda21a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 -- 172.21.15.59:0/3714968305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebda05860 con 0x562ebd9feac0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cce800640 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004af0 tx=0x7f5cc802f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25988463e9d61ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cbea00640 1 -- 172.21.15.59:0/3714968305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8056070 con 0x562ebd9feac0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cbea00640 1 -- 172.21.15.59:0/3714968305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8036070 con 0x562ebd9feac0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.525+0000 7f5cbea00640 1 -- 172.21.15.59:0/3714968305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc804a040 con 0x562ebd9feac0 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3714968305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 msgr2=0x562ebda21500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004af0 tx=0x7f5cc802f390 comp rx=0 tx=0).stop 2024-09-05T11:59:05.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3714968305 shutdown_connections 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebda09ca0 0x562ebda03b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/3714968305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebd9feac0 0x562ebda21500 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3714968305 >> 172.21.15.59:0/3714968305 conn(0x562ebd9fb050 msgr2=0x562ebda08930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3714968305 shutdown_connections 2024-09-05T11:59:05.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.526+0000 7f5cd7a71740 1 -- 172.21.15.59:0/3714968305 wait complete. 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 -- start start 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 Processor -- start 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebd9feac0 0x562ebda0fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01afd40 con 0x562ebda09ca0 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01b11b0 con 0x562ebd9ff3e0 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd01b0410 con 0x562ebd9feac0 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cd7a71740 10 client.0 _ll_register_callbacks cb 0x562ebda07810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:05.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.527+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50612/0 (socket says 172.21.15.59:50612) 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cce800640 1 -- 172.21.15.59:0/2049078038 learned_addr learned my addr 172.21.15.59:0/2049078038 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58598/0 (socket says 172.21.15.59:58598) 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cce800640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebd9feac0 0x562ebda0fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 -- 172.21.15.59:0/2049078038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebd9feac0 msgr2=0x562ebda0fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebd9feac0 0x562ebda0fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 -- 172.21.15.59:0/2049078038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 msgr2=0x562ebda834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 -- 172.21.15.59:0/2049078038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebda10a70 con 0x562ebd9ff3e0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cce800640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cc7e00640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562ebda1c4a0 tx=0x7f5cb8010950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc40a83bc0cac7c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.528+0000 7f5cb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb802a060 con 0x562ebd9ff3e0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5cb8005220 con 0x562ebd9ff3e0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc8036070 con 0x562ebd9ff3e0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5cb8031050 con 0x562ebd9ff3e0 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 10 client.25640 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:05.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562ebda84bd0 con 0x562ebd9ff3e0 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8036070 con 0x562ebd9ff3e0 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5cb8025070 con 0x562ebd9ff3e0 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 10 client.25640 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 20 client.25640 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda85320 0x562ebda87740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 4 client.25640 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.529+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ebd96cd10 con 0x562ebda85320 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.530+0000 7f5cce800640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda85320 0x562ebda87740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.530+0000 7f5cce800640 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda85320 0x562ebda87740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654399ac5d2f1fd5 server_cookie=293585c4ccece5de in_seq=0 out_seq=0 2024-09-05T11:59:05.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.530+0000 7f5cb3e00640 10 client.25640 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.531+0000 7f5cb3e00640 1 -- 172.21.15.59:0/2049078038 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ebd96cd10 con 0x562ebda85320 2024-09-05T11:59:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.531+0000 7f5cb3e00640 10 client.25640 handle_command_reply: tid=0 2024-09-05T11:59:05.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.531+0000 7f5cd7a71740 1 client.25640 shutdown 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 msgr2=0x562ebda10190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x562ebda1c4a0 tx=0x7f5cb8010950 comp rx=0 tx=0).stop 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 shutdown_connections 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562ebda85320 0x562ebda87740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ebda09ca0 0x562ebda834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ebd9ff3e0 0x562ebda10190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 --2- 172.21.15.59:0/2049078038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ebd9feac0 0x562ebda0fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cb3e00640 0 client.25640 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 >> 172.21.15.59:0/2049078038 conn(0x562ebd9fb050 msgr2=0x562ebd9f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 shutdown_connections 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 1 -- 172.21.15.59:0/2049078038 wait complete. 2024-09-05T11:59:05.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.532+0000 7f5cd7a71740 20 client.25640 trim_cache size 0 max 16384 2024-09-05T11:59:05.538 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:59:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "5a4cc664-fb09-432e-9888-fffeeebd3033", 2024-09-05T11:59:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:59:05.538 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 msgr2=0x7f5cb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 0x7f5cb40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cc009b960 tx=0x7f5cc0000bc0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 msgr2=0x7f5cd019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9f0 tx=0x7f5cc000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 shutdown_connections 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5cb40a2b80 0x7f5cb40a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cd0109120 0x7f5cd019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cd01049a0 0x7f5cd019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 --2- 172.21.15.59:0/1057221307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd0103ff0 0x7f5cd019a5b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 >> 172.21.15.59:0/1057221307 conn(0x7f5cd00ffa90 msgr2=0x7f5cd0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:05.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 shutdown_connections 2024-09-05T11:59:05.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:05.533+0000 7f5cd4e00640 1 -- 172.21.15.59:0/1057221307 wait complete. 2024-09-05T11:59:06.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:59:06.789 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:06.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:06 smithi059 ceph-mon[30025]: pgmap v437: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:06.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:06 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:59:06.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:06 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:06.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:06 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:59:06.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:06 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:06 smithi089 ceph-mon[34499]: pgmap v437: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:06 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:59:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:06 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:06 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:59:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:06 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:06 smithi189 ceph-mon[34772]: pgmap v437: 129 pgs: 129 active+clean; 843 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-09-05T11:59:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:06 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:59:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:06 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:06 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:59:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:06 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 -- 172.21.15.59:0/995730866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc1012e0 msgr2=0x7fb6dc10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 --2- 172.21.15.59:0/995730866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc10dfa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009900 tx=0x7fb6c402a410 comp rx=0 tx=0).stop 2024-09-05T11:59:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 -- 172.21.15.59:0/995730866 shutdown_connections 2024-09-05T11:59:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 --2- 172.21.15.59:0/995730866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc10e4e0 0x7fb6dc1108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 --2- 172.21.15.59:0/995730866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc10dfa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 --2- 172.21.15.59:0/995730866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc100930 0x7fb6dc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 -- 172.21.15.59:0/995730866 >> 172.21.15.59:0/995730866 conn(0x7fb6dc0fc7a0 msgr2=0x7fb6dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 -- 172.21.15.59:0/995730866 shutdown_connections 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.372+0000 7fb6e3000640 1 -- 172.21.15.59:0/995730866 wait complete. 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.373+0000 7fb6e3000640 1 Processor -- start 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.373+0000 7fb6e3000640 1 -- start start 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc10e4e0 0x7fb6dc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1141b0 con 0x7fb6dc100930 2024-09-05T11:59:07.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc114030 con 0x7fb6dc1012e0 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e3000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc114330 con 0x7fb6dc10e4e0 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50630/0 (socket says 172.21.15.59:50630) 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1289066 learned_addr learned my addr 172.21.15.59:0/1289066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6d3400640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6e1000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc10e4e0 0x7fb6dc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1289066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc10e4e0 msgr2=0x7fb6dc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc10e4e0 0x7fb6dc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1289066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc1012e0 msgr2=0x7fb6dc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.374+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1289066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc19f640 con 0x7fb6dc100930 2024-09-05T11:59:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00ebc0 tx=0x7fb6cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6d9600640 1 -- 172.21.15.59:0/1289066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc010630 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc19f930 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6d9600640 1 -- 172.21.15.59:0/1289066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc010cb0 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc1a7210 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.375+0000 7fb6d9600640 1 -- 172.21.15.59:0/1289066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc018b70 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.378+0000 7fb6d9600640 1 -- 172.21.15.59:0/1289066 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb6cc018d10 con 0x7fb6dc100930 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.378+0000 7fb6e5bd2740 1 Processor -- start 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.378+0000 7fb6e5bd2740 1 -- start start 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 0x55a14a8b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a14a80d3e0 con 0x55a14a8a6b90 2024-09-05T11:59:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a14a7b4d80 con 0x55a14a8a74b0 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a14a8093a0 con 0x55a14a8b1d70 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 0x55a14a8b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50634/0 (socket says 172.21.15.59:50634) 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 -- 172.21.15.59:0/3415879841 learned_addr learned my addr 172.21.15.59:0/3415879841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6d3400640 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 -- 172.21.15.59:0/3415879841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 msgr2=0x55a14a8b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 -- 172.21.15.59:0/3415879841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 msgr2=0x55a14a8b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 0x55a14a8b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6e1000640 1 -- 172.21.15.59:0/3415879841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14a7d3ae0 con 0x55a14a8a6b90 2024-09-05T11:59:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.379+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 0x55a14a8b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009b30 tx=0x7fb6d40113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9779eab62c5ba32d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6d9600640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 0x7fb6c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6d9600640 1 -- 172.21.15.59:0/1289066 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb6cc0cfb20 con 0x7fb6dc100930 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6d1600640 1 -- 172.21.15.59:0/3415879841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4011e10 con 0x55a14a8a6b90 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 0x7fb6c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6d1600640 1 -- 172.21.15.59:0/3415879841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4012930 con 0x55a14a8a6b90 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.380+0000 7fb6d1600640 1 -- 172.21.15.59:0/3415879841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400f930 con 0x55a14a8a6b90 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3415879841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 msgr2=0x55a14a8a6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009b30 tx=0x7fb6d40113e0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 0x7fb6c00a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc098370 tx=0x7fb6cc0d3560 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3415879841 shutdown_connections 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a74b0 0x55a14a8b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3415879841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a6b90 0x55a14a8a6f70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3415879841 >> 172.21.15.59:0/3415879841 conn(0x55a14a8a3120 msgr2=0x55a14a8a3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3415879841 shutdown_connections 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.381+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3415879841 wait complete. 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 -- start start 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 Processor -- start 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a14a8b49e0 con 0x55a14a8a74b0 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a14a7c80f0 con 0x55a14a8b1d70 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a14a8b4b60 con 0x55a14a8a6b90 2024-09-05T11:59:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.382+0000 7fb6e5bd2740 10 client.0 _ll_register_callbacks cb 0x55a14a7b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41732/0 (socket says 172.21.15.59:41732) 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50650/0 (socket says 172.21.15.59:50650) 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6e1000640 1 -- 172.21.15.59:0/1227303894 learned_addr learned my addr 172.21.15.59:0/1227303894 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6e1000640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 -- 172.21.15.59:0/1227303894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 msgr2=0x55a14a8aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 -- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 msgr2=0x55a14a8af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6d3400640 1 -- 172.21.15.59:0/1227303894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a14a57e3c0 con 0x55a14a8a74b0 2024-09-05T11:59:07.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.383+0000 7fb6e1000640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:07.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6d3400640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb6c402aa00 tx=0x7fb6c4004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba6de4c81ff1e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6ca000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:07.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4035040 con 0x55a14a8a74b0 2024-09-05T11:59:07.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d4012930 con 0x55a14a8a74b0 2024-09-05T11:59:07.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb6c4005450 con 0x55a14a8a74b0 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb6c404f050 con 0x55a14a8a74b0 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6e5bd2740 10 client.15630 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a14a8c0710 con 0x55a14a8a74b0 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.384+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4012930 con 0x55a14a8a74b0 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb6c4003920 con 0x55a14a8a74b0 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 10 client.15630 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 20 client.15630 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a8c1bd0 0x55a14a8c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 4 client.15630 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:07.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a14a81a950 con 0x55a14a8c1bd0 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.385+0000 7fb6e1000640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a8c1bd0 0x55a14a8c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.386+0000 7fb6e1000640 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a8c1bd0 0x55a14a8c3fc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d537cfd733f54f server_cookie=b390dce388eb09a4 in_seq=0 out_seq=0 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.386+0000 7fb6ca000640 10 client.15630 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6ca000640 1 -- 172.21.15.59:0/1227303894 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a14a81a950 con 0x55a14a8c1bd0 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6ca000640 10 client.15630 handle_command_reply: tid=0 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 client.15630 shutdown 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 msgr2=0x55a14a8af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb6c402aa00 tx=0x7fb6c4004440 comp rx=0 tx=0).stop 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 shutdown_connections 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a8c1bd0 0x55a14a8c3fc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af1c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/1227303894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6ca000640 0 client.15630 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 >> 172.21.15.59:0/1227303894 conn(0x55a14a8a3120 msgr2=0x55a14a8a8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 shutdown_connections 2024-09-05T11:59:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.387+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/1227303894 wait complete. 2024-09-05T11:59:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.388+0000 7fb6e5bd2740 20 client.15630 trim_cache size 0 max 16384 2024-09-05T11:59:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 Processor -- start 2024-09-05T11:59:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 -- start start 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1af100 con 0x55a14a8a74b0 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1b1980 con 0x55a14a8b1d70 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1af770 con 0x55a14a8a6b90 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50666/0 (socket says 172.21.15.59:50666) 2024-09-05T11:59:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6d3400640 1 -- 172.21.15.59:0/4129630547 learned_addr learned my addr 172.21.15.59:0/4129630547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.406+0000 7fb6e1000640 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 -- 172.21.15.59:0/4129630547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 msgr2=0x55a14a8a8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8a8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 -- 172.21.15.59:0/4129630547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 msgr2=0x55a14a8af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 -- 172.21.15.59:0/4129630547 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14a8aa860 con 0x55a14a8a6b90 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6d3400640 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6e1000640 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8ad550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55a14a8c4800 tx=0x7fb6d40113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e171471822238e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6d0c00640 1 -- 172.21.15.59:0/4129630547 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4005110 con 0x55a14a8a6b90 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6d0c00640 1 -- 172.21.15.59:0/4129630547 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4012de0 con 0x55a14a8a6b90 2024-09-05T11:59:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.407+0000 7fb6d0c00640 1 -- 172.21.15.59:0/4129630547 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4003c30 con 0x55a14a8a6b90 2024-09-05T11:59:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/4129630547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 msgr2=0x55a14a8ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8ad550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55a14a8c4800 tx=0x7fb6d40113e0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/4129630547 shutdown_connections 2024-09-05T11:59:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8b1d70 0x55a14a8a8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/4129630547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8a6b90 0x55a14a8ad550 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/4129630547 >> 172.21.15.59:0/4129630547 conn(0x55a14a8a3120 msgr2=0x55a14a8a85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/4129630547 shutdown_connections 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.408+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/4129630547 wait complete. 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 -- start start 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 Processor -- start 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 0x55a14a8ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1afe40 con 0x55a14a8a74b0 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1b12b0 con 0x55a14a8a6b90 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc1b0510 con 0x55a14a8b1d70 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.409+0000 7fb6e5bd2740 10 client.0 _ll_register_callbacks cb 0x7fb6c40040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 0x55a14a8ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6e1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 0x55a14a8ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58660/0 (socket says 172.21.15.59:58660) 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6e1000640 1 -- 172.21.15.59:0/3700434412 learned_addr learned my addr 172.21.15.59:0/3700434412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 -- 172.21.15.59:0/3700434412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 msgr2=0x55a14a8b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 -- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 msgr2=0x55a14a8ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 0x55a14a8ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 -- 172.21.15.59:0/3700434412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a14a8b65b0 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6d3400640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8b5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4003e60 tx=0x7fb6c4005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2225c905b4ca926a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.410+0000 7fb6ca000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4035070 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb6c404c040 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d4012de0 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4061050 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6e5bd2740 10 client.15636 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a14a92c8a0 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.411+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4012de0 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb6c4003800 con 0x55a14a8a74b0 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 10 client.15636 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 20 client.15636 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a92d070 0x55a14a92f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 4 client.15636 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:59:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a14a814e60 con 0x55a14a92d070 2024-09-05T11:59:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a92d070 0x55a14a92f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6e1000640 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a92d070 0x55a14a92f490 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8f34b9936483b server_cookie=4b619b7ebcca01fc in_seq=0 out_seq=0 2024-09-05T11:59:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.412+0000 7fb6ca000640 10 client.15636 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6ca000640 1 -- 172.21.15.59:0/3700434412 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a14a814e60 con 0x55a14a92d070 2024-09-05T11:59:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6ca000640 10 client.15636 handle_command_reply: tid=0 2024-09-05T11:59:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 client.15636 shutdown 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 msgr2=0x55a14a8b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8b5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4003e60 tx=0x7fb6c4005480 comp rx=0 tx=0).stop 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 shutdown_connections 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14a8b1d70 0x55a14a8b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a14a92d070 0x55a14a92f490 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a14a8a74b0 0x55a14a8b5870 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 --2- 172.21.15.59:0/3700434412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a14a8a6b90 0x55a14a8ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 >> 172.21.15.59:0/3700434412 conn(0x55a14a8a3120 msgr2=0x55a14a8a8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6ca000640 0 client.15636 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 shutdown_connections 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 1 -- 172.21.15.59:0/3700434412 wait complete. 2024-09-05T11:59:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.413+0000 7fb6e5bd2740 20 client.15636 trim_cache size 0 max 16384 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "5a4cc664-fb09-432e-9888-fffeeebd3033": { 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "5a4cc664-fb09-432e-9888-fffeeebd3033", 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:07.420 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 msgr2=0x7fb6c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 0x7fb6c00a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc098370 tx=0x7fb6cc0d3560 comp rx=0 tx=0).stop 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 msgr2=0x7fb6dc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00ebc0 tx=0x7fb6cc00c6a0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 shutdown_connections 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb6c00a2b80 0x7fb6c00a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6dc10e4e0 0x7fb6dc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6dc1012e0 0x7fb6dc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 --2- 172.21.15.59:0/1289066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6dc100930 0x7fb6dc19a690 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 >> 172.21.15.59:0/1289066 conn(0x7fb6dc0fc7a0 msgr2=0x7fb6dc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 shutdown_connections 2024-09-05T11:59:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:07.414+0000 7fb6e3000640 1 -- 172.21.15.59:0/1289066 wait complete. 2024-09-05T11:59:08.142 INFO:tasks.cephfs.filesystem:scrub status for tag:5a4cc664-fb09-432e-9888-fffeeebd3033 - {'path': '/', 'tag': '5a4cc664-fb09-432e-9888-fffeeebd3033', 'options': 'recursive,force'} 2024-09-05T11:59:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:08 smithi089 ceph-mon[34499]: pgmap v438: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T11:59:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:08 smithi059 ceph-mon[30025]: pgmap v438: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T11:59:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:08 smithi189 ceph-mon[34772]: pgmap v438: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T11:59:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:10 smithi089 ceph-mon[34499]: pgmap v439: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.4 KiB/s wr, 4 op/s 2024-09-05T11:59:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:10 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:59:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:10 smithi059 ceph-mon[30025]: pgmap v439: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.4 KiB/s wr, 4 op/s 2024-09-05T11:59:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:10 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:59:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:10 smithi189 ceph-mon[34772]: pgmap v439: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.4 KiB/s wr, 4 op/s 2024-09-05T11:59:11.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:10 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:59:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:12 smithi089 ceph-mon[34499]: pgmap v440: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 5 op/s 2024-09-05T11:59:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:12 smithi059 ceph-mon[30025]: pgmap v440: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 5 op/s 2024-09-05T11:59:13.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:12 smithi189 ceph-mon[34772]: pgmap v440: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 5 op/s 2024-09-05T11:59:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:14 smithi089 ceph-mon[34499]: pgmap v441: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:14 smithi059 ceph-mon[30025]: pgmap v441: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:14 smithi189 ceph-mon[34772]: pgmap v441: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:16.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:16 smithi059 ceph-mon[30025]: pgmap v442: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T11:59:16.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:59:16.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:16 smithi089 ceph-mon[34499]: pgmap v442: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T11:59:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:16 smithi189 ceph-mon[34772]: pgmap v442: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T11:59:17.149 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 -- 172.21.15.59:0/945728526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19281027a0 msgr2=0x7f19281067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 --2- 172.21.15.59:0/945728526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19281027a0 0x7f19281067f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f191002a3f0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 -- 172.21.15.59:0/945728526 shutdown_connections 2024-09-05T11:59:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 --2- 172.21.15.59:0/945728526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 0x7f192810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 --2- 172.21.15.59:0/945728526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19281027a0 0x7f19281067f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 --2- 172.21.15.59:0/945728526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1928101df0 0x7f19281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 -- 172.21.15.59:0/945728526 >> 172.21.15.59:0/945728526 conn(0x7f19280fd8b0 msgr2=0x7f19280ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 -- 172.21.15.59:0/945728526 shutdown_connections 2024-09-05T11:59:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.726+0000 7f192fa00640 1 -- 172.21.15.59:0/945728526 wait complete. 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.727+0000 7f192fa00640 1 Processor -- start 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.727+0000 7f192fa00640 1 -- start start 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.727+0000 7f192fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 0x7f1928198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192fa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 0x7f1928198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192fa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 0x7f192819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1928111fd0 con 0x7f1928101df0 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192fa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1928111e50 con 0x7f19281027a0 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192fa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1928112150 con 0x7f1928106f20 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 0x7f1928198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 0x7f1928198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:17.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 0x7f1928198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57096/0 (socket says 172.21.15.59:57096) 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192d000640 1 -- 172.21.15.59:0/4171056264 learned_addr learned my addr 172.21.15.59:0/4171056264 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f192da00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 0x7f192819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 -- 172.21.15.59:0/4171056264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 msgr2=0x7f192819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 0x7f192819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 -- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 msgr2=0x7f1928198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 0x7f1928198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.728+0000 7f191fe00640 1 -- 172.21.15.59:0/4171056264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192819d3c0 con 0x7f19281027a0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f191fe00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 0x7f1928198950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f191002a910 tx=0x7f1910003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191003f070 con 0x7f19281027a0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192819d650 con 0x7f19281027a0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1910031c90 con 0x7f19281027a0 2024-09-05T11:59:17.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910035030 con 0x7f19281027a0 2024-09-05T11:59:17.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.729+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19281a4f90 con 0x7f19281027a0 2024-09-05T11:59:17.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.730+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19281032d0 con 0x7f19281027a0 2024-09-05T11:59:17.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.732+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1910042070 con 0x7f19281027a0 2024-09-05T11:59:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.733+0000 7f1926000640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 0x7f190c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.733+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f19100efde0 con 0x7f19281027a0 2024-09-05T11:59:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.734+0000 7f192d000640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 0x7f190c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.734+0000 7f192d000640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 0x7f190c0a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1918009c90 tx=0x7f1918005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.738+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19100bc280 con 0x7f19281027a0 2024-09-05T11:59:17.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.958+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f192810a100 con 0x7f19281027a0 2024-09-05T11:59:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.959+0000 7f1926000640 1 -- 172.21.15.59:0/4171056264 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f19100c0580 con 0x7f19281027a0 2024-09-05T11:59:17.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:59:17.961 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:59:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 msgr2=0x7f190c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 0x7f190c0a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1918009c90 tx=0x7f1918005f90 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 msgr2=0x7f1928198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 0x7f1928198950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f191002a910 tx=0x7f1910003b20 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 shutdown_connections 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f190c0a2b80 0x7f190c0a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1928106f20 0x7f192819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19281027a0 0x7f1928198950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 --2- 172.21.15.59:0/4171056264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1928101df0 0x7f1928198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 >> 172.21.15.59:0/4171056264 conn(0x7f19280fd8b0 msgr2=0x7f19280feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 shutdown_connections 2024-09-05T11:59:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:17.962+0000 7f192fa00640 1 -- 172.21.15.59:0/4171056264 wait complete. 2024-09-05T11:59:18.811 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id b020b871 --release --await' 2024-09-05T11:59:18.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id b020b871 --release --await 2024-09-05T11:59:18.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:18 smithi059 ceph-mon[30025]: pgmap v443: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:18.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:18 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4171056264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:18 smithi189 ceph-mon[34772]: pgmap v443: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:19.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:18 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4171056264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:18 smithi089 ceph-mon[34499]: pgmap v443: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T11:59:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:18 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4171056264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 -- 172.21.15.59:0/1639998623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe990108f60 msgr2=0x7fe990110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1639998623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe990108f60 0x7fe990110ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe988009ca0 tx=0x7fe98802a8c0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 -- 172.21.15.59:0/1639998623 shutdown_connections 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1639998623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe990108f60 0x7fe990110ae0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1639998623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9901047e0 0x7fe990108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1639998623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe990104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 -- 172.21.15.59:0/1639998623 >> 172.21.15.59:0/1639998623 conn(0x7fe9900ff910 msgr2=0x7fe990101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 -- 172.21.15.59:0/1639998623 shutdown_connections 2024-09-05T11:59:19.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.747+0000 7fe98fe00640 1 -- 172.21.15.59:0/1639998623 wait complete. 2024-09-05T11:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.748+0000 7fe98fe00640 1 Processor -- start 2024-09-05T11:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.748+0000 7fe98fe00640 1 -- start start 2024-09-05T11:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe99019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe990108f60 0x7fe99019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe990113fc0 con 0x7fe9901047e0 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe990113e40 con 0x7fe990108f60 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe990114140 con 0x7fe990103e30 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe99019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57112/0 (socket says 172.21.15.59:57112) 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98e000640 1 -- 172.21.15.59:0/1427024076 learned_addr learned my addr 172.21.15.59:0/1427024076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.749+0000 7fe98f400640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe990108f60 0x7fe99019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 -- 172.21.15.59:0/1427024076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe990108f60 msgr2=0x7fe99019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe990108f60 0x7fe99019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 -- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 msgr2=0x7fe99019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 -- 172.21.15.59:0/1427024076 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019f470 con 0x7fe990103e30 2024-09-05T11:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98e000640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98ea00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe99019a4c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e9f0 tx=0x7fe98000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe987400640 1 -- 172.21.15.59:0/1427024076 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000ccc0 con 0x7fe990103e30 2024-09-05T11:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019f760 con 0x7fe990103e30 2024-09-05T11:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe987400640 1 -- 172.21.15.59:0/1427024076 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980005020 con 0x7fe990103e30 2024-09-05T11:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe987400640 1 -- 172.21.15.59:0/1427024076 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9800049e0 con 0x7fe990103e30 2024-09-05T11:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.750+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a7040 con 0x7fe990103e30 2024-09-05T11:59:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.753+0000 7fe987400640 1 -- 172.21.15.59:0/1427024076 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe980005560 con 0x7fe990103e30 2024-09-05T11:59:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.753+0000 7fe9982d0740 1 Processor -- start 2024-09-05T11:59:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 -- start start 2024-09-05T11:59:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df95900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d2defb650 con 0x556d2df95e40 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556d2dea2f10 con 0x556d2df95520 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe9982d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556d2def7610 con 0x556d2dfa0700 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57128/0 (socket says 172.21.15.59:57128) 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe98ea00640 1 -- 172.21.15.59:0/515314871 learned_addr learned my addr 172.21.15.59:0/515314871 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.754+0000 7fe98f400640 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df95900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98e000640 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 -- 172.21.15.59:0/515314871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 msgr2=0x556d2dfa2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 -- 172.21.15.59:0/515314871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 msgr2=0x556d2dfa01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 -- 172.21.15.59:0/515314871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2dc0a580 con 0x556d2df95520 2024-09-05T11:59:19.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.755+0000 7fe98f400640 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df95900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe988005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea48bb1cf0e419a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe987400640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 0x7fe9740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe97be00640 1 -- 172.21.15.59:0/515314871 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988031db0 con 0x556d2df95520 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe97be00640 1 -- 172.21.15.59:0/515314871 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988036040 con 0x556d2df95520 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe97be00640 1 -- 172.21.15.59:0/515314871 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988004240 con 0x556d2df95520 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe987400640 1 -- 172.21.15.59:0/1427024076 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe9800cf420 con 0x7fe990103e30 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe98ea00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 0x7fe9740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 -- 172.21.15.59:0/515314871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 msgr2=0x556d2df95900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df95900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe988006010 tx=0x7fe988005f10 comp rx=0 tx=0).stop 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 -- 172.21.15.59:0/515314871 shutdown_connections 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 --2- 172.21.15.59:0/515314871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df95900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.756+0000 7fe9982d0740 1 -- 172.21.15.59:0/515314871 >> 172.21.15.59:0/515314871 conn(0x556d2df91ab0 msgr2=0x556d2df91e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.757+0000 7fe98ea00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 0x7fe9740a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe98009b510 tx=0x7fe9800d2e60 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.757+0000 7fe9982d0740 1 -- 172.21.15.59:0/515314871 shutdown_connections 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.757+0000 7fe9982d0740 1 -- 172.21.15.59:0/515314871 wait complete. 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.757+0000 7fe9982d0740 1 -- start start 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.757+0000 7fe9982d0740 1 Processor -- start 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df9d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2dfa0700 0x556d2df9e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556d2dfa30f0 con 0x556d2dfa0700 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556d2deb6350 con 0x556d2df95520 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556d2dfa3270 con 0x556d2df95e40 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe9982d0740 10 client.0 _ll_register_callbacks cb 0x556d2ded9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df9d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2dfa0700 0x556d2df9e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:59186/0 (socket says 172.21.15.59:59186) 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df9d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53078/0 (socket says 172.21.15.59:53078) 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.758+0000 7fe98e000640 1 -- 172.21.15.59:0/1454249738 learned_addr learned my addr 172.21.15.59:0/1454249738 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 -- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 msgr2=0x556d2df9d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df9d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 -- 172.21.15.59:0/1454249738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2dfa0700 msgr2=0x556d2df9e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2dfa0700 0x556d2df9e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 -- 172.21.15.59:0/1454249738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2dcd4960 con 0x556d2df95e40 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe98e000640 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe97c012d50 tx=0x7fe97c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e342c8cd5e12ad81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe978c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c012110 con 0x556d2df95e40 2024-09-05T11:59:19.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988036040 con 0x556d2df95e40 2024-09-05T11:59:19.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.759+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe97c004050 con 0x556d2df95e40 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.760+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe97c0122b0 con 0x556d2df95e40 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.760+0000 7fe9982d0740 10 client.25594 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.760+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556d2dfaede0 con 0x556d2df95e40 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.760+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988036040 con 0x556d2df95e40 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe97c0105f0 con 0x556d2df95e40 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 10 client.25594 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 10 client.25594 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 20 client.25594 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2dfb02f0 0x556d2dfb26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 4 client.25594 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2df08bc0 con 0x556d2dfb02f0 2024-09-05T11:59:19.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.761+0000 7fe98f400640 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2dfb02f0 0x556d2dfb26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.762+0000 7fe98f400640 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2dfb02f0 0x556d2dfb26e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0135cb9ce0ebd91 server_cookie=8c0805e01555adee in_seq=0 out_seq=0 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.762+0000 7fe978c00640 10 client.25594 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe978c00640 1 -- 172.21.15.59:0/1454249738 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d2df08bc0 con 0x556d2dfb02f0 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe978c00640 10 client.25594 handle_command_reply: tid=0 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 client.25594 shutdown 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 msgr2=0x556d2df9dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe97c012d50 tx=0x7fe97c00f790 comp rx=0 tx=0).stop 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 shutdown_connections 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2dfb02f0 0x556d2dfb26e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2dfa0700 0x556d2df9e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95e40 0x556d2df9dbb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 --2- 172.21.15.59:0/1454249738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2df9d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe978c00640 0 client.25594 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:19.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.763+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 >> 172.21.15.59:0/1454249738 conn(0x556d2df91ab0 msgr2=0x556d2df90cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.764+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 shutdown_connections 2024-09-05T11:59:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.764+0000 7fe9982d0740 1 -- 172.21.15.59:0/1454249738 wait complete. 2024-09-05T11:59:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.764+0000 7fe9982d0740 20 client.25594 trim_cache size 0 max 16384 2024-09-05T11:59:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.781+0000 7fe9982d0740 1 Processor -- start 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.781+0000 7fe9982d0740 1 -- start start 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95520 0x556d2df9df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2dfa0700 0x556d2dfb7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901aee60 con 0x556d2df95e40 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901b16e0 con 0x556d2dfa0700 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe9982d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901af4d0 con 0x556d2df95520 2024-09-05T11:59:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57150/0 (socket says 172.21.15.59:57150) 2024-09-05T11:59:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 -- 172.21.15.59:0/413706275 learned_addr learned my addr 172.21.15.59:0/413706275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98f400640 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95520 0x556d2df9df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98f400640 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2dfa0700 0x556d2dfb7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 -- 172.21.15.59:0/413706275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95520 msgr2=0x556d2df9df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95520 0x556d2df9df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 -- 172.21.15.59:0/413706275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2dfa0700 msgr2=0x556d2dfb7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2dfa0700 0x556d2dfb7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.782+0000 7fe98e000640 1 -- 172.21.15.59:0/413706275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2df9c620 con 0x556d2df95e40 2024-09-05T11:59:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.783+0000 7fe98e000640 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe97c010a90 tx=0x7fe97c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6ad7c3177fa75 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.783+0000 7fe97b400640 1 -- 172.21.15.59:0/413706275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c010430 con 0x556d2df95e40 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.783+0000 7fe97b400640 1 -- 172.21.15.59:0/413706275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c005520 con 0x556d2df95e40 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.783+0000 7fe97b400640 1 -- 172.21.15.59:0/413706275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004920 con 0x556d2df95e40 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- 172.21.15.59:0/413706275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 msgr2=0x556d2df9f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe97c010a90 tx=0x7fe97c005990 comp rx=0 tx=0).stop 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- 172.21.15.59:0/413706275 shutdown_connections 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2dfa0700 0x556d2dfb7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2df9f4d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 --2- 172.21.15.59:0/413706275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2df95520 0x556d2df9df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- 172.21.15.59:0/413706275 >> 172.21.15.59:0/413706275 conn(0x556d2df91ab0 msgr2=0x556d2df9edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- 172.21.15.59:0/413706275 shutdown_connections 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- 172.21.15.59:0/413706275 wait complete. 2024-09-05T11:59:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.784+0000 7fe9982d0740 1 -- start start 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 Processor -- start 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2dfa8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901afba0 con 0x556d2df95e40 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901b1010 con 0x556d2df95520 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901b0270 con 0x556d2dfa0700 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe9982d0740 10 client.0 _ll_register_callbacks cb 0x7fe97c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe98e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57154/0 (socket says 172.21.15.59:57154) 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe98e000640 1 -- 172.21.15.59:0/2663568231 learned_addr learned my addr 172.21.15.59:0/2663568231 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.785+0000 7fe98f400640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2dfa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98f400640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 -- 172.21.15.59:0/2663568231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 msgr2=0x556d2dfa5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 -- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 msgr2=0x556d2dfa8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2dfa8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 -- 172.21.15.59:0/2663568231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2df95c70 con 0x556d2df95e40 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98f400640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2dfa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe98e000640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe97c012d50 tx=0x7fe97c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811cbc16869031fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe978c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c026630 con 0x556d2df95e40 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe97c0267d0 con 0x556d2df95e40 2024-09-05T11:59:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.786+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c005520 con 0x556d2df95e40 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe97c0269d0 con 0x556d2df95e40 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 10 client.15660 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556d2e01bae0 con 0x556d2df95e40 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c005520 con 0x556d2df95e40 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe97c036400 con 0x556d2df95e40 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 10 client.15660 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 10 client.15660 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 20 client.15660 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2e01c1f0 0x556d2e01e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 4 client.15660 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b020b871", "release": true, "await": true}] 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.787+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b020b871", "release": true, "await": true}) -- 0x556d2df030d0 con 0x556d2e01c1f0 2024-09-05T11:59:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.788+0000 7fe98f400640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2e01c1f0 0x556d2e01e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.788+0000 7fe98f400640 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2e01c1f0 0x556d2e01e610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671bea93171cb980 server_cookie=86314ccf04907607 in_seq=0 out_seq=0 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.788+0000 7fe978c00640 10 client.15660 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe978c00640 1 -- 172.21.15.59:0/2663568231 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556d2df030d0 con 0x556d2e01c1f0 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe978c00640 10 client.15660 handle_command_reply: tid=0 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 client.15660 shutdown 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 msgr2=0x556d2dfa4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe97c012d50 tx=0x7fe97c009780 comp rx=0 tx=0).stop 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 shutdown_connections 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d2dfa0700 0x556d2dfa5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556d2e01c1f0 0x556d2e01e610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556d2df95e40 0x556d2dfa4c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 --2- 172.21.15.59:0/2663568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d2df95520 0x556d2dfa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 >> 172.21.15.59:0/2663568231 conn(0x556d2df91ab0 msgr2=0x556d2df97790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe978c00640 0 client.15660 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 shutdown_connections 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 1 -- 172.21.15.59:0/2663568231 wait complete. 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.790+0000 7fe9982d0740 20 client.15660 trim_cache size 0 max 16384 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:19.796 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 12, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "b020b871": { 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 12, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 137.7, 2024-09-05T11:59:19.797 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:19.798 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 msgr2=0x7fe9740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 0x7fe9740a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe98009b510 tx=0x7fe9800d2e60 comp rx=0 tx=0).stop 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 msgr2=0x7fe99019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe99019a4c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e9f0 tx=0x7fe98000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 shutdown_connections 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9740a2b80 0x7fe9740a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe990108f60 0x7fe99019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9901047e0 0x7fe99019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 --2- 172.21.15.59:0/1427024076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe990103e30 0x7fe99019a4c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 >> 172.21.15.59:0/1427024076 conn(0x7fe9900ff910 msgr2=0x7fe990100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 shutdown_connections 2024-09-05T11:59:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:19.791+0000 7fe98fe00640 1 -- 172.21.15.59:0/1427024076 wait complete. 2024-09-05T11:59:20.648 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b020b871', total seconds elapsed: 21.6 2024-09-05T11:59:20.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:20 smithi059 ceph-mon[30025]: pgmap v444: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-05T11:59:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:20 smithi189 ceph-mon[34772]: pgmap v444: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-05T11:59:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:20 smithi089 ceph-mon[34499]: pgmap v444: 129 pgs: 129 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-05T11:59:22.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:22 smithi059 ceph-mon[30025]: pgmap v445: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:59:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:22 smithi189 ceph-mon[34772]: pgmap v445: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:59:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:22 smithi089 ceph-mon[34499]: pgmap v445: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T11:59:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:24 smithi059 ceph-mon[30025]: pgmap v446: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:24 smithi189 ceph-mon[34772]: pgmap v446: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:24 smithi089 ceph-mon[34499]: pgmap v446: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:26 smithi059 ceph-mon[30025]: pgmap v447: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:26 smithi189 ceph-mon[34772]: pgmap v447: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:26 smithi089 ceph-mon[34499]: pgmap v447: 129 pgs: 129 active+clean; 906 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T11:59:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:28 smithi059 ceph-mon[30025]: pgmap v448: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T11:59:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:28 smithi189 ceph-mon[34772]: pgmap v448: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T11:59:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:28 smithi089 ceph-mon[34499]: pgmap v448: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T11:59:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:30 smithi189 ceph-mon[34772]: pgmap v449: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 4 op/s 2024-09-05T11:59:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:30 smithi089 ceph-mon[34499]: pgmap v449: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 4 op/s 2024-09-05T11:59:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:30 smithi059 ceph-mon[30025]: pgmap v449: 129 pgs: 129 active+clean; 964 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 KiB/s wr, 4 op/s 2024-09-05T11:59:32.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:32 smithi089 ceph-mon[34499]: pgmap v450: 129 pgs: 129 active+clean; 1017 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:32 smithi189 ceph-mon[34772]: pgmap v450: 129 pgs: 129 active+clean; 1017 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:32 smithi059 ceph-mon[30025]: pgmap v450: 129 pgs: 129 active+clean; 1017 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:34 smithi189 ceph-mon[34772]: pgmap v451: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:34 smithi089 ceph-mon[34499]: pgmap v451: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:34 smithi059 ceph-mon[30025]: pgmap v451: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:36 smithi189 ceph-mon[34772]: pgmap v452: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:36 smithi089 ceph-mon[34499]: pgmap v452: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:36 smithi059 ceph-mon[30025]: pgmap v452: 129 pgs: 129 active+clean; 1017 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T11:59:38.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:59:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:38 smithi059 ceph-mon[30025]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T11:59:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:38 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:59:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:38 smithi189 ceph-mon[34772]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T11:59:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:38 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 -- 172.21.15.59:0/1528549742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c102340 msgr2=0x7fdb6c10e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 --2- 172.21.15.59:0/1528549742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c102340 0x7fdb6c10e870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009cd0 tx=0x7fdb6402a8f0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 -- 172.21.15.59:0/1528549742 shutdown_connections 2024-09-05T11:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 --2- 172.21.15.59:0/1528549742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c102340 0x7fdb6c10e870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 --2- 172.21.15.59:0/1528549742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 --2- 172.21.15.59:0/1528549742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c1079a0 0x7fdb6c107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 -- 172.21.15.59:0/1528549742 >> 172.21.15.59:0/1528549742 conn(0x7fdb6c0fd680 msgr2=0x7fdb6c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 -- 172.21.15.59:0/1528549742 shutdown_connections 2024-09-05T11:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.167+0000 7fdb71e00640 1 -- 172.21.15.59:0/1528549742 wait complete. 2024-09-05T11:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.168+0000 7fdb71e00640 1 Processor -- start 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 -- start start 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c102340 0x7fdb6c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c1079a0 0x7fdb6c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c111df0 con 0x7fdb6c1079a0 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c111c70 con 0x7fdb6c102340 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c111f70 con 0x7fdb6c1019a0 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c1079a0 0x7fdb6c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41030/0 (socket says 172.21.15.59:41030) 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb70a00640 1 -- 172.21.15.59:0/3830754933 learned_addr learned my addr 172.21.15.59:0/3830754933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.169+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c102340 0x7fdb6c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 -- 172.21.15.59:0/3830754933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c102340 msgr2=0x7fdb6c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c102340 0x7fdb6c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 -- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c1079a0 msgr2=0x7fdb6c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c1079a0 0x7fdb6c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 -- 172.21.15.59:0/3830754933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c19f420 con 0x7fdb6c1019a0 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e9f0 tx=0x7fdb6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb69600640 1 -- 172.21.15.59:0/3830754933 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000cd80 con 0x7fdb6c1019a0 2024-09-05T11:59:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb69600640 1 -- 172.21.15.59:0/3830754933 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60010910 con 0x7fdb6c1019a0 2024-09-05T11:59:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb69600640 1 -- 172.21.15.59:0/3830754933 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60018990 con 0x7fdb6c1019a0 2024-09-05T11:59:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.170+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c19f710 con 0x7fdb6c1019a0 2024-09-05T11:59:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.171+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a6ff0 con 0x7fdb6c1019a0 2024-09-05T11:59:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.174+0000 7fdb75e8e740 1 Processor -- start 2024-09-05T11:59:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 -- start start 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x564933033320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564932f8c3e0 con 0x564933025d50 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564932f33d80 con 0x564933026670 2024-09-05T11:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564932f883a0 con 0x564933030f30 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb69600640 1 -- 172.21.15.59:0/3830754933 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb60010ac0 con 0x7fdb6c1019a0 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.175+0000 7fdb71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47776/0 (socket says 172.21.15.59:47776) 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 -- 172.21.15.59:0/919556800 learned_addr learned my addr 172.21.15.59:0/919556800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb70a00640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb6be00640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x564933033320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 -- 172.21.15.59:0/919556800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 msgr2=0x564933033320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x564933033320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 -- 172.21.15.59:0/919556800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 msgr2=0x5649330309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 -- 172.21.15.59:0/919556800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564932f52ae0 con 0x564933025d50 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb70a00640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb6be00640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x564933033320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.176+0000 7fdb71400640 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009cd0 tx=0x7fdb64005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61feef91484a1fad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb5dc00640 1 -- 172.21.15.59:0/919556800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64030760 con 0x564933025d50 2024-09-05T11:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb5dc00640 1 -- 172.21.15.59:0/919556800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64036030 con 0x564933025d50 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb5dc00640 1 -- 172.21.15.59:0/919556800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640043a0 con 0x564933025d50 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 -- 172.21.15.59:0/919556800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 msgr2=0x564933026130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009cd0 tx=0x7fdb64005f80 comp rx=0 tx=0).stop 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 -- 172.21.15.59:0/919556800 shutdown_connections 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x564933033320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/919556800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x564933026130 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 -- 172.21.15.59:0/919556800 >> 172.21.15.59:0/919556800 conn(0x5649330222e0 msgr2=0x5649330226c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb69600640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 0x7fdb540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.177+0000 7fdb75e8e740 1 -- 172.21.15.59:0/919556800 shutdown_connections 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.178+0000 7fdb75e8e740 1 -- 172.21.15.59:0/919556800 wait complete. 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.178+0000 7fdb69600640 1 -- 172.21.15.59:0/3830754933 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb600cf680 con 0x7fdb6c1019a0 2024-09-05T11:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.178+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 0x7fdb540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.178+0000 7fdb75e8e740 1 -- start start 2024-09-05T11:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.178+0000 7fdb75e8e740 1 Processor -- start 2024-09-05T11:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 0x7fdb540a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x564933023660 tx=0x7fdb58005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x56493302e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x56493302e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564933033b40 con 0x564933025d50 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564932f470f0 con 0x564933026670 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564933033cc0 con 0x564933030f30 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb75e8e740 10 client.0 _ll_register_callbacks cb 0x564932f389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47780/0 (socket says 172.21.15.59:47780) 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb70a00640 1 -- 172.21.15.59:0/4110369900 learned_addr learned my addr 172.21.15.59:0/4110369900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb71400640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x56493302e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.179+0000 7fdb6be00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x56493302e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 -- 172.21.15.59:0/4110369900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 msgr2=0x56493302e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x56493302e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 -- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 msgr2=0x56493302e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x56493302e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 -- 172.21.15.59:0/4110369900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564932cfd3c0 con 0x564933025d50 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb70a00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdb600ccd10 tx=0x7fdb6009b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922ca44f18fe4d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb52000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6009fe20 con 0x564933025d50 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb64036030 con 0x564933025d50 2024-09-05T11:59:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb600cf2d0 con 0x564933025d50 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.180+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb600a46e0 con 0x564933025d50 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 10 client.15672 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56493303f860 con 0x564933025d50 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64036030 con 0x564933025d50 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdb600ba2a0 con 0x564933025d50 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 10 client.15672 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 20 client.15672 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564933040d20 0x564933043110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 4 client.15672 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.181+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564932f99950 con 0x564933040d20 2024-09-05T11:59:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.182+0000 7fdb6be00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564933040d20 0x564933043110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.182+0000 7fdb6be00640 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564933040d20 0x564933043110 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9543f36ceb37479 server_cookie=df4695b7a907e4b4 in_seq=0 out_seq=0 2024-09-05T11:59:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.182+0000 7fdb52000640 10 client.15672 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:39.187 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:38 smithi089 ceph-mon[34499]: pgmap v453: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T11:59:39.187 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:38 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T11:59:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb52000640 1 -- 172.21.15.59:0/4110369900 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564932f99950 con 0x564933040d20 2024-09-05T11:59:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb52000640 10 client.15672 handle_command_reply: tid=0 2024-09-05T11:59:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 client.15672 shutdown 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 msgr2=0x56493302de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdb600ccd10 tx=0x7fdb6009b6e0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 shutdown_connections 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933030f30 0x56493302e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x56493302e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564933040d20 0x564933043110 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/4110369900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933025d50 0x56493302de30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 >> 172.21.15.59:0/4110369900 conn(0x5649330222e0 msgr2=0x564933028010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.183+0000 7fdb52000640 0 client.15672 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.184+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 shutdown_connections 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.184+0000 7fdb75e8e740 1 -- 172.21.15.59:0/4110369900 wait complete. 2024-09-05T11:59:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.184+0000 7fdb75e8e740 20 client.15672 trim_cache size 0 max 16384 2024-09-05T11:59:39.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.202+0000 7fdb75e8e740 1 Processor -- start 2024-09-05T11:59:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.202+0000 7fdb75e8e740 1 -- start start 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.202+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x564933048990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933048ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1aef60 con 0x564933030f30 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1b17e0 con 0x564933026670 2024-09-05T11:59:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1af5d0 con 0x564933025d50 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x564933048990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41042/0 (socket says 172.21.15.59:41042) 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb70a00640 1 -- 172.21.15.59:0/684205390 learned_addr learned my addr 172.21.15.59:0/684205390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb71400640 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933048ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 -- 172.21.15.59:0/684205390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 msgr2=0x564933028800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 -- 172.21.15.59:0/684205390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 msgr2=0x564933048ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933048ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb6be00640 1 -- 172.21.15.59:0/684205390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564933023390 con 0x564933026670 2024-09-05T11:59:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.203+0000 7fdb70a00640 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb6be00640 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x564933048990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x564933043900 tx=0x7fdb5801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8fb26aa9d65cb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb5d200640 1 -- 172.21.15.59:0/684205390 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58003a40 con 0x564933026670 2024-09-05T11:59:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb5d200640 1 -- 172.21.15.59:0/684205390 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58003be0 con 0x564933026670 2024-09-05T11:59:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb5d200640 1 -- 172.21.15.59:0/684205390 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5801f680 con 0x564933026670 2024-09-05T11:59:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb75e8e740 1 -- 172.21.15.59:0/684205390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 msgr2=0x564933048990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.204+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x564933048990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x564933043900 tx=0x7fdb5801a000 comp rx=0 tx=0).stop 2024-09-05T11:59:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 -- 172.21.15.59:0/684205390 shutdown_connections 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933048ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x564933048990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/684205390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933028800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 -- 172.21.15.59:0/684205390 >> 172.21.15.59:0/684205390 conn(0x5649330222e0 msgr2=0x56493302cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 -- 172.21.15.59:0/684205390 shutdown_connections 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 -- 172.21.15.59:0/684205390 wait complete. 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 -- start start 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.205+0000 7fdb75e8e740 1 Processor -- start 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933034660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1afca0 con 0x564933030f30 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1b1110 con 0x564933026670 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1b0370 con 0x564933025d50 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb75e8e740 10 client.0 _ll_register_callbacks cb 0x7fdb6009b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933034660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933034660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47806/0 (socket says 172.21.15.59:47806) 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb70a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41056/0 (socket says 172.21.15.59:41056) 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb71400640 1 -- 172.21.15.59:0/3012754304 learned_addr learned my addr 172.21.15.59:0/3012754304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.206+0000 7fdb70a00640 1 -- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 msgr2=0x5649330397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330397a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb70a00640 1 -- 172.21.15.59:0/3012754304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 msgr2=0x564933034660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933034660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb70a00640 1 -- 172.21.15.59:0/3012754304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564933034e90 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb70a00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb600a41b0 tx=0x7fdb600cf4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8755c32001d9a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb52000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60010640 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb600ba2e0 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb58003be0 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb60097870 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb75e8e740 10 client.25618 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.207+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5649330ab570 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58003be0 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdb600a0fb0 con 0x564933025d50 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 10 client.25618 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 20 client.25618 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5649330abc90 0x5649330ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 4 client.25618 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:59:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564932f93e60 con 0x5649330abc90 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.208+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5649330abc90 0x5649330ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb6be00640 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5649330abc90 0x5649330ae0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29465fdcf1667a85 server_cookie=2403def0643780c in_seq=0 out_seq=0 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb52000640 10 client.25618 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb52000640 1 -- 172.21.15.59:0/3012754304 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564932f93e60 con 0x5649330abc90 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb52000640 10 client.25618 handle_command_reply: tid=0 2024-09-05T11:59:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 client.25618 shutdown 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 msgr2=0x564933039260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb600a41b0 tx=0x7fdb600cf4e0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 shutdown_connections 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5649330abc90 0x5649330ae0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564933030f30 0x564933034660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564933026670 0x5649330397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 --2- 172.21.15.59:0/3012754304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564933025d50 0x564933039260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 >> 172.21.15.59:0/3012754304 conn(0x5649330222e0 msgr2=0x56493302f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb52000640 0 client.25618 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 shutdown_connections 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.209+0000 7fdb75e8e740 1 -- 172.21.15.59:0/3012754304 wait complete. 2024-09-05T11:59:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb75e8e740 20 client.25618 trim_cache size 0 max 16384 2024-09-05T11:59:39.216 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:39.216 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T11:59:39.216 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T11:59:39.216 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 msgr2=0x7fdb540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 0x7fdb540a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x564933023660 tx=0x7fdb58005f90 comp rx=0 tx=0).stop 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 msgr2=0x7fdb6c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e9f0 tx=0x7fdb6000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 shutdown_connections 2024-09-05T11:59:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb540a2c00 0x7fdb540a50c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6c1079a0 0x7fdb6c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb6c102340 0x7fdb6c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 --2- 172.21.15.59:0/3830754933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb6c1019a0 0x7fdb6c19a470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.210+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 >> 172.21.15.59:0/3830754933 conn(0x7fdb6c0fd680 msgr2=0x7fdb6c10d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.211+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 shutdown_connections 2024-09-05T11:59:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:39.211+0000 7fdb71e00640 1 -- 172.21.15.59:0/3830754933 wait complete. 2024-09-05T11:59:39.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:59:39.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:59:39.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:59:40.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:59:40.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:59:40.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:59:40.138 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T11:59:40.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T11:59:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T11:59:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T11:59:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T11:59:40.516 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:40.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:40 smithi059 ceph-mon[30025]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:41.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 -- 172.21.15.59:0/2988428679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec1047e0 msgr2=0x7fdcec108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/2988428679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec1047e0 0x7fdcec108830 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd802a400 comp rx=0 tx=0).stop 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 -- 172.21.15.59:0/2988428679 shutdown_connections 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/2988428679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec108f60 0x7fdcec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/2988428679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec1047e0 0x7fdcec108830 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/2988428679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec103e30 0x7fdcec104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 -- 172.21.15.59:0/2988428679 >> 172.21.15.59:0/2988428679 conn(0x7fdcec0ff910 msgr2=0x7fdcec101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 -- 172.21.15.59:0/2988428679 shutdown_connections 2024-09-05T11:59:41.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.073+0000 7fdcf2e00640 1 -- 172.21.15.59:0/2988428679 wait complete. 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.074+0000 7fdcf2e00640 1 Processor -- start 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.074+0000 7fdcf2e00640 1 -- start start 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.074+0000 7fdcf2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.074+0000 7fdcf2e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 0x7fdcec19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.074+0000 7fdcf2e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec108f60 0x7fdcec19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf2e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec114010 con 0x7fdcec108f60 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf2e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec113e90 con 0x7fdcec1047e0 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec114190 con 0x7fdcec103e30 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 0x7fdcec19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:39186/0 (socket says 172.21.15.59:39186) 2024-09-05T11:59:41.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 -- 172.21.15.59:0/3261152985 learned_addr learned my addr 172.21.15.59:0/3261152985 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf2400640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec108f60 0x7fdcec19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 -- 172.21.15.59:0/3261152985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 msgr2=0x7fdcec19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 0x7fdcec19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 -- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec108f60 msgr2=0x7fdcec19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec108f60 0x7fdcec19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1a00640 1 -- 172.21.15.59:0/3261152985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec19f410 con 0x7fdcec103e30 2024-09-05T11:59:41.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.075+0000 7fdcf1000640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 0x7fdcec19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcf1a00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdce000e9f0 tx=0x7fdce000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce000cd80 con 0x7fdcec103e30 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec19f700 con 0x7fdcec103e30 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce0010910 con 0x7fdcec103e30 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00188f0 con 0x7fdcec103e30 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.076+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a6fe0 con 0x7fdcec103e30 2024-09-05T11:59:41.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.077+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec105310 con 0x7fdcec103e30 2024-09-05T11:59:41.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.079+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdce0010ac0 con 0x7fdcec103e30 2024-09-05T11:59:41.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.081+0000 7fdcea000640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 0x7fdcd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:41.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.081+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdce00cfda0 con 0x7fdcec103e30 2024-09-05T11:59:41.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.081+0000 7fdcf1000640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 0x7fdcd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:41.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.082+0000 7fdcf1000640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 0x7fdcd40a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd8031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:41.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.083+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdce009c240 con 0x7fdcec103e30 2024-09-05T11:59:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:40 smithi189 ceph-mon[34772]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:40 smithi089 ceph-mon[34499]: pgmap v454: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:41.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.306+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcec104210 con 0x7fdcec103e30 2024-09-05T11:59:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.307+0000 7fdcea000640 1 -- 172.21.15.59:0/3261152985 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fdce00a0540 con 0x7fdcec103e30 2024-09-05T11:59:41.309 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T11:59:41.309 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T11:59:41.310 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T11:59:41.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 msgr2=0x7fdcd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 0x7fdcd40a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd8031510 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 msgr2=0x7fdcec19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdce000e9f0 tx=0x7fdce000eeb0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 shutdown_connections 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdcd40a2b80 0x7fdcd40a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcec108f60 0x7fdcec19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdcec1047e0 0x7fdcec19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 --2- 172.21.15.59:0/3261152985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec103e30 0x7fdcec19a460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 >> 172.21.15.59:0/3261152985 conn(0x7fdcec0ff910 msgr2=0x7fdcec100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 shutdown_connections 2024-09-05T11:59:41.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:41.310+0000 7fdcf2e00640 1 -- 172.21.15.59:0/3261152985 wait complete. 2024-09-05T11:59:41.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3261152985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:42.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3261152985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:42.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T11:59:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3261152985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T11:59:42.556 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:42 smithi059 ceph-mon[30025]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:42 smithi189 ceph-mon[34772]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:43.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 -- 172.21.15.59:0/3108073322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 msgr2=0x7f87c8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 --2- 172.21.15.59:0/3108073322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c8076b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009900 tx=0x7f87b802a3f0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 -- 172.21.15.59:0/3108073322 shutdown_connections 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 --2- 172.21.15.59:0/3108073322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 0x7f87c8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 --2- 172.21.15.59:0/3108073322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 0x7f87c81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 --2- 172.21.15.59:0/3108073322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c8076b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 -- 172.21.15.59:0/3108073322 >> 172.21.15.59:0/3108073322 conn(0x7f87c80ffa90 msgr2=0x7f87c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 -- 172.21.15.59:0/3108073322 shutdown_connections 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.138+0000 7f87cd000640 1 -- 172.21.15.59:0/3108073322 wait complete. 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 Processor -- start 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 -- start start 2024-09-05T11:59:43.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 0x7f87c81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 0x7f87c819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8118610 con 0x7f87c8076720 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8118490 con 0x7f87c8112a30 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87cd000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8118790 con 0x7f87c8077040 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.139+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40862/0 (socket says 172.21.15.59:40862) 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 -- 172.21.15.59:0/3454831514 learned_addr learned my addr 172.21.15.59:0/3454831514 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87bd400640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 0x7f87c81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c6800640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 0x7f87c819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 -- 172.21.15.59:0/3454831514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 msgr2=0x7f87c81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 0x7f87c81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 -- 172.21.15.59:0/3454831514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 msgr2=0x7f87c819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 0x7f87c819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 -- 172.21.15.59:0/3454831514 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c819f000 con 0x7f87c8076720 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.140+0000 7f87c5e00640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87b8004770 tx=0x7f87b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.141+0000 7f87bf400640 1 -- 172.21.15.59:0/3454831514 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b803f070 con 0x7f87c8076720 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.141+0000 7f87bf400640 1 -- 172.21.15.59:0/3454831514 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b8031ca0 con 0x7f87c8076720 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.141+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c819f290 con 0x7f87c8076720 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.141+0000 7f87bf400640 1 -- 172.21.15.59:0/3454831514 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8035030 con 0x7f87c8076720 2024-09-05T11:59:43.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.141+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81ab5e0 con 0x7f87c8076720 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 Processor -- start 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 -- start start 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87bf400640 1 -- 172.21.15.59:0/3454831514 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87b8042070 con 0x7f87c8076720 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99101050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac9910b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac9910e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac990673e0 con 0x55ac99100c70 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac9900ed80 con 0x55ac9910be50 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.145+0000 7f87cfa14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac990633a0 con 0x55ac99101590 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87c6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87c6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99101050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40878/0 (socket says 172.21.15.59:40878) 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87c6800640 1 -- 172.21.15.59:0/3938325872 learned_addr learned my addr 172.21.15.59:0/3938325872 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac9910e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87c5e00640 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac9910b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 -- 172.21.15.59:0/3938325872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 msgr2=0x55ac9910b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac9910b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 -- 172.21.15.59:0/3938325872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 msgr2=0x55ac99101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.146+0000 7f87bd400640 1 -- 172.21.15.59:0/3938325872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9902dae0 con 0x55ac9910be50 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87bd400640 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac9910e240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87c81a00c0 tx=0x7f87b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a365cffc54dec7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87b7400640 1 -- 172.21.15.59:0/3938325872 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004230 con 0x55ac9910be50 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87b7400640 1 -- 172.21.15.59:0/3938325872 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b0009e10 con 0x55ac9910be50 2024-09-05T11:59:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87b7400640 1 -- 172.21.15.59:0/3938325872 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004d70 con 0x55ac9910be50 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87bf400640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 0x7f87ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.147+0000 7f87bf400640 1 -- 172.21.15.59:0/3454831514 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f87b80ef620 con 0x7f87c8076720 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87c6800640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 0x7f87ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 -- 172.21.15.59:0/3938325872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 msgr2=0x55ac9910e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac9910e240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87c81a00c0 tx=0x7f87b0011810 comp rx=0 tx=0).stop 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 -- 172.21.15.59:0/3938325872 shutdown_connections 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac9910e240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac9910b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 --2- 172.21.15.59:0/3938325872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 -- 172.21.15.59:0/3938325872 >> 172.21.15.59:0/3938325872 conn(0x55ac990fd200 msgr2=0x55ac990fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 -- 172.21.15.59:0/3938325872 shutdown_connections 2024-09-05T11:59:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87cfa14740 1 -- 172.21.15.59:0/3938325872 wait complete. 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.148+0000 7f87c6800640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 0x7f87ac0a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87c0011630 tx=0x7f87c0009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 -- start start 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 Processor -- start 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 0x55ac99108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 0x55ac991092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99109810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac9910ea20 con 0x55ac99101590 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac990220f0 con 0x55ac9910be50 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac9910eba0 con 0x55ac99100c70 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.149+0000 7f87cfa14740 10 client.0 _ll_register_callbacks cb 0x55ac99009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 0x55ac991092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 0x55ac991092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40892/0 (socket says 172.21.15.59:40892) 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87bd400640 1 -- 172.21.15.59:0/2860255031 learned_addr learned my addr 172.21.15.59:0/2860255031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 0x55ac99108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c6800640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 -- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 msgr2=0x55ac99109810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99109810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 -- 172.21.15.59:0/2860255031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 msgr2=0x55ac991092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 0x55ac991092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 -- 172.21.15.59:0/2860255031 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac98dd83c0 con 0x55ac99100c70 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c6800640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87c5e00640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 0x55ac99108d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87b80ecd60 tx=0x7f87b8031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b410217e4dda304 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.150+0000 7f87abe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8049070 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f87b80b9ee0 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b0009e10 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87b80bce60 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87cfa14740 10 client.25636 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac9910ea80 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b0009e10 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.151+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f87b804c060 con 0x55ac99100c70 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 10 client.25636 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 20 client.25636 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac9911bf30 0x55ac9911e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 4 client.25636 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac99074950 con 0x55ac9911bf30 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.152+0000 7f87bd400640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac9911bf30 0x55ac9911e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.153+0000 7f87bd400640 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac9911bf30 0x55ac9911e390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e7185356428a6a server_cookie=e9f95352b7b90f7b in_seq=0 out_seq=0 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.153+0000 7f87abe00640 10 client.25636 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87abe00640 1 -- 172.21.15.59:0/2860255031 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac99074950 con 0x55ac9911bf30 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87abe00640 10 client.25636 handle_command_reply: tid=0 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 client.25636 shutdown 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 msgr2=0x55ac99108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 0x55ac99108d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87b80ecd60 tx=0x7f87b8031f10 comp rx=0 tx=0).stop 2024-09-05T11:59:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 shutdown_connections 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac9911bf30 0x55ac9911e390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99101590 0x55ac991092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 --2- 172.21.15.59:0/2860255031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99100c70 0x55ac99108d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87abe00640 0 client.25636 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 >> 172.21.15.59:0/2860255031 conn(0x55ac990fd200 msgr2=0x55ac99102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 shutdown_connections 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 1 -- 172.21.15.59:0/2860255031 wait complete. 2024-09-05T11:59:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.154+0000 7f87cfa14740 20 client.25636 trim_cache size 0 max 16384 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.168+0000 7f87cfa14740 1 Processor -- start 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.168+0000 7f87cfa14740 1 -- start start 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.168+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99105ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87cfa14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b34f0 con 0x55ac99100c70 2024-09-05T11:59:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87cfa14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b5d70 con 0x55ac9910be50 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87cfa14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b3b60 con 0x55ac99101590 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40908/0 (socket says 172.21.15.59:40908) 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 -- 172.21.15.59:0/1985649119 learned_addr learned my addr 172.21.15.59:0/1985649119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c6800640 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99105ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87bd400640 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 -- 172.21.15.59:0/1985649119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 msgr2=0x55ac991241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 -- 172.21.15.59:0/1985649119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 msgr2=0x55ac99105ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99105ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.169+0000 7f87c5e00640 1 -- 172.21.15.59:0/1985649119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac990fde90 con 0x55ac99100c70 2024-09-05T11:59:43.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.170+0000 7f87c5e00640 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87b80b8040 tx=0x7f87b803b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c229aa1ef21c1ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.170+0000 7f87b6a00640 1 -- 172.21.15.59:0/1985649119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8049070 con 0x55ac99100c70 2024-09-05T11:59:43.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.170+0000 7f87b6a00640 1 -- 172.21.15.59:0/1985649119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b80ef300 con 0x55ac99100c70 2024-09-05T11:59:43.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.170+0000 7f87b6a00640 1 -- 172.21.15.59:0/1985649119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80a6040 con 0x55ac99100c70 2024-09-05T11:59:43.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- 172.21.15.59:0/1985649119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 msgr2=0x55ac99123ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87b80b8040 tx=0x7f87b803b890 comp rx=0 tx=0).stop 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- 172.21.15.59:0/1985649119 shutdown_connections 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac9910be50 0x55ac99105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 --2- 172.21.15.59:0/1985649119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac99100c70 0x55ac99123ca0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- 172.21.15.59:0/1985649119 >> 172.21.15.59:0/1985649119 conn(0x55ac990fd200 msgr2=0x55ac99102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- 172.21.15.59:0/1985649119 shutdown_connections 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- 172.21.15.59:0/1985649119 wait complete. 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 -- start start 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.171+0000 7f87cfa14740 1 Processor -- start 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9910be50 0x55ac99110610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b4230 con 0x55ac9910be50 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b56a0 con 0x55ac99100c70 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b4900 con 0x55ac99101590 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87cfa14740 10 client.0 _ll_register_callbacks cb 0x7f87b80f5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87c6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9910be50 0x55ac99110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45232/0 (socket says 172.21.15.59:45232) 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87c5e00640 1 -- 172.21.15.59:0/4282509845 learned_addr learned my addr 172.21.15.59:0/4282509845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.172+0000 7f87bd400640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 -- 172.21.15.59:0/4282509845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 msgr2=0x55ac991100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 -- 172.21.15.59:0/4282509845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9910be50 msgr2=0x55ac99110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9910be50 0x55ac99110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 -- 172.21.15.59:0/4282509845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac991021f0 con 0x55ac99100c70 2024-09-05T11:59:43.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87bd400640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87c5e00640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87b80bb8e0 tx=0x7f87b8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c7a9c4ed979b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87abe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80ab060 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b80ef300 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f87b803b4f0 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87b80b2050 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87cfa14740 10 client.25724 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.173+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac991021f0 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b80ef300 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f87b80c4310 con 0x55ac99100c70 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 10 client.25724 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 20 client.25724 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:43.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac99187070 0x55ac99189460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 4 client.25724 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac9906ee60 con 0x55ac99187070 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.174+0000 7f87bd400640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac99187070 0x55ac99189460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87bd400640 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac99187070 0x55ac99189460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7326d26b26e7524a server_cookie=1d098283428963f3 in_seq=0 out_seq=0 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87abe00640 10 client.25724 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87abe00640 1 -- 172.21.15.59:0/4282509845 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac9906ee60 con 0x55ac99187070 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87abe00640 10 client.25724 handle_command_reply: tid=0 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 client.25724 shutdown 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 msgr2=0x55ac9910fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87b80bb8e0 tx=0x7f87b8005e50 comp rx=0 tx=0).stop 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 shutdown_connections 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac99187070 0x55ac99189460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac9910be50 0x55ac99110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac99101590 0x55ac991100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 --2- 172.21.15.59:0/4282509845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac99100c70 0x55ac9910fb90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87abe00640 0 client.25724 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 >> 172.21.15.59:0/4282509845 conn(0x55ac990fd200 msgr2=0x55ac990fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 shutdown_connections 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 1 -- 172.21.15.59:0/4282509845 wait complete. 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.175+0000 7f87cfa14740 20 client.25724 trim_cache size 0 max 16384 2024-09-05T11:59:43.182 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T11:59:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 msgr2=0x7f87ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 0x7f87ac0a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87c0011630 tx=0x7f87c0009500 comp rx=0 tx=0).stop 2024-09-05T11:59:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 msgr2=0x7f87c81a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87b8004770 tx=0x7f87b8003c10 comp rx=0 tx=0).stop 2024-09-05T11:59:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 shutdown_connections 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87ac0a2c00 0x7f87ac0a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87c8112a30 0x7f87c819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8077040 0x7f87c81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 --2- 172.21.15.59:0/3454831514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87c8076720 0x7f87c81a4710 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 >> 172.21.15.59:0/3454831514 conn(0x7f87c80ffa90 msgr2=0x7f87c8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 shutdown_connections 2024-09-05T11:59:43.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:43.176+0000 7f87cd000640 1 -- 172.21.15.59:0/3454831514 wait complete. 2024-09-05T11:59:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:42 smithi089 ceph-mon[34499]: pgmap v455: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:45.026 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T11:59:45.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T11:59:45.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:44 smithi059 ceph-mon[30025]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:44 smithi189 ceph-mon[34772]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:44 smithi089 ceph-mon[34499]: pgmap v456: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:46.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 -- 172.21.15.59:0/1443197221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46941019c0 msgr2=0x7f4694101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 --2- 172.21.15.59:0/1443197221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46941019c0 0x7f4694101e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4680009860 tx=0x7f468002a340 comp rx=0 tx=0).stop 2024-09-05T11:59:46.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 -- 172.21.15.59:0/1443197221 shutdown_connections 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 --2- 172.21.15.59:0/1443197221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4694102360 0x7f469410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 --2- 172.21.15.59:0/1443197221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46941019c0 0x7f4694101e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 --2- 172.21.15.59:0/1443197221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941079c0 0x7f4694107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.015+0000 7f4698c00640 1 -- 172.21.15.59:0/1443197221 >> 172.21.15.59:0/1443197221 conn(0x7f46940fd740 msgr2=0x7f46940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.016+0000 7f4698c00640 1 -- 172.21.15.59:0/1443197221 shutdown_connections 2024-09-05T11:59:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.016+0000 7f4698c00640 1 -- 172.21.15.59:0/1443197221 wait complete. 2024-09-05T11:59:46.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.016+0000 7f4698c00640 1 Processor -- start 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 -- start start 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 0x7f469419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46941079c0 0x7f469419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4694111d90 con 0x7f46941019c0 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4694111c10 con 0x7f4694102360 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4698c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4694111f10 con 0x7f46941079c0 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4692a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4693400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 0x7f469419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4692a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45242/0 (socket says 172.21.15.59:45242) 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.017+0000 7f4692a00640 1 -- 172.21.15.59:0/782185872 learned_addr learned my addr 172.21.15.59:0/782185872 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4693e00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46941079c0 0x7f469419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 -- 172.21.15.59:0/782185872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46941079c0 msgr2=0x7f469419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46941079c0 0x7f469419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 -- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 msgr2=0x7f469419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 0x7f469419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 -- 172.21.15.59:0/782185872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469419f460 con 0x7f4694102360 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4693400640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 0x7f469419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.018+0000 7f4692a00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4680035040 tx=0x7f4680033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.019+0000 7f468be00640 1 -- 172.21.15.59:0/782185872 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468003f020 con 0x7f4694102360 2024-09-05T11:59:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.019+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469419f6f0 con 0x7f4694102360 2024-09-05T11:59:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.019+0000 7f468be00640 1 -- 172.21.15.59:0/782185872 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680004040 con 0x7f4694102360 2024-09-05T11:59:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.019+0000 7f468be00640 1 -- 172.21.15.59:0/782185872 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680037040 con 0x7f4694102360 2024-09-05T11:59:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.019+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a7030 con 0x7f4694102360 2024-09-05T11:59:46.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 Processor -- start 2024-09-05T11:59:46.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 -- start start 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f468be00640 1 -- 172.21.15.59:0/782185872 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4680042020 con 0x7f4694102360 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40f0aa290 con 0x55e40f14ee90 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e40f051b40 con 0x55e40f1445d0 2024-09-05T11:59:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.022+0000 7f469cc7c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e40f0a6250 con 0x55e40f143cb0 2024-09-05T11:59:46.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:39282/0 (socket says 172.21.15.59:39282) 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 -- 172.21.15.59:0/1517383704 learned_addr learned my addr 172.21.15.59:0/1517383704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4692a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693400640 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 -- 172.21.15.59:0/1517383704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 msgr2=0x55e40f14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 -- 172.21.15.59:0/1517383704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 msgr2=0x55e40f151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 -- 172.21.15.59:0/1517383704 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40ee1b6a0 con 0x55e40f143cb0 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.023+0000 7f4693e00640 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9870e800d75dfb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f468be00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 0x7f46780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f467fe00640 1 -- 172.21.15.59:0/1517383704 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c004d20 con 0x55e40f143cb0 2024-09-05T11:59:46.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f467fe00640 1 -- 172.21.15.59:0/1517383704 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c004050 con 0x55e40f143cb0 2024-09-05T11:59:46.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f467fe00640 1 -- 172.21.15.59:0/1517383704 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c005240 con 0x55e40f143cb0 2024-09-05T11:59:46.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f468be00640 1 -- 172.21.15.59:0/782185872 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f46800ef570 con 0x7f4694102360 2024-09-05T11:59:46.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.024+0000 7f4693400640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 0x7f46780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- 172.21.15.59:0/1517383704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 msgr2=0x55e40f144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c0118d0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- 172.21.15.59:0/1517383704 shutdown_connections 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 --2- 172.21.15.59:0/1517383704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f144090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- 172.21.15.59:0/1517383704 >> 172.21.15.59:0/1517383704 conn(0x55e40f140240 msgr2=0x55e40f140620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f4693400640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 0x7f46780a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4684009530 tx=0x7f4684014040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- 172.21.15.59:0/1517383704 shutdown_connections 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- 172.21.15.59:0/1517383704 wait complete. 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.025+0000 7f469cc7c740 1 -- start start 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 Processor -- start 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40f151b00 con 0x55e40f14ee90 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e40f064fa0 con 0x55e40f1445d0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e40f151c80 con 0x55e40f143cb0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f469cc7c740 10 client.0 _ll_register_callbacks cb 0x55e40edb8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f4693400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.026+0000 7f4693400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:39298/0 (socket says 172.21.15.59:39298) 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4692a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 -- 172.21.15.59:0/3469438901 learned_addr learned my addr 172.21.15.59:0/3469438901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693e00640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 -- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 msgr2=0x55e40f14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 -- 172.21.15.59:0/3469438901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 msgr2=0x55e40f14bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f14bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 -- 172.21.15.59:0/3469438901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40f14c7a0 con 0x55e40f143cb0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4692a00640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693e00640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f4693400640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46840058c0 tx=0x7f4684004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777bcaf6e2a6d334 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.027+0000 7f467cc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468401ee20 con 0x55e40f143cb0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c004050 con 0x55e40f143cb0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f468401c610 con 0x55e40f143cb0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4684033050 con 0x55e40f143cb0 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f469cc7c740 10 client.25654 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:46.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e40f15dac0 con 0x55e40f143cb0 2024-09-05T11:59:46.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.028+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c004050 con 0x55e40f143cb0 2024-09-05T11:59:46.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f46840289f0 con 0x55e40f143cb0 2024-09-05T11:59:46.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 10 client.25654 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 20 client.25654 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f15eed0 0x55e40f161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 4 client.25654 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40f0b7800 con 0x55e40f15eed0 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.029+0000 7f4692a00640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f15eed0 0x55e40f161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.030+0000 7f4692a00640 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f15eed0 0x55e40f161330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69fc876cdc8440a server_cookie=a30d1e90d8717bb5 in_seq=0 out_seq=0 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.030+0000 7f467cc00640 10 client.25654 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f467cc00640 1 -- 172.21.15.59:0/3469438901 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e40f0b7800 con 0x55e40f15eed0 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f467cc00640 10 client.25654 handle_command_reply: tid=0 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 client.25654 shutdown 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 msgr2=0x55e40f14e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46840058c0 tx=0x7f4684004730 comp rx=0 tx=0).stop 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 shutdown_connections 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f15eed0 0x55e40f161330 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f14ee90 0x55e40f14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 --2- 172.21.15.59:0/3469438901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f143cb0 0x55e40f14e7b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f467cc00640 0 client.25654 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 >> 172.21.15.59:0/3469438901 conn(0x55e40f140240 msgr2=0x55e40f145f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 shutdown_connections 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 1 -- 172.21.15.59:0/3469438901 wait complete. 2024-09-05T11:59:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.031+0000 7f469cc7c740 20 client.25654 trim_cache size 0 max 16384 2024-09-05T11:59:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.049+0000 7f469cc7c740 1 Processor -- start 2024-09-05T11:59:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.049+0000 7f469cc7c740 1 -- start start 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f156f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46941aef90 con 0x55e40f143cb0 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46941b1810 con 0x55e40f1445d0 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f469cc7c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46941af600 con 0x55e40f14ee90 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f4693400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f4693400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40964/0 (socket says 172.21.15.59:40964) 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f4693400640 1 -- 172.21.15.59:0/198545718 learned_addr learned my addr 172.21.15.59:0/198545718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f4692a00640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.050+0000 7f4693e00640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 -- 172.21.15.59:0/198545718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 msgr2=0x55e40f156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 -- 172.21.15.59:0/198545718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 msgr2=0x55e40f14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 -- 172.21.15.59:0/198545718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40f14c5a0 con 0x55e40f143cb0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4692a00640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f4693400640 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4684006010 tx=0x7f468400f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299184587aca3905 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f467f400640 1 -- 172.21.15.59:0/198545718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840122f0 con 0x55e40f143cb0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f467f400640 1 -- 172.21.15.59:0/198545718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468403c070 con 0x55e40f143cb0 2024-09-05T11:59:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.051+0000 7f467f400640 1 -- 172.21.15.59:0/198545718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468401cbe0 con 0x55e40f143cb0 2024-09-05T11:59:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 -- 172.21.15.59:0/198545718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 msgr2=0x55e40f13f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4684006010 tx=0x7f468400f5e0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 -- 172.21.15.59:0/198545718 shutdown_connections 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 --2- 172.21.15.59:0/198545718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f13f0c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 -- 172.21.15.59:0/198545718 >> 172.21.15.59:0/198545718 conn(0x55e40f140240 msgr2=0x55e40f15a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 -- 172.21.15.59:0/198545718 shutdown_connections 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.052+0000 7f469cc7c740 1 -- 172.21.15.59:0/198545718 wait complete. 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 -- start start 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 Processor -- start 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f1c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46941afcd0 con 0x55e40f143cb0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46941b1140 con 0x55e40f1445d0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46941b03a0 con 0x55e40f14ee90 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.053+0000 7f469cc7c740 10 client.0 _ll_register_callbacks cb 0x7f4684009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4693400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45278/0 (socket says 172.21.15.59:45278) 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 -- 172.21.15.59:0/996923951 learned_addr learned my addr 172.21.15.59:0/996923951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4693e00640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f1c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 -- 172.21.15.59:0/996923951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 msgr2=0x55e40f1c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f1c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 -- 172.21.15.59:0/996923951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 msgr2=0x55e40f1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 -- 172.21.15.59:0/996923951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40f1c32b0 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f4692a00640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46800b8340 tx=0x7f4680033f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6dfb7ac97abca42 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f467cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.054+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800bb790 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f46800041f0 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468403c070 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f46800bb320 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 10 client.25742 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e40f1ca700 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468403c070 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f468003a030 con 0x55e40f1445d0 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 10 client.25742 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 20 client.25742 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f1cae30 0x55e40f1cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 4 client.25742 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.055+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e40f0b1d10 con 0x55e40f1cae30 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.056+0000 7f4693400640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f1cae30 0x55e40f1cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.056+0000 7f4693400640 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f1cae30 0x55e40f1cd250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5196abbc2f7032a1 server_cookie=8e0601bc20a21191 in_seq=0 out_seq=0 2024-09-05T11:59:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.056+0000 7f467cc00640 10 client.25742 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f467cc00640 1 -- 172.21.15.59:0/996923951 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e40f0b1d10 con 0x55e40f1cae30 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f467cc00640 10 client.25742 handle_command_reply: tid=0 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 client.25742 shutdown 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 msgr2=0x55e40f155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46800b8340 tx=0x7f4680033f50 comp rx=0 tx=0).stop 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 shutdown_connections 2024-09-05T11:59:46.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40f14ee90 0x55e40f1c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40f1445d0 0x55e40f155b10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e40f1cae30 0x55e40f1cd250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 --2- 172.21.15.59:0/996923951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40f143cb0 0x55e40f1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 >> 172.21.15.59:0/996923951 conn(0x55e40f140240 msgr2=0x55e40f14d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f467cc00640 0 client.25742 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 shutdown_connections 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.057+0000 7f469cc7c740 1 -- 172.21.15.59:0/996923951 wait complete. 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f469cc7c740 20 client.25742 trim_cache size 0 max 16384 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:46.063 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T11:59:46.064 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "ff0b2efd-d44d-471b-89b0-c5024536f3a7", 2024-09-05T11:59:46.064 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T11:59:46.064 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:46.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 msgr2=0x7f46780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 0x7f46780a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4684009530 tx=0x7f4684014040 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 msgr2=0x7f469419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4680035040 tx=0x7f4680033b60 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 shutdown_connections 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f46780a2b80 0x7f46780a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46941079c0 0x7f469419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4694102360 0x7f469419a9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 --2- 172.21.15.59:0/782185872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46941019c0 0x7f469419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.058+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 >> 172.21.15.59:0/782185872 conn(0x7f46940fd740 msgr2=0x7f46940fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.059+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 shutdown_connections 2024-09-05T11:59:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:46.059+0000 7f4698c00640 1 -- 172.21.15.59:0/782185872 wait complete. 2024-09-05T11:59:46.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T11:59:47.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:46 smithi059 ceph-mon[30025]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:46 smithi189 ceph-mon[34772]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:46 smithi089 ceph-mon[34499]: pgmap v457: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T11:59:47.866 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:47 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T11:59:47.866 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:47 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:47.866 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:47 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T11:59:47.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:47 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 -- 172.21.15.59:0/742835542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 msgr2=0x7f1a481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/742835542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a481067f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009860 tx=0x7f1a3002a340 comp rx=0 tx=0).stop 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 -- 172.21.15.59:0/742835542 shutdown_connections 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/742835542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48106f20 0x7f1a4810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/742835542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a481067f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/742835542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48101df0 0x7f1a481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 -- 172.21.15.59:0/742835542 >> 172.21.15.59:0/742835542 conn(0x7f1a480fd8b0 msgr2=0x7f1a480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 -- 172.21.15.59:0/742835542 shutdown_connections 2024-09-05T11:59:47.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.979+0000 7f1a4ec00640 1 -- 172.21.15.59:0/742835542 wait complete. 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 Processor -- start 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 -- start start 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a4819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 0x7f1a4819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111f80 con 0x7f1a48106f20 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111e00 con 0x7f1a48101df0 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.980+0000 7f1a4ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48112100 con 0x7f1a481027a0 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45290/0 (socket says 172.21.15.59:45290) 2024-09-05T11:59:47.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 -- 172.21.15.59:0/3258162633 learned_addr learned my addr 172.21.15.59:0/3258162633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a4819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 0x7f1a4819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 -- 172.21.15.59:0/3258162633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 msgr2=0x7f1a4819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a4819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 -- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 msgr2=0x7f1a4819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 0x7f1a4819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 -- 172.21.15.59:0/3258162633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4819f5d0 con 0x7f1a48101df0 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 0x7f1a4819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T11:59:47.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.981+0000 7f1a47e00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800e9c0 tx=0x7f1a3800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:47.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.982+0000 7f1a45600640 1 -- 172.21.15.59:0/3258162633 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800ccc0 con 0x7f1a48101df0 2024-09-05T11:59:47.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.982+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4819f8c0 con 0x7f1a48101df0 2024-09-05T11:59:47.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.982+0000 7f1a45600640 1 -- 172.21.15.59:0/3258162633 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a38005500 con 0x7f1a48101df0 2024-09-05T11:59:47.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.982+0000 7f1a45600640 1 -- 172.21.15.59:0/3258162633 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38010430 con 0x7f1a48101df0 2024-09-05T11:59:47.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.982+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a71a0 con 0x7f1a48101df0 2024-09-05T11:59:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.985+0000 7f1a515cd740 1 Processor -- start 2024-09-05T11:59:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.985+0000 7f1a45600640 1 -- 172.21.15.59:0/3258162633 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a380046c0 con 0x7f1a48101df0 2024-09-05T11:59:47.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.985+0000 7f1a515cd740 1 -- start start 2024-09-05T11:59:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b3f011a3e0 con 0x55b3f01b3b90 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b3f00c1d80 con 0x55b3f01bed70 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a515cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3f01163a0 con 0x55b3f01b44b0 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41010/0 (socket says 172.21.15.59:41010) 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2169928184 learned_addr learned my addr 172.21.15.59:0/2169928184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:47.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2169928184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 msgr2=0x55b3f01be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:47.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2169928184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 msgr2=0x55b3f01c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.986+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2169928184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f00e0ae0 con 0x55b3f01b3b90 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a40010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f287d1f89bb8a63e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a45600640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 0x7f1a2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a3d200640 1 -- 172.21.15.59:0/2169928184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40011700 con 0x55b3f01b3b90 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a3d200640 1 -- 172.21.15.59:0/2169928184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40011d80 con 0x55b3f01b3b90 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a3d200640 1 -- 172.21.15.59:0/2169928184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000e3c0 con 0x55b3f01b3b90 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a45600640 1 -- 172.21.15.59:0/3258162633 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a380cf830 con 0x7f1a48101df0 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.987+0000 7f1a47e00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 0x7f1a2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 -- 172.21.15.59:0/2169928184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 msgr2=0x55b3f01b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a40010860 comp rx=0 tx=0).stop 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 -- 172.21.15.59:0/2169928184 shutdown_connections 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2169928184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01b3f70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 -- 172.21.15.59:0/2169928184 >> 172.21.15.59:0/2169928184 conn(0x55b3f01b0120 msgr2=0x55b3f01b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a47e00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 0x7f1a2c0a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a380a4360 tx=0x7f1a380d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.988+0000 7f1a515cd740 1 -- 172.21.15.59:0/2169928184 shutdown_connections 2024-09-05T11:59:47.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.989+0000 7f1a515cd740 1 -- 172.21.15.59:0/2169928184 wait complete. 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.989+0000 7f1a515cd740 1 -- start start 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.989+0000 7f1a515cd740 1 Processor -- start 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b3f01c19e0 con 0x55b3f01b3b90 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b3f00d50f0 con 0x55b3f01bed70 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3f01c1b60 con 0x55b3f01b44b0 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a515cd740 10 client.0 _ll_register_callbacks cb 0x55b3f00c69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:39356/0 (socket says 172.21.15.59:39356) 2024-09-05T11:59:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a3fe00640 1 -- 172.21.15.59:0/2951036614 learned_addr learned my addr 172.21.15.59:0/2951036614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2951036614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 msgr2=0x55b3f01bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.990+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 msgr2=0x55b3f01bc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01bc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2951036614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3efe8b3c0 con 0x55b3f01b3b90 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bbc70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a4001bc20 tx=0x7f1a4001bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfd0d4174f1bb2a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a35600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40011700 con 0x55b3f01b3b90 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a400118a0 con 0x55b3f01b3b90 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40011d80 con 0x55b3f01b3b90 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1a40034050 con 0x55b3f01b3b90 2024-09-05T11:59:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a515cd740 10 client.15732 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.991+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3f01cd6a0 con 0x55b3f01b3b90 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40011d80 con 0x55b3f01b3b90 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1a40029020 con 0x55b3f01b3b90 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 10 client.15732 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 20 client.15732 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f01ceb00 0x55b3f01d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 4 client.15732 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T11:59:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.992+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f0127950 con 0x55b3f01ceb00 2024-09-05T11:59:47.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.993+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f01ceb00 0x55b3f01d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.993+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f01ceb00 0x55b3f01d0ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7890330223caca23 server_cookie=6778f3004d50f061 in_seq=0 out_seq=0 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.993+0000 7f1a35600640 10 client.15732 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a35600640 1 -- 172.21.15.59:0/2951036614 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3f0127950 con 0x55b3f01ceb00 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a35600640 10 client.15732 handle_command_reply: tid=0 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 client.15732 shutdown 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 msgr2=0x55b3f01bbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bbc70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a4001bc20 tx=0x7f1a4001bb20 comp rx=0 tx=0).stop 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 shutdown_connections 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f01ceb00 0x55b3f01d0ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2951036614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bbc70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 >> 172.21.15.59:0/2951036614 conn(0x55b3f01b0120 msgr2=0x55b3f01b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.994+0000 7f1a35600640 0 client.15732 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.995+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 shutdown_connections 2024-09-05T11:59:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.995+0000 7f1a515cd740 1 -- 172.21.15.59:0/2951036614 wait complete. 2024-09-05T11:59:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:47.995+0000 7f1a515cd740 20 client.15732 trim_cache size 0 max 16384 2024-09-05T11:59:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 Processor -- start 2024-09-05T11:59:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 -- start start 2024-09-05T11:59:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481af080 con 0x55b3f01b3b90 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b1900 con 0x55b3f01bed70 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a515cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481af6f0 con 0x55b3f01b44b0 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41032/0 (socket says 172.21.15.59:41032) 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.013+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2618107616 learned_addr learned my addr 172.21.15.59:0/2618107616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2618107616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 msgr2=0x55b3f01d6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01d6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2618107616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 msgr2=0x55b3f01d69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01d69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 -- 172.21.15.59:0/2618107616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f01ba1c0 con 0x55b3f01b3b90 2024-09-05T11:59:48.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a400108a0 tx=0x7f1a4001bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4162dfec7f64329 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a37e00640 1 -- 172.21.15.59:0/2618107616 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40031040 con 0x55b3f01b3b90 2024-09-05T11:59:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a37e00640 1 -- 172.21.15.59:0/2618107616 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40020030 con 0x55b3f01b3b90 2024-09-05T11:59:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.014+0000 7f1a37e00640 1 -- 172.21.15.59:0/2618107616 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40011760 con 0x55b3f01b3b90 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 -- 172.21.15.59:0/2618107616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 msgr2=0x55b3f01bdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a400108a0 tx=0x7f1a4001bb90 comp rx=0 tx=0).stop 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 -- 172.21.15.59:0/2618107616 shutdown_connections 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01bed70 0x55b3f01d69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01b44b0 0x55b3f01d6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 --2- 172.21.15.59:0/2618107616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01bdbe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 -- 172.21.15.59:0/2618107616 >> 172.21.15.59:0/2618107616 conn(0x55b3f01b0120 msgr2=0x55b3f01bd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 -- 172.21.15.59:0/2618107616 shutdown_connections 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.015+0000 7f1a515cd740 1 -- 172.21.15.59:0/2618107616 wait complete. 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 -- start start 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 Processor -- start 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01bed70 0x55b3f01b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481afdc0 con 0x55b3f01b3b90 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b1230 con 0x55b3f01b44b0 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b0490 con 0x55b3f01bed70 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.016+0000 7f1a515cd740 10 client.0 _ll_register_callbacks cb 0x7f1a40009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45328/0 (socket says 172.21.15.59:45328) 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 -- 172.21.15.59:0/896127290 learned_addr learned my addr 172.21.15.59:0/896127290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01bed70 0x55b3f01b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 -- 172.21.15.59:0/896127290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01bed70 msgr2=0x55b3f01b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01bed70 0x55b3f01b8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 -- 172.21.15.59:0/896127290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 msgr2=0x55b3f01c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 -- 172.21.15.59:0/896127290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f01b97c0 con 0x55b3f01b44b0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a3fe00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a30005940 tx=0x7f1a300058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c035d89b1f1a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.017+0000 7f1a35600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30037040 con 0x55b3f01b44b0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40020030 con 0x55b3f01b44b0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a30030550 con 0x55b3f01b44b0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1a30057050 con 0x55b3f01b44b0 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a515cd740 10 client.25760 fetch_fsmap learned FSMap version 21 2024-09-05T11:59:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3f0239580 con 0x55b3f01b44b0 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.018+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40020030 con 0x55b3f01b44b0 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1a30032440 con 0x55b3f01b44b0 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 10 client.25760 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 20 client.25760 populate_metadata read hostname 'smithi059' 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f0239d00 0x55b3f023c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 4 client.25760 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3f0121e60 con 0x55b3f0239d00 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f0239d00 0x55b3f023c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a4cc00640 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f0239d00 0x55b3f023c120 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c6c14e7e6498b server_cookie=edc45bff35d9dabc in_seq=0 out_seq=0 2024-09-05T11:59:48.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.019+0000 7f1a35600640 10 client.25760 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a35600640 1 -- 172.21.15.59:0/896127290 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3f0121e60 con 0x55b3f0239d00 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a35600640 10 client.25760 handle_command_reply: tid=0 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 client.25760 shutdown 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 msgr2=0x55b3f01b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a30005940 tx=0x7f1a300058d0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 shutdown_connections 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3f01bed70 0x55b3f01b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3f01b44b0 0x55b3f01b8a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3f0239d00 0x55b3f023c120 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 --2- 172.21.15.59:0/896127290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b3f01b3b90 0x55b3f01c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 >> 172.21.15.59:0/896127290 conn(0x55b3f01b0120 msgr2=0x55b3f01af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a35600640 0 client.25760 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 shutdown_connections 2024-09-05T11:59:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 1 -- 172.21.15.59:0/896127290 wait complete. 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.020+0000 7f1a515cd740 20 client.25760 trim_cache size 0 max 16384 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "ff0b2efd-d44d-471b-89b0-c5024536f3a7": { 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "ff0b2efd-d44d-471b-89b0-c5024536f3a7", 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T11:59:48.026 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 msgr2=0x7f1a2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 0x7f1a2c0a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a380a4360 tx=0x7f1a380d5530 comp rx=0 tx=0).stop 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 msgr2=0x7f1a4819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800e9c0 tx=0x7f1a3800ee80 comp rx=0 tx=0).stop 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 shutdown_connections 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1a2c0a2b80 0x7f1a2c0a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a48106f20 0x7f1a4819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a481027a0 0x7f1a4819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 --2- 172.21.15.59:0/3258162633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a48101df0 0x7f1a4819a620 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T11:59:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 >> 172.21.15.59:0/3258162633 conn(0x7f1a480fd8b0 msgr2=0x7f1a480feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T11:59:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 shutdown_connections 2024-09-05T11:59:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T11:59:48.021+0000 7f1a4ec00640 1 -- 172.21.15.59:0/3258162633 wait complete. 2024-09-05T11:59:48.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:47 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T11:59:48.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:47 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:48.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:47 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T11:59:48.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:47 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:47 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T11:59:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:47 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:47 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T11:59:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:47 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T11:59:48.895 INFO:tasks.cephfs.filesystem:scrub status for tag:ff0b2efd-d44d-471b-89b0-c5024536f3a7 - {'path': '/', 'tag': 'ff0b2efd-d44d-471b-89b0-c5024536f3a7', 'options': 'recursive,force'} 2024-09-05T11:59:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:48 smithi189 ceph-mon[34772]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T11:59:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:48 smithi089 ceph-mon[34499]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T11:59:49.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:48 smithi059 ceph-mon[30025]: pgmap v458: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T11:59:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:50 smithi189 ceph-mon[34772]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T11:59:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:50 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T11:59:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:50 smithi089 ceph-mon[34499]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T11:59:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:50 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T11:59:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:50 smithi059 ceph-mon[30025]: pgmap v459: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T11:59:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:50 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T11:59:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:52 smithi189 ceph-mon[34772]: pgmap v460: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:52 smithi089 ceph-mon[34499]: pgmap v460: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:52 smithi059 ceph-mon[30025]: pgmap v460: 129 pgs: 129 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T11:59:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:54 smithi189 ceph-mon[34772]: pgmap v461: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:55.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:54 smithi059 ceph-mon[30025]: pgmap v461: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:54 smithi089 ceph-mon[34499]: pgmap v461: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T11:59:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:56 smithi189 ceph-mon[34772]: pgmap v462: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:56 smithi089 ceph-mon[34499]: pgmap v462: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:56 smithi059 ceph-mon[30025]: pgmap v462: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T11:59:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 11:59:58 smithi189 ceph-mon[34772]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T11:59:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 11:59:58 smithi089 ceph-mon[34499]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T11:59:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 11:59:58 smithi059 ceph-mon[30025]: pgmap v463: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:00:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:00 smithi189 ceph-mon[34772]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:00 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T12:00:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:00 smithi089 ceph-mon[34499]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:00 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T12:00:01.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:00 smithi059 ceph-mon[30025]: pgmap v464: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:00 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T12:00:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:02 smithi189 ceph-mon[34772]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:02 smithi089 ceph-mon[34499]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:03.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:02 smithi059 ceph-mon[30025]: pgmap v465: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:04.112 INFO:tasks.workunit.client.0.smithi059.stdout:1/600: open fd = 3 2024-09-05T12:00:04.112 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.112 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.112 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:2/600: open fd = 3 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:3/600: open fd = 3 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:4/600: open fd = 3 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.113 INFO:tasks.workunit.client.0.smithi059.stdout:5/600: open fd = 3 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:6/600: open fd = 3 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:7/600: open fd = 3 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:8/600: open fd = 3 2024-09-05T12:00:04.114 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:9/600: open fd = 3 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:10/600: open fd = 3 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:11/600: open fd = 3 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.115 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:12/600: open fd = 3 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:13/600: open fd = 3 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:14/600: open fd = 3 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.116 INFO:tasks.workunit.client.0.smithi059.stdout:15/600: open fd = 3 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:16/600: open fd = 3 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:17/600: open fd = 3 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.117 INFO:tasks.workunit.client.0.smithi059.stdout:18/600: open fd = 3 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:19/600: open fd = 3 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:20/600: open fd = 3 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:21/600: open fd = 3 2024-09-05T12:00:04.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:22/600: open fd = 3 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:23/600: open fd = 3 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:24/600: open fd = 3 2024-09-05T12:00:04.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:25/600: open fd = 3 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:26/600: open fd = 3 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:27/600: open fd = 3 2024-09-05T12:00:04.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:28/600: open fd = 3 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:29/600: open fd = 3 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:30/600: open fd = 3 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:31/600: open fd = 3 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:32/600: open fd = 3 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:33/600: open fd = 3 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:34/600: open fd = 3 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:35/600: open fd = 3 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:36/600: open fd = 3 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.123 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:37/600: open fd = 3 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:38/600: open fd = 3 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:39/600: open fd = 3 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.124 INFO:tasks.workunit.client.0.smithi059.stdout:40/600: open fd = 3 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:41/600: open fd = 3 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:42/600: open fd = 3 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.125 INFO:tasks.workunit.client.0.smithi059.stdout:43/600: open fd = 3 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:44/600: open fd = 3 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:45/600: open fd = 3 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:46/600: open fd = 3 2024-09-05T12:00:04.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:47/600: open fd = 3 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:48/600: open fd = 3 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:49/600: open fd = 3 2024-09-05T12:00:04.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:50/600: open fd = 3 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:51/600: open fd = 3 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:52/600: open fd = 3 2024-09-05T12:00:04.128 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:53/600: open fd = 3 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:54/600: open fd = 3 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:55/600: open fd = 3 2024-09-05T12:00:04.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:56/600: open fd = 3 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:57/600: open fd = 3 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:58/600: open fd = 3 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:59/600: open fd = 3 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:60/600: open fd = 3 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:61/600: open fd = 3 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.131 INFO:tasks.workunit.client.0.smithi059.stdout:62/600: open fd = 3 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:63/600: open fd = 3 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:00:04.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:00:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:04 smithi189 ceph-mon[34772]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:04 smithi089 ceph-mon[34499]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:04 smithi059 ceph-mon[30025]: pgmap v466: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:06 smithi189 ceph-mon[34772]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:06 smithi089 ceph-mon[34499]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:06 smithi059 ceph-mon[30025]: pgmap v467: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:08 smithi189 ceph-mon[34772]: pgmap v468: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:00:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:08 smithi089 ceph-mon[34499]: pgmap v468: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:00:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:08 smithi059 ceph-mon[30025]: pgmap v468: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:00:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:10 smithi189 ceph-mon[34772]: pgmap v469: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:10 smithi089 ceph-mon[34499]: pgmap v469: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:10 smithi059 ceph-mon[30025]: pgmap v469: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:12 smithi189 ceph-mon[34772]: pgmap v470: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:12 smithi089 ceph-mon[34499]: pgmap v470: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:12 smithi059 ceph-mon[30025]: pgmap v470: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:15.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:14 smithi189 ceph-mon[34772]: pgmap v471: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:00:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:14 smithi089 ceph-mon[34499]: pgmap v471: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:00:15.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:14 smithi059 ceph-mon[30025]: pgmap v471: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:00:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:16 smithi189 ceph-mon[34772]: pgmap v472: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:16 smithi089 ceph-mon[34499]: pgmap v472: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:16 smithi059 ceph-mon[30025]: pgmap v472: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:18.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:00:19.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:18 smithi189 ceph-mon[34772]: pgmap v473: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:18 smithi089 ceph-mon[34499]: pgmap v473: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:19.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:18 smithi059 ceph-mon[30025]: pgmap v473: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:19.295 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 -- 172.21.15.59:0/286568687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f481079c0 msgr2=0x7f6f48107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 --2- 172.21.15.59:0/286568687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f481079c0 0x7f6f48107da0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009930 tx=0x7f6f3802a440 comp rx=0 tx=0).stop 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 -- 172.21.15.59:0/286568687 shutdown_connections 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 --2- 172.21.15.59:0/286568687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f48102360 0x7f6f4810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 --2- 172.21.15.59:0/286568687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481019c0 0x7f6f48101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 --2- 172.21.15.59:0/286568687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f481079c0 0x7f6f48107da0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 -- 172.21.15.59:0/286568687 >> 172.21.15.59:0/286568687 conn(0x7f6f480fd740 msgr2=0x7f6f480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 -- 172.21.15.59:0/286568687 shutdown_connections 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.899+0000 7f6f4d400640 1 -- 172.21.15.59:0/286568687 wait complete. 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.900+0000 7f6f4d400640 1 Processor -- start 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.900+0000 7f6f4d400640 1 -- start start 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f481019c0 0x7f6f4819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481079c0 0x7f6f4819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48111e10 con 0x7f6f48102360 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48111c90 con 0x7f6f481019c0 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48111f90 con 0x7f6f481079c0 2024-09-05T12:00:19.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60368/0 (socket says 172.21.15.59:60368) 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 -- 172.21.15.59:0/2681720550 learned_addr learned my addr 172.21.15.59:0/2681720550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f47e00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f481019c0 0x7f6f4819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481079c0 0x7f6f4819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 -- 172.21.15.59:0/2681720550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481079c0 msgr2=0x7f6f4819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481079c0 0x7f6f4819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 -- 172.21.15.59:0/2681720550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f481019c0 msgr2=0x7f6f4819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f481019c0 0x7f6f4819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.901+0000 7f6f3fe00640 1 -- 172.21.15.59:0/2681720550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4819f4a0 con 0x7f6f48102360 2024-09-05T12:00:19.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000e9f0 tx=0x7f6f3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f45600640 1 -- 172.21.15.59:0/2681720550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000cd80 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f45600640 1 -- 172.21.15.59:0/2681720550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f30005500 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f45600640 1 -- 172.21.15.59:0/2681720550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30010690 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4819f790 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.902+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f481a7070 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.905+0000 7f6f51486740 1 Processor -- start 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.905+0000 7f6f45600640 1 -- 172.21.15.59:0/2681720550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f300046c0 con 0x7f6f48102360 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 -- start start 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795dd50 0x55b38795e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b3879689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b3878c43e0 con 0x55b38795e670 2024-09-05T12:00:19.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b38786bd80 con 0x55b387968f30 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f51486740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3878c03a0 con 0x55b38795dd50 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b3879689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38660/0 (socket says 172.21.15.59:38660) 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.906+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3597765211 learned_addr learned my addr 172.21.15.59:0/3597765211 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795dd50 0x55b38795e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3597765211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795dd50 msgr2=0x55b38795e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795dd50 0x55b38795e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3597765211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 msgr2=0x55b3879689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b3879689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3597765211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38788aae0 con 0x55b387968f30 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f45600640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 0x7f6f2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.907+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f300d5a90 tx=0x7f6f300d5b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b7d5faa3a362dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f45600640 1 -- 172.21.15.59:0/2681720550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f300cf830 con 0x7f6f48102360 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f3d200640 1 -- 172.21.15.59:0/3597765211 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30097ec0 con 0x55b387968f30 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 0x7f6f2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f3d200640 1 -- 172.21.15.59:0/3597765211 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3009c320 con 0x55b387968f30 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f3d200640 1 -- 172.21.15.59:0/3597765211 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3009b580 con 0x55b387968f30 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f51486740 1 -- 172.21.15.59:0/3597765211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 msgr2=0x55b38796b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 0x7f6f2c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005fd0 tx=0x7f6f40009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.908+0000 7f6f51486740 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f300d5a90 tx=0x7f6f300d5b80 comp rx=0 tx=0).stop 2024-09-05T12:00:19.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 -- 172.21.15.59:0/3597765211 shutdown_connections 2024-09-05T12:00:19.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796b320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b3879689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 --2- 172.21.15.59:0/3597765211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795dd50 0x55b38795e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 -- 172.21.15.59:0/3597765211 >> 172.21.15.59:0/3597765211 conn(0x55b38795a2e0 msgr2=0x55b38795a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 -- 172.21.15.59:0/3597765211 shutdown_connections 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 -- 172.21.15.59:0/3597765211 wait complete. 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 -- start start 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.909+0000 7f6f51486740 1 Processor -- start 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795e670 0x55b3879663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b387966930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b38796bba0 con 0x55b38795dd50 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b38787f0f0 con 0x55b387968f30 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b38796bd20 con 0x55b38795e670 2024-09-05T12:00:19.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f51486740 10 client.0 _ll_register_callbacks cb 0x55b3878709e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60384/0 (socket says 172.21.15.59:60384) 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f47e00640 1 -- 172.21.15.59:0/2470678754 learned_addr learned my addr 172.21.15.59:0/2470678754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b387966930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.910+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795e670 0x55b3879663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 -- 172.21.15.59:0/2470678754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795e670 msgr2=0x55b3879663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795e670 0x55b3879663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 -- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 msgr2=0x55b387966930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b387966930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 -- 172.21.15.59:0/2470678754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3876353c0 con 0x55b38795dd50 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f47e00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009930 tx=0x7f6f380043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ac7a79328b5fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f35600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:00:19.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035040 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f38005450 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.911+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f3009c320 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f38005650 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 10 client.15750 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3879778d0 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3009c320 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6f38003a30 con 0x55b38795dd50 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 10 client.15750 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 20 client.15750 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b387978d90 0x55b38797b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 4 client.15750 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:00:19.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.912+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3878d1950 con 0x55b387978d90 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.913+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b387978d90 0x55b38797b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.913+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b387978d90 0x55b38797b180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b2585546c35f54 server_cookie=a424fa9edef242e0 in_seq=0 out_seq=0 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.913+0000 7f6f35600640 10 client.15750 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.914+0000 7f6f35600640 1 -- 172.21.15.59:0/2470678754 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3878d1950 con 0x55b387978d90 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.914+0000 7f6f35600640 10 client.15750 handle_command_reply: tid=0 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.914+0000 7f6f51486740 1 client.15750 shutdown 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.914+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 msgr2=0x55b387965eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.914+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009930 tx=0x7f6f380043b0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 shutdown_connections 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b387966930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b38795e670 0x55b3879663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b387978d90 0x55b38797b180 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 --2- 172.21.15.59:0/2470678754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795dd50 0x55b387965eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 >> 172.21.15.59:0/2470678754 conn(0x55b38795a2e0 msgr2=0x55b387960010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f35600640 0 client.15750 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:19.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 shutdown_connections 2024-09-05T12:00:19.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 1 -- 172.21.15.59:0/2470678754 wait complete. 2024-09-05T12:00:19.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.915+0000 7f6f51486740 20 client.15750 trim_cache size 0 max 16384 2024-09-05T12:00:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.933+0000 7f6f51486740 1 Processor -- start 2024-09-05T12:00:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.933+0000 7f6f51486740 1 -- start start 2024-09-05T12:00:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.933+0000 7f6f51486740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b38795e670 0x55b387964780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.933+0000 7f6f51486740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f51486740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b387963110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f51486740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481aeed0 con 0x55b387968f30 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f51486740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481b1750 con 0x55b38795e670 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f51486740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481af540 con 0x55b387980f70 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60390/0 (socket says 172.21.15.59:60390) 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3580619719 learned_addr learned my addr 172.21.15.59:0/3580619719 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b387963110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f47e00640 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b38795e670 0x55b387964780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3580619719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 msgr2=0x55b387963110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b387963110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3580619719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b38795e670 msgr2=0x55b387964780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b38795e670 0x55b387964780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.934+0000 7f6f3fe00640 1 -- 172.21.15.59:0/3580619719 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b387964e60 con 0x55b387968f30 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c6a0 tx=0x7f6f30009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8891b7748b01fcce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f37e00640 1 -- 172.21.15.59:0/3580619719 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3009b760 con 0x55b387968f30 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f37e00640 1 -- 172.21.15.59:0/3580619719 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3009c320 con 0x55b387968f30 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f37e00640 1 -- 172.21.15.59:0/3580619719 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3009b9f0 con 0x55b387968f30 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f51486740 1 -- 172.21.15.59:0/3580619719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 msgr2=0x55b387980a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.935+0000 7f6f51486740 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c6a0 tx=0x7f6f30009310 comp rx=0 tx=0).stop 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 -- 172.21.15.59:0/3580619719 shutdown_connections 2024-09-05T12:00:19.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b387963110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b387968f30 0x55b387980a30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 --2- 172.21.15.59:0/3580619719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b38795e670 0x55b387964780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 -- 172.21.15.59:0/3580619719 >> 172.21.15.59:0/3580619719 conn(0x55b38795a2e0 msgr2=0x55b387967f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 -- 172.21.15.59:0/3580619719 shutdown_connections 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 -- 172.21.15.59:0/3580619719 wait complete. 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 -- start start 2024-09-05T12:00:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.936+0000 7f6f51486740 1 Processor -- start 2024-09-05T12:00:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b3879e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481afc10 con 0x55b38795e670 2024-09-05T12:00:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481b1080 con 0x55b387968f30 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481b02e0 con 0x55b387980f70 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f51486740 10 client.0 _ll_register_callbacks cb 0x7f6f38004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60394/0 (socket says 172.21.15.59:60394) 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f47e00640 1 -- 172.21.15.59:0/260330410 learned_addr learned my addr 172.21.15.59:0/260330410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.937+0000 7f6f4ca00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b3879e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 -- 172.21.15.59:0/260330410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 msgr2=0x55b3879e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b3879e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 -- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 msgr2=0x55b38796d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 -- 172.21.15.59:0/260330410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38796ddd0 con 0x55b38795e670 2024-09-05T12:00:19.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f47e00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f38005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a812e6c241b08d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f35600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38058060 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f38031c50 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.938+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f3009c320 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f38035070 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 10 client.15756 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3879e4020 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3009c320 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6f380048a0 con 0x55b38795e670 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 10 client.15756 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 20 client.15756 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3879e4740 0x55b3879e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 4 client.15756 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.939+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3878cbe60 con 0x55b3879e4740 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3879e4740 0x55b3879e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f3fe00640 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3879e4740 0x55b3879e6b50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12cab62429cd6755 server_cookie=d1ee68392e42b34d in_seq=0 out_seq=0 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f35600640 10 client.15756 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f35600640 1 -- 172.21.15.59:0/260330410 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b3878cbe60 con 0x55b3879e4740 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f35600640 10 client.15756 handle_command_reply: tid=0 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 client.15756 shutdown 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 msgr2=0x55b38796d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f38005850 comp rx=0 tx=0).stop 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 shutdown_connections 2024-09-05T12:00:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b387980f70 0x55b3879e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b387968f30 0x55b38796d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b3879e4740 0x55b3879e6b50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 --2- 172.21.15.59:0/260330410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b38795e670 0x55b38796d000 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 >> 172.21.15.59:0/260330410 conn(0x55b38795a2e0 msgr2=0x55b387965de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.940+0000 7f6f35600640 0 client.15756 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 shutdown_connections 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f51486740 1 -- 172.21.15.59:0/260330410 wait complete. 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f51486740 20 client.15756 trim_cache size 0 max 16384 2024-09-05T12:00:19.946 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 msgr2=0x7f6f2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 0x7f6f2c0a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005fd0 tx=0x7f6f40009500 comp rx=0 tx=0).stop 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 msgr2=0x7f6f4819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000e9f0 tx=0x7f6f3000edf0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.941+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 shutdown_connections 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f2c0a2c00 0x7f6f2c0a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f481079c0 0x7f6f4819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f48102360 0x7f6f4819aa30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 --2- 172.21.15.59:0/2681720550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f481019c0 0x7f6f4819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 >> 172.21.15.59:0/2681720550 conn(0x7f6f480fd740 msgr2=0x7f6f4810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 shutdown_connections 2024-09-05T12:00:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:19.942+0000 7f6f4d400640 1 -- 172.21.15.59:0/2681720550 wait complete. 2024-09-05T12:00:20.910 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:00:20.911 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:00:20.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:20 smithi059 ceph-mon[30025]: pgmap v474: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:20 smithi189 ceph-mon[34772]: pgmap v474: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:20 smithi089 ceph-mon[34499]: pgmap v474: 129 pgs: 129 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:21.292 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 -- 172.21.15.59:0/2619914852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 msgr2=0x7f16a01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 --2- 172.21.15.59:0/2619914852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a01043d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f169402a410 comp rx=0 tx=0).stop 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 -- 172.21.15.59:0/2619914852 shutdown_connections 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 --2- 172.21.15.59:0/2619914852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a0109120 0x7f16a0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 --2- 172.21.15.59:0/2619914852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a01049a0 0x7f16a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 --2- 172.21.15.59:0/2619914852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a01043d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:21.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 -- 172.21.15.59:0/2619914852 >> 172.21.15.59:0/2619914852 conn(0x7f16a00ffa90 msgr2=0x7f16a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:21.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 -- 172.21.15.59:0/2619914852 shutdown_connections 2024-09-05T12:00:21.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.874+0000 7f16a7400640 1 -- 172.21.15.59:0/2619914852 wait complete. 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.875+0000 7f16a7400640 1 Processor -- start 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.875+0000 7f16a7400640 1 -- start start 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.875+0000 7f16a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.875+0000 7f16a7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 0x7f16a019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.875+0000 7f16a7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a7400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0114190 con 0x7f16a0103ff0 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a7400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0114010 con 0x7f16a01049a0 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a7400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0114310 con 0x7f16a0109120 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44722/0 (socket says 172.21.15.59:44722) 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a5400640 1 -- 172.21.15.59:0/1673593372 learned_addr learned my addr 172.21.15.59:0/1673593372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 0x7f16a019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 -- 172.21.15.59:0/1673593372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 msgr2=0x7f16a019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 -- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 msgr2=0x7f16a019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f169fe00640 1 -- 172.21.15.59:0/1673593372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a019f5c0 con 0x7f16a01049a0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.876+0000 7f16a5400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f169fe00640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 0x7f16a019ab50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d720 tx=0x7f168c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c017070 con 0x7f16a01049a0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c00de30 con 0x7f16a01049a0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a019f8b0 con 0x7f16a01049a0 2024-09-05T12:00:21.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0047a0 con 0x7f16a01049a0 2024-09-05T12:00:21.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.877+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a7190 con 0x7f16a01049a0 2024-09-05T12:00:21.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.878+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a01043d0 con 0x7f16a01049a0 2024-09-05T12:00:21.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.880+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f168c0099d0 con 0x7f16a01049a0 2024-09-05T12:00:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.882+0000 7f169d600640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.882+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f168c0cf090 con 0x7f16a01049a0 2024-09-05T12:00:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.882+0000 7f16a4a00640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.883+0000 7f16a4a00640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 0x7f16880a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1694000c00 tx=0x7f1694005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:21.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:21.886+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f168c020020 con 0x7f16a01049a0 2024-09-05T12:00:22.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.109+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16a010bd70 con 0x7f16a01049a0 2024-09-05T12:00:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.110+0000 7f169d600640 1 -- 172.21.15.59:0/1673593372 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f168c09bb80 con 0x7f16a01049a0 2024-09-05T12:00:22.112 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:00:22.112 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:00:22.112 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:00:22.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 msgr2=0x7f16880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:22.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 0x7f16880a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1694000c00 tx=0x7f1694005b50 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 msgr2=0x7f16a019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 0x7f16a019ab50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d720 tx=0x7f168c00dbe0 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 shutdown_connections 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16a0109120 0x7f16a019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a01049a0 0x7f16a019ab50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 --2- 172.21.15.59:0/1673593372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a0103ff0 0x7f16a019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 >> 172.21.15.59:0/1673593372 conn(0x7f16a00ffa90 msgr2=0x7f16a0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 shutdown_connections 2024-09-05T12:00:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:22.113+0000 7f16a7400640 1 -- 172.21.15.59:0/1673593372 wait complete. 2024-09-05T12:00:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:22 smithi189 ceph-mon[34772]: pgmap v475: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-05T12:00:23.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:22 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1673593372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:22 smithi089 ceph-mon[34499]: pgmap v475: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-05T12:00:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:22 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1673593372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:22 smithi059 ceph-mon[30025]: pgmap v475: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-09-05T12:00:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:22 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1673593372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:23.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:00:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:24.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.290+0000 7f523c400640 1 -- 172.21.15.59:0/470615020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234077040 msgr2=0x7f52341124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.290+0000 7f523c400640 1 --2- 172.21.15.59:0/470615020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234077040 0x7f52341124f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5220009900 tx=0x7f522002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 -- 172.21.15.59:0/470615020 shutdown_connections 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 --2- 172.21.15.59:0/470615020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234112a30 0x7f5234114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 --2- 172.21.15.59:0/470615020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234077040 0x7f52341124f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 --2- 172.21.15.59:0/470615020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234076720 0x7f5234076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 -- 172.21.15.59:0/470615020 >> 172.21.15.59:0/470615020 conn(0x7f52340ffa90 msgr2=0x7f5234101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 -- 172.21.15.59:0/470615020 shutdown_connections 2024-09-05T12:00:24.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.291+0000 7f523c400640 1 -- 172.21.15.59:0/470615020 wait complete. 2024-09-05T12:00:24.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 Processor -- start 2024-09-05T12:00:24.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 -- start start 2024-09-05T12:00:24.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234077040 0x7f52341a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 0x7f523419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52341185a0 con 0x7f5234076720 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5234118420 con 0x7f5234112a30 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f523c400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5234118720 con 0x7f5234077040 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.292+0000 7f5239a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 0x7f523419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f5239a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37960/0 (socket says 172.21.15.59:37960) 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f5239a00640 1 -- 172.21.15.59:0/656510155 learned_addr learned my addr 172.21.15.59:0/656510155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f5239000640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234077040 0x7f52341a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 -- 172.21.15.59:0/656510155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234077040 msgr2=0x7f52341a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234077040 0x7f52341a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 -- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 msgr2=0x7f52341a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 -- 172.21.15.59:0/656510155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523419efc0 con 0x7f5234112a30 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f5239a00640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.293+0000 7f523a400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 0x7f523419e890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f523407aa10 tx=0x7f522c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.294+0000 7f5232000640 1 -- 172.21.15.59:0/656510155 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c004d20 con 0x7f5234112a30 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.294+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523419f2b0 con 0x7f5234112a30 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.294+0000 7f5232000640 1 -- 172.21.15.59:0/656510155 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c005020 con 0x7f5234112a30 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.294+0000 7f5232000640 1 -- 172.21.15.59:0/656510155 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c01f060 con 0x7f5234112a30 2024-09-05T12:00:24.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.294+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341ab5a0 con 0x7f5234112a30 2024-09-05T12:00:24.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.297+0000 7f5232000640 1 -- 172.21.15.59:0/656510155 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f522c005560 con 0x7f5234112a30 2024-09-05T12:00:24.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.297+0000 7f523eedb740 1 Processor -- start 2024-09-05T12:00:24.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.297+0000 7f523eedb740 1 -- start start 2024-09-05T12:00:24.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fd7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a51f303e0 con 0x564a51fd4e50 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564a51ed7d80 con 0x564a51fc9c70 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f523eedb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a51f2c3a0 con 0x564a51fca590 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44760/0 (socket says 172.21.15.59:44760) 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 -- 172.21.15.59:0/2168364345 learned_addr learned my addr 172.21.15.59:0/2168364345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:24.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239000640 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fd7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 -- 172.21.15.59:0/2168364345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 msgr2=0x564a51fca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 -- 172.21.15.59:0/2168364345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 msgr2=0x564a51fd7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.298+0000 7f5239a00640 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fd7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5239a00640 1 -- 172.21.15.59:0/2168364345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a51ef6ae0 con 0x564a51fca590 2024-09-05T12:00:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5239000640 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fd7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5239a00640 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5228010580 tx=0x7f5228010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8581c5cdffe942a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5232000640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 0x7f521c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5226a00640 1 -- 172.21.15.59:0/2168364345 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228005220 con 0x564a51fca590 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5226a00640 1 -- 172.21.15.59:0/2168364345 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228004c80 con 0x564a51fca590 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5232000640 1 -- 172.21.15.59:0/656510155 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f522c0cf410 con 0x7f5234112a30 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5226a00640 1 -- 172.21.15.59:0/2168364345 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522800eb70 con 0x564a51fca590 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.299+0000 7f5239000640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 0x7f521c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 -- 172.21.15.59:0/2168364345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 msgr2=0x564a51fd4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5228010580 tx=0x7f5228010a40 comp rx=0 tx=0).stop 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f5239000640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 0x7f521c0a50c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5220009230 tx=0x7f52200058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 -- 172.21.15.59:0/2168364345 shutdown_connections 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fd7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd4910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 --2- 172.21.15.59:0/2168364345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 -- 172.21.15.59:0/2168364345 >> 172.21.15.59:0/2168364345 conn(0x564a51fc6200 msgr2=0x564a51fc65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 -- 172.21.15.59:0/2168364345 shutdown_connections 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.300+0000 7f523eedb740 1 -- 172.21.15.59:0/2168364345 wait complete. 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 -- start start 2024-09-05T12:00:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 Processor -- start 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fc9c70 0x564a51fd1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 0x564a51fd2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a51fd7a20 con 0x564a51fca590 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564a51eeb0f0 con 0x564a51fd4e50 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a51fd7ba0 con 0x564a51fc9c70 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.301+0000 7f523eedb740 10 client.0 _ll_register_callbacks cb 0x564a51ed2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42408/0 (socket says 172.21.15.59:42408) 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 -- 172.21.15.59:0/538395477 learned_addr learned my addr 172.21.15.59:0/538395477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f5239a00640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fc9c70 0x564a51fd1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f5239000640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 0x564a51fd2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 -- 172.21.15.59:0/538395477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fc9c70 msgr2=0x564a51fd1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fc9c70 0x564a51fd1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 -- 172.21.15.59:0/538395477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 msgr2=0x564a51fd2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 0x564a51fd2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 -- 172.21.15.59:0/538395477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a51ca13c0 con 0x564a51fd4e50 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f5239000640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 0x564a51fd2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f523a400640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f522c004af0 tx=0x7f522c09b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881bdcfe9a267439 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.302+0000 7f521b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:00:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0a4470 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5228004c80 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f522c0051d0 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f522c0860d0 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f523eedb740 10 client.25796 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564a51fe3a90 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.303+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228004c80 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f522c00f040 con 0x564a51fd4e50 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 10 client.25796 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 20 client.25796 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a51fe4ef0 0x564a51fe7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 4 client.25796 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:00:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a51f3d950 con 0x564a51fe4ef0 2024-09-05T12:00:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.304+0000 7f5239a00640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a51fe4ef0 0x564a51fe7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.305+0000 7f5239a00640 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a51fe4ef0 0x564a51fe7340 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba3eb2d1ebe17e9 server_cookie=4121112ebf5acac in_seq=0 out_seq=0 2024-09-05T12:00:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.305+0000 7f521b400640 10 client.25796 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f521b400640 1 -- 172.21.15.59:0/538395477 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a51f3d950 con 0x564a51fe4ef0 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f521b400640 10 client.25796 handle_command_reply: tid=0 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 client.25796 shutdown 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 msgr2=0x564a51fd27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f522c004af0 tx=0x7f522c09b780 comp rx=0 tx=0).stop 2024-09-05T12:00:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 shutdown_connections 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fd4e50 0x564a51fd27a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a51fe4ef0 0x564a51fe7340 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fca590 0x564a51fd2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 --2- 172.21.15.59:0/538395477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fc9c70 0x564a51fd1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 >> 172.21.15.59:0/538395477 conn(0x564a51fc6200 msgr2=0x564a51fcbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f521b400640 0 client.25796 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 shutdown_connections 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.306+0000 7f523eedb740 1 -- 172.21.15.59:0/538395477 wait complete. 2024-09-05T12:00:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.307+0000 7f523eedb740 20 client.25796 trim_cache size 0 max 16384 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 Processor -- start 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 -- start start 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b3460 con 0x564a51fd4e50 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b5ce0 con 0x564a51fc9c70 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.320+0000 7f523eedb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b3ad0 con 0x564a51fca590 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42422/0 (socket says 172.21.15.59:42422) 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 -- 172.21.15.59:0/824053599 learned_addr learned my addr 172.21.15.59:0/824053599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f523a400640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239000640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 -- 172.21.15.59:0/824053599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 msgr2=0x564a51fd3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 -- 172.21.15.59:0/824053599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 msgr2=0x564a51fecc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fecc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 -- 172.21.15.59:0/824053599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a51fc1770 con 0x564a51fc9c70 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239000640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.321+0000 7f5239a00640 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5228010580 tx=0x7f5228005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615b9085ff6e04d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f5226000640 1 -- 172.21.15.59:0/824053599 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228005480 con 0x564a51fc9c70 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f5226000640 1 -- 172.21.15.59:0/824053599 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522802d070 con 0x564a51fc9c70 2024-09-05T12:00:24.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f5226000640 1 -- 172.21.15.59:0/824053599 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228004a30 con 0x564a51fc9c70 2024-09-05T12:00:24.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 -- 172.21.15.59:0/824053599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 msgr2=0x564a51fcc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5228010580 tx=0x7f5228005190 comp rx=0 tx=0).stop 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 -- 172.21.15.59:0/824053599 shutdown_connections 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fd3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 --2- 172.21.15.59:0/824053599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fcc6d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 -- 172.21.15.59:0/824053599 >> 172.21.15.59:0/824053599 conn(0x564a51fc6200 msgr2=0x564a51fd3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 -- 172.21.15.59:0/824053599 shutdown_connections 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.322+0000 7f523eedb740 1 -- 172.21.15.59:0/824053599 wait complete. 2024-09-05T12:00:24.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 -- start start 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 Processor -- start 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fdd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fcee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b41a0 con 0x564a51fd4e50 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b5610 con 0x564a51fc9c70 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52341b4870 con 0x564a51fca590 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.323+0000 7f523eedb740 10 client.0 _ll_register_callbacks cb 0x7f522c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fdd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f523a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fcee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44768/0 (socket says 172.21.15.59:44768) 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239000640 1 -- 172.21.15.59:0/2959068862 learned_addr learned my addr 172.21.15.59:0/2959068862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 -- 172.21.15.59:0/2959068862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 msgr2=0x564a51fce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 -- 172.21.15.59:0/2959068862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 msgr2=0x564a51fcee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fcee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239a00640 1 -- 172.21.15.59:0/2959068862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a51fcf510 con 0x564a51fc9c70 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f5239000640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.324+0000 7f523a400640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f5239a00640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fdd450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f522800b4e0 tx=0x7f522800e3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2165f3b6d3c687 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:24.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f521b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035070 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522802d070 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5228005480 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5228041050 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f523eedb740 10 client.25802 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.325+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564a5204fa70 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522802d070 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5228029430 con 0x564a51fc9c70 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 10 client.25802 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 20 client.25802 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a52050030 0x564a52052450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:24.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 4 client.25802 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a51f37e60 con 0x564a52050030 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.326+0000 7f5239000640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a52050030 0x564a52052450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f5239000640 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a52050030 0x564a52052450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8382229f9b57f30b server_cookie=ae71a137925ac0ab in_seq=0 out_seq=0 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f521b400640 10 client.25802 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f521b400640 1 -- 172.21.15.59:0/2959068862 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a51f37e60 con 0x564a52050030 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f521b400640 10 client.25802 handle_command_reply: tid=0 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 client.25802 shutdown 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 msgr2=0x564a51fdd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fdd450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f522800b4e0 tx=0x7f522800e3d0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 shutdown_connections 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564a52050030 0x564a52052450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a51fd4e50 0x564a51fcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a51fca590 0x564a51fce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 --2- 172.21.15.59:0/2959068862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a51fc9c70 0x564a51fdd450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f521b400640 0 client.25802 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 >> 172.21.15.59:0/2959068862 conn(0x564a51fc6200 msgr2=0x564a51fd3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 shutdown_connections 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 1 -- 172.21.15.59:0/2959068862 wait complete. 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.327+0000 7f523eedb740 20 client.25802 trim_cache size 0 max 16384 2024-09-05T12:00:24.335 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 msgr2=0x7f521c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 0x7f521c0a50c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5220009230 tx=0x7f52200058d0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 msgr2=0x7f523419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 0x7f523419e890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f523407aa10 tx=0x7f522c00edd0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 shutdown_connections 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f521c0a2c00 0x7f521c0a50c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5234112a30 0x7f523419e890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5234077040 0x7f52341a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 --2- 172.21.15.59:0/656510155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5234076720 0x7f52341a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:24.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 >> 172.21.15.59:0/656510155 conn(0x7f52340ffa90 msgr2=0x7f5234101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:24.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 shutdown_connections 2024-09-05T12:00:24.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:24.328+0000 7f523c400640 1 -- 172.21.15.59:0/656510155 wait complete. 2024-09-05T12:00:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:24 smithi189 ceph-mon[34772]: pgmap v476: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:24 smithi089 ceph-mon[34499]: pgmap v476: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:25.185 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:24 smithi059 ceph-mon[30025]: pgmap v476: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:26.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:00:26.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:00:26.392 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.1 2024-09-05T12:00:26.392 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:00:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:26 smithi189 ceph-mon[34772]: pgmap v477: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:27.121 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:26 smithi059 ceph-mon[30025]: pgmap v477: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:26 smithi089 ceph-mon[34499]: pgmap v477: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 -- 172.21.15.59:0/629589811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5781047e0 msgr2=0x7fe578108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 --2- 172.21.15.59:0/629589811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5781047e0 0x7fe578108830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe564009900 tx=0x7fe56402a410 comp rx=0 tx=0).stop 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 -- 172.21.15.59:0/629589811 shutdown_connections 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 --2- 172.21.15.59:0/629589811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 0x7fe578110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 --2- 172.21.15.59:0/629589811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5781047e0 0x7fe578108830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 --2- 172.21.15.59:0/629589811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe578103e30 0x7fe578104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 -- 172.21.15.59:0/629589811 >> 172.21.15.59:0/629589811 conn(0x7fe5780ff910 msgr2=0x7fe578101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 -- 172.21.15.59:0/629589811 shutdown_connections 2024-09-05T12:00:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.236+0000 7fe57d000640 1 -- 172.21.15.59:0/629589811 wait complete. 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.237+0000 7fe57d000640 1 Processor -- start 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.237+0000 7fe57d000640 1 -- start start 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5781047e0 0x7fe57819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 0x7fe57819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe578114010 con 0x7fe578108f60 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe578113e90 con 0x7fe5781047e0 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe57d000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe578114190 con 0x7fe578103e30 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe577400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe577400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44780/0 (socket says 172.21.15.59:44780) 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe577400640 1 -- 172.21.15.59:0/982974446 learned_addr learned my addr 172.21.15.59:0/982974446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe577e00640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 0x7fe57819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.238+0000 7fe576a00640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5781047e0 0x7fe57819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 -- 172.21.15.59:0/982974446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5781047e0 msgr2=0x7fe57819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5781047e0 0x7fe57819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 -- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 msgr2=0x7fe57819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 0x7fe57819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 -- 172.21.15.59:0/982974446 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57819f400 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe577400640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe56800e9f0 tx=0x7fe56800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe56fe00640 1 -- 172.21.15.59:0/982974446 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800cd80 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe56fe00640 1 -- 172.21.15.59:0/982974446 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568010910 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57819f6f0 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe56fe00640 1 -- 172.21.15.59:0/982974446 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568018990 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.239+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5781a6fd0 con 0x7fe578103e30 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.242+0000 7fe58124b740 1 Processor -- start 2024-09-05T12:00:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.242+0000 7fe56fe00640 1 -- 172.21.15.59:0/982974446 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe568010ac0 con 0x7fe578103e30 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.242+0000 7fe58124b740 1 -- start start 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 0x55ffa3129090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa31295d0 0x55ffa3133950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 0x55ffa3136280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ffa308f290 con 0x55ffa31295d0 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffa3036b40 con 0x55ffa3128cb0 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe58124b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ffa308b250 con 0x55ffa3133e90 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe577e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 0x55ffa3129090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 0x55ffa3136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe577400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa31295d0 0x55ffa3133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe577e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 0x55ffa3129090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42462/0 (socket says 172.21.15.59:42462) 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe577e00640 1 -- 172.21.15.59:0/3126644207 learned_addr learned my addr 172.21.15.59:0/3126644207 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 -- 172.21.15.59:0/3126644207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 msgr2=0x55ffa3129090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 0x55ffa3129090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 -- 172.21.15.59:0/3126644207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa31295d0 msgr2=0x55ffa3133950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa31295d0 0x55ffa3133950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.243+0000 7fe576a00640 1 -- 172.21.15.59:0/3126644207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa2e006a0 con 0x55ffa3133e90 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe576a00640 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 0x55ffa3136280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe56402aa00 tx=0x7fe564005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91fc377c438ba5d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe56fe00640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 0x7fe55c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe56cc00640 1 -- 172.21.15.59:0/3126644207 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564035040 con 0x55ffa3133e90 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe56fe00640 1 -- 172.21.15.59:0/982974446 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5680cf720 con 0x7fe578103e30 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe56cc00640 1 -- 172.21.15.59:0/3126644207 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564004400 con 0x55ffa3133e90 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe56cc00640 1 -- 172.21.15.59:0/3126644207 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640035f0 con 0x55ffa3133e90 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.244+0000 7fe577e00640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 0x7fe55c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- 172.21.15.59:0/3126644207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 msgr2=0x55ffa3136280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 0x55ffa3136280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe56402aa00 tx=0x7fe564005de0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe577e00640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 0x7fe55c0a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe570011630 tx=0x7fe570009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- 172.21.15.59:0/3126644207 shutdown_connections 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3133e90 0x55ffa3136280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa31295d0 0x55ffa3133950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 --2- 172.21.15.59:0/3126644207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3128cb0 0x55ffa3129090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- 172.21.15.59:0/3126644207 >> 172.21.15.59:0/3126644207 conn(0x55ffa3125240 msgr2=0x55ffa3125620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- 172.21.15.59:0/3126644207 shutdown_connections 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- 172.21.15.59:0/3126644207 wait complete. 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.245+0000 7fe58124b740 1 -- start start 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 Processor -- start 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa31295d0 0x55ffa3130ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3133e90 0x55ffa3131040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ffa3136aa0 con 0x55ffa3133e90 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffa3049fa0 con 0x55ffa31295d0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ffa3136c20 con 0x55ffa3128cb0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe58124b740 10 client.0 _ll_register_callbacks cb 0x55ffa2d9dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe577400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe577400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44812/0 (socket says 172.21.15.59:44812) 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.246+0000 7fe577400640 1 -- 172.21.15.59:0/4097953976 learned_addr learned my addr 172.21.15.59:0/4097953976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577e00640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3133e90 0x55ffa3131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe576a00640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa31295d0 0x55ffa3130ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 -- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa31295d0 msgr2=0x55ffa3130ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa31295d0 0x55ffa3130ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 -- 172.21.15.59:0/4097953976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3133e90 msgr2=0x55ffa3131040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3133e90 0x55ffa3131040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 -- 172.21.15.59:0/4097953976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa3131870 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe577400640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe568098020 tx=0x7fe56809b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2c36e664e28fed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe561600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56809fec0 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5680d1be0 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe564004400 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.247+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe5680a4780 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 10 client.25732 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ffa3142aa0 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564004400 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe56809b0a0 con 0x55ffa3128cb0 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 10 client.25732 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 20 client.25732 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa3143f60 0x55ffa3146350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 4 client.25732 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa309c800 con 0x55ffa3143f60 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.248+0000 7fe576a00640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa3143f60 0x55ffa3146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.249+0000 7fe576a00640 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa3143f60 0x55ffa3146350 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e38d5cf31d91843 server_cookie=4bfdb05fc8972ea6 in_seq=0 out_seq=0 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.249+0000 7fe561600640 10 client.25732 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe561600640 1 -- 172.21.15.59:0/4097953976 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffa309c800 con 0x55ffa3143f60 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe561600640 10 client.25732 handle_command_reply: tid=0 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 client.25732 shutdown 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 msgr2=0x55ffa3133810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe568098020 tx=0x7fe56809b7f0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 shutdown_connections 2024-09-05T12:00:27.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa3143f60 0x55ffa3146350 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3133e90 0x55ffa3131040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa31295d0 0x55ffa3130ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 --2- 172.21.15.59:0/4097953976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa3128cb0 0x55ffa3133810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe561600640 0 client.25732 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 >> 172.21.15.59:0/4097953976 conn(0x55ffa3125240 msgr2=0x55ffa312af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 shutdown_connections 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 1 -- 172.21.15.59:0/4097953976 wait complete. 2024-09-05T12:00:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.250+0000 7fe58124b740 20 client.25732 trim_cache size 0 max 16384 2024-09-05T12:00:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.269+0000 7fe58124b740 1 Processor -- start 2024-09-05T12:00:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.269+0000 7fe58124b740 1 -- start start 2024-09-05T12:00:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa312ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa314bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa314c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781aee60 con 0x55ffa3128cb0 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781b16e0 con 0x55ffa3133e90 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe58124b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781af4d0 con 0x55ffa31295d0 2024-09-05T12:00:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe576a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa314bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe576a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa314bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44818/0 (socket says 172.21.15.59:44818) 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe576a00640 1 -- 172.21.15.59:0/2285435680 learned_addr learned my addr 172.21.15.59:0/2285435680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe577400640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa312ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.270+0000 7fe577e00640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa314c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 -- 172.21.15.59:0/2285435680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 msgr2=0x55ffa314bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa314bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 -- 172.21.15.59:0/2285435680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 msgr2=0x55ffa314c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa314c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 -- 172.21.15.59:0/2285435680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa312f010 con 0x55ffa3128cb0 2024-09-05T12:00:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577e00640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa314c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe577400640 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa312ae60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe568098540 tx=0x7fe56809bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe807c8ad0e2f85f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe563e00640 1 -- 172.21.15.59:0/2285435680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680cfb80 con 0x55ffa3128cb0 2024-09-05T12:00:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe563e00640 1 -- 172.21.15.59:0/2285435680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568018b30 con 0x55ffa3128cb0 2024-09-05T12:00:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.271+0000 7fe563e00640 1 -- 172.21.15.59:0/2285435680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568097b70 con 0x55ffa3128cb0 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 -- 172.21.15.59:0/2285435680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 msgr2=0x55ffa312ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa312ae60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe568098540 tx=0x7fe56809bd40 comp rx=0 tx=0).stop 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 -- 172.21.15.59:0/2285435680 shutdown_connections 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa314c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa314bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 --2- 172.21.15.59:0/2285435680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa312ae60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.272+0000 7fe58124b740 1 -- 172.21.15.59:0/2285435680 >> 172.21.15.59:0/2285435680 conn(0x55ffa3125240 msgr2=0x55ffa312a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.273+0000 7fe58124b740 1 -- 172.21.15.59:0/2285435680 shutdown_connections 2024-09-05T12:00:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.273+0000 7fe58124b740 1 -- 172.21.15.59:0/2285435680 wait complete. 2024-09-05T12:00:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.273+0000 7fe58124b740 1 -- start start 2024-09-05T12:00:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.273+0000 7fe58124b740 1 Processor -- start 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa31384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa31a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781afba0 con 0x55ffa3128cb0 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781b1010 con 0x55ffa3133e90 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781b0270 con 0x55ffa31295d0 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe58124b740 10 client.0 _ll_register_callbacks cb 0x7fe56800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe576a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe577400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa31384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe576a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44824/0 (socket says 172.21.15.59:44824) 2024-09-05T12:00:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe576a00640 1 -- 172.21.15.59:0/1688720239 learned_addr learned my addr 172.21.15.59:0/1688720239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.274+0000 7fe577e00640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa31a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 -- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 msgr2=0x55ffa31a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa31a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 -- 172.21.15.59:0/1688720239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 msgr2=0x55ffa31384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa31384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 -- 172.21.15.59:0/1688720239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa31a81d0 con 0x55ffa31295d0 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe577e00640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa31a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe576a00640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe564009900 tx=0x7fe564004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11d5ce27904c21dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe561600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564035050 con 0x55ffa31295d0 2024-09-05T12:00:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe568018b30 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.275+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5640048c0 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe564060050 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 10 client.25738 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ffa31af590 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe568018b30 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe564031980 con 0x55ffa31295d0 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 10 client.25738 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 10 client.25738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 20 client.25738 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa31af810 0x55ffa31b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 4 client.25738 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:00:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ffa3096d10 con 0x55ffa31af810 2024-09-05T12:00:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.276+0000 7fe577400640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa31af810 0x55ffa31b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.277+0000 7fe577400640 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa31af810 0x55ffa31b1c00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace4e5491b8160a3 server_cookie=7ba9d43b26b392cb in_seq=0 out_seq=0 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.277+0000 7fe561600640 10 client.25738 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.278+0000 7fe561600640 1 -- 172.21.15.59:0/1688720239 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ffa3096d10 con 0x55ffa31af810 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.278+0000 7fe561600640 10 client.25738 handle_command_reply: tid=0 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.278+0000 7fe58124b740 1 client.25738 shutdown 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 msgr2=0x55ffa3138a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe564009900 tx=0x7fe564004b80 comp rx=0 tx=0).stop 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 shutdown_connections 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffa3133e90 0x55ffa31a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa31295d0 0x55ffa3138a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ffa31af810 0x55ffa31b1c00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 --2- 172.21.15.59:0/1688720239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffa3128cb0 0x55ffa31384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 >> 172.21.15.59:0/1688720239 conn(0x55ffa3125240 msgr2=0x55ffa3132320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe561600640 0 client.25738 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 shutdown_connections 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 1 -- 172.21.15.59:0/1688720239 wait complete. 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.279+0000 7fe58124b740 20 client.25738 trim_cache size 0 max 16384 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "51c190f3-d99c-43c3-b601-13836bf76a84", 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:00:27.283 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:00:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 msgr2=0x7fe55c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 0x7fe55c0a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe570011630 tx=0x7fe570009500 comp rx=0 tx=0).stop 2024-09-05T12:00:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 msgr2=0x7fe57819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe56800e9f0 tx=0x7fe56800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 shutdown_connections 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe55c0a2c00 0x7fe55c0a50c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe578108f60 0x7fe57819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5781047e0 0x7fe57819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 --2- 172.21.15.59:0/982974446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe578103e30 0x7fe57819a450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.280+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 >> 172.21.15.59:0/982974446 conn(0x7fe5780ff910 msgr2=0x7fe578100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.281+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 shutdown_connections 2024-09-05T12:00:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:27.281+0000 7fe57d000640 1 -- 172.21.15.59:0/982974446 wait complete. 2024-09-05T12:00:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:28.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 -- 172.21.15.59:0/3599656562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d4103ff0 msgr2=0x7fe6d41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 --2- 172.21.15.59:0/3599656562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d4103ff0 0x7fe6d41043d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 -- 172.21.15.59:0/3599656562 shutdown_connections 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 --2- 172.21.15.59:0/3599656562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4109120 0x7fe6d4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 --2- 172.21.15.59:0/3599656562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d41049a0 0x7fe6d41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 --2- 172.21.15.59:0/3599656562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d4103ff0 0x7fe6d41043d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 -- 172.21.15.59:0/3599656562 >> 172.21.15.59:0/3599656562 conn(0x7fe6d40ffa90 msgr2=0x7fe6d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 -- 172.21.15.59:0/3599656562 shutdown_connections 2024-09-05T12:00:28.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.811+0000 7fe6db800640 1 -- 172.21.15.59:0/3599656562 wait complete. 2024-09-05T12:00:28.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.812+0000 7fe6db800640 1 Processor -- start 2024-09-05T12:00:28.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.812+0000 7fe6db800640 1 -- start start 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.812+0000 7fe6db800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.812+0000 7fe6db800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d41049a0 0x7fe6d419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6db800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 0x7fe6d419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6db800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d41141a0 con 0x7fe6d41049a0 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6db800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4114020 con 0x7fe6d4109120 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6db800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4114320 con 0x7fe6d4103ff0 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44840/0 (socket says 172.21.15.59:44840) 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d41049a0 0x7fe6d419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 -- 172.21.15.59:0/21081973 learned_addr learned my addr 172.21.15.59:0/21081973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:28.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d9800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 0x7fe6d419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:28.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 -- 172.21.15.59:0/21081973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 msgr2=0x7fe6d419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:28.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 0x7fe6d419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 -- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d41049a0 msgr2=0x7fe6d419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:28.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d41049a0 0x7fe6d419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d8e00640 1 -- 172.21.15.59:0/21081973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d419f5f0 con 0x7fe6d4103ff0 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.813+0000 7fe6d9800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 0x7fe6d419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6d8e00640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802a960 tx=0x7fe6c8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c803f070 con 0x7fe6d4103ff0 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8031c90 con 0x7fe6d4103ff0 2024-09-05T12:00:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d419f880 con 0x7fe6d4103ff0 2024-09-05T12:00:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41a71c0 con 0x7fe6d4103ff0 2024-09-05T12:00:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.814+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035030 con 0x7fe6d4103ff0 2024-09-05T12:00:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.816+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d41043d0 con 0x7fe6d4103ff0 2024-09-05T12:00:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.817+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe6c8042070 con 0x7fe6d4103ff0 2024-09-05T12:00:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.819+0000 7fe6d1600640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 0x7fe6bc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.819+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe6c80ef9c0 con 0x7fe6d4103ff0 2024-09-05T12:00:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.819+0000 7fe6d3e00640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 0x7fe6bc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:28.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.820+0000 7fe6d3e00640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 0x7fe6bc0a5100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40ffeb0 tx=0x7fe6c0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:28.823+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6c80bbe60 con 0x7fe6d4103ff0 2024-09-05T12:00:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:28 smithi059 ceph-mon[30025]: pgmap v478: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:28 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:00:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:28 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:28 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:00:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:28 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:29.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.059+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d410c300 con 0x7fe6d4103ff0 2024-09-05T12:00:29.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.060+0000 7fe6d1600640 1 -- 172.21.15.59:0/21081973 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fe6c80c0160 con 0x7fe6d4103ff0 2024-09-05T12:00:29.062 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:00:29.062 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:00:29.062 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:00:29.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.062+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 msgr2=0x7fe6bc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.062+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 0x7fe6bc0a5100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40ffeb0 tx=0x7fe6c0005f90 comp rx=0 tx=0).stop 2024-09-05T12:00:29.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.062+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 msgr2=0x7fe6d419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.062+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802a960 tx=0x7fe6c8003b90 comp rx=0 tx=0).stop 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 shutdown_connections 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe6bc0a2c40 0x7fe6bc0a5100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d4109120 0x7fe6d419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6d41049a0 0x7fe6d419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 --2- 172.21.15.59:0/21081973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6d4103ff0 0x7fe6d419a640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 >> 172.21.15.59:0/21081973 conn(0x7fe6d40ffa90 msgr2=0x7fe6d4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 shutdown_connections 2024-09-05T12:00:29.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.063+0000 7fe6db800640 1 -- 172.21.15.59:0/21081973 wait complete. 2024-09-05T12:00:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:28 smithi189 ceph-mon[34772]: pgmap v478: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:28 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:00:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:28 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:28 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:00:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:28 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:28 smithi089 ceph-mon[34499]: pgmap v478: 129 pgs: 129 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:00:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:28 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:00:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:28 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:28 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:00:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:28 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 -- 172.21.15.59:0/3014687058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24103e30 msgr2=0x7f8b24104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 --2- 172.21.15.59:0/3014687058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24103e30 0x7f8b24104210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009900 tx=0x7f8b1402a410 comp rx=0 tx=0).stop 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 -- 172.21.15.59:0/3014687058 shutdown_connections 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 --2- 172.21.15.59:0/3014687058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24108f60 0x7f8b24110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 --2- 172.21.15.59:0/3014687058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 --2- 172.21.15.59:0/3014687058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24103e30 0x7f8b24104210 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 -- 172.21.15.59:0/3014687058 >> 172.21.15.59:0/3014687058 conn(0x7f8b240ff910 msgr2=0x7f8b24101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 -- 172.21.15.59:0/3014687058 shutdown_connections 2024-09-05T12:00:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.533+0000 7f8b29a00640 1 -- 172.21.15.59:0/3014687058 wait complete. 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.534+0000 7f8b29a00640 1 Processor -- start 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.534+0000 7f8b29a00640 1 -- start start 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 0x7f8b24076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 0x7f8b24077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24114010 con 0x7f8b24108f60 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24113e90 con 0x7f8b241047e0 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24114190 con 0x7f8b24103e30 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 0x7f8b24077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 0x7f8b24076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 0x7f8b24076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44850/0 (socket says 172.21.15.59:44850) 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b23e00640 1 -- 172.21.15.59:0/1621061402 learned_addr learned my addr 172.21.15.59:0/1621061402 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b23400640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 -- 172.21.15.59:0/1621061402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 msgr2=0x7f8b24076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 0x7f8b24076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 -- 172.21.15.59:0/1621061402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 msgr2=0x7f8b24076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b29000640 1 -- 172.21.15.59:0/1621061402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b241aab40 con 0x7f8b24108f60 2024-09-05T12:00:29.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.535+0000 7f8b23400640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b29000640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 0x7f8b24077180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009fd0 tx=0x7f8b1c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b20c00640 1 -- 172.21.15.59:0/1621061402 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00ce20 con 0x7f8b24108f60 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b20c00640 1 -- 172.21.15.59:0/1621061402 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c005020 con 0x7f8b24108f60 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b20c00640 1 -- 172.21.15.59:0/1621061402 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c010720 con 0x7f8b24108f60 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b241aae30 con 0x7f8b24108f60 2024-09-05T12:00:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.536+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b241ab340 con 0x7f8b24108f60 2024-09-05T12:00:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.539+0000 7f8b20c00640 1 -- 172.21.15.59:0/1621061402 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b1c005540 con 0x7f8b24108f60 2024-09-05T12:00:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.539+0000 7f8b2da36740 1 Processor -- start 2024-09-05T12:00:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.539+0000 7f8b2da36740 1 -- start start 2024-09-05T12:00:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6baefd50 0x560d6baf0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 0x560d6bafd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d6ba563e0 con 0x560d6baf0670 2024-09-05T12:00:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560d6b9fdd80 con 0x560d6bafaf30 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b2da36740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d6ba523a0 con 0x560d6baefd50 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38102/0 (socket says 172.21.15.59:38102) 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b23e00640 1 -- 172.21.15.59:0/2808458416 learned_addr learned my addr 172.21.15.59:0/2808458416 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b23400640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 0x560d6bafd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.540+0000 7f8b23400640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6baefd50 0x560d6baf0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 -- 172.21.15.59:0/2808458416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6baefd50 msgr2=0x560d6baf0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6baefd50 0x560d6baf0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 -- 172.21.15.59:0/2808458416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 msgr2=0x560d6bafd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 0x560d6bafd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 -- 172.21.15.59:0/2808458416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6ba1cae0 con 0x560d6baf0670 2024-09-05T12:00:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23400640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 0x560d6bafd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b20c00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 0x7f8b080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23e00640 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005e80 tx=0x7f8b14005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fe8a37124ed28e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b20c00640 1 -- 172.21.15.59:0/1621061402 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8b1c0cfb30 con 0x7f8b24108f60 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.541+0000 7f8b23400640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 0x7f8b080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b19600640 1 -- 172.21.15.59:0/2808458416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14035040 con 0x560d6baf0670 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b19600640 1 -- 172.21.15.59:0/2808458416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b140044b0 con 0x560d6baf0670 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b19600640 1 -- 172.21.15.59:0/2808458416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003850 con 0x560d6baf0670 2024-09-05T12:00:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b2da36740 1 -- 172.21.15.59:0/2808458416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 msgr2=0x560d6bafa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b23400640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 0x7f8b080a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x560d6baed660 tx=0x7f8b100046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.542+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005e80 tx=0x7f8b14005c00 comp rx=0 tx=0).stop 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 -- 172.21.15.59:0/2808458416 shutdown_connections 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6bafaf30 0x560d6bafd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafa9f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2808458416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6baefd50 0x560d6baf0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 -- 172.21.15.59:0/2808458416 >> 172.21.15.59:0/2808458416 conn(0x560d6baec2e0 msgr2=0x560d6baec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 -- 172.21.15.59:0/2808458416 shutdown_connections 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 -- 172.21.15.59:0/2808458416 wait complete. 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 -- start start 2024-09-05T12:00:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.543+0000 7f8b2da36740 1 Processor -- start 2024-09-05T12:00:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6baf7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baf8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560d6bafdb40 con 0x560d6baf0670 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560d6ba110f0 con 0x560d6baefd50 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d6bafdcc0 con 0x560d6bafaf30 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b2da36740 10 client.0 _ll_register_callbacks cb 0x560d6ba029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38116/0 (socket says 172.21.15.59:38116) 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b23400640 1 -- 172.21.15.59:0/1630800650 learned_addr learned my addr 172.21.15.59:0/1630800650 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b29000640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baf8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.544+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6baf7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 -- 172.21.15.59:0/1630800650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 msgr2=0x560d6baf8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baf8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 -- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 msgr2=0x560d6baf7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6baf7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 -- 172.21.15.59:0/1630800650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6b7c73c0 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6baf7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b23400640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8b10014500 tx=0x7f8b10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6d172f9469135f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.545+0000 7f8b0e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10012650 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8b100127f0 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b140044b0 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8b10032050 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b2da36740 10 client.15816 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d6bb097a0 con 0x560d6baf0670 2024-09-05T12:00:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b140044b0 con 0x560d6baf0670 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.546+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8b10021400 con 0x560d6baf0670 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 10 client.15816 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 20 client.15816 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb0ac60 0x560d6bb0d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 4 client.15816 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d6ba63950 con 0x560d6bb0ac60 2024-09-05T12:00:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.547+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb0ac60 0x560d6bb0d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.548+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb0ac60 0x560d6bb0d0d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517a307c942313a9 server_cookie=35163881b75a25e in_seq=0 out_seq=0 2024-09-05T12:00:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.548+0000 7f8b0e000640 10 client.15816 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.548+0000 7f8b0e000640 1 -- 172.21.15.59:0/1630800650 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d6ba63950 con 0x560d6bb0ac60 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.548+0000 7f8b0e000640 10 client.15816 handle_command_reply: tid=0 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.548+0000 7f8b2da36740 1 client.15816 shutdown 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 msgr2=0x560d6baf8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8b10014500 tx=0x7f8b10004730 comp rx=0 tx=0).stop 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 shutdown_connections 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baf8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb0ac60 0x560d6bb0d0d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6baf8310 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1630800650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6baf7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 >> 172.21.15.59:0/1630800650 conn(0x560d6baec2e0 msgr2=0x560d6baf1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b0e000640 0 client.15816 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 shutdown_connections 2024-09-05T12:00:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 1 -- 172.21.15.59:0/1630800650 wait complete. 2024-09-05T12:00:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.549+0000 7f8b2da36740 20 client.15816 trim_cache size 0 max 16384 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 Processor -- start 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 -- start start 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 0x560d6bb129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6bb12ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b32d0 con 0x560d6baefd50 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b5b50 con 0x560d6baf0670 2024-09-05T12:00:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.568+0000 7f8b2da36740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b3940 con 0x560d6bafaf30 2024-09-05T12:00:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 0x560d6bb129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38128/0 (socket says 172.21.15.59:38128) 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23e00640 1 -- 172.21.15.59:0/2770001286 learned_addr learned my addr 172.21.15.59:0/2770001286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b29000640 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6bb12ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 -- 172.21.15.59:0/2770001286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 msgr2=0x560d6bb12ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6bb12ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 -- 172.21.15.59:0/2770001286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 msgr2=0x560d6bae7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23400640 1 -- 172.21.15.59:0/2770001286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6baf6d80 con 0x560d6baf0670 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.569+0000 7f8b23e00640 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.570+0000 7f8b23400640 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 0x560d6bb129a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b100219c0 tx=0x7f8b1000b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ae6a93f8f5dae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.570+0000 7f8b18c00640 1 -- 172.21.15.59:0/2770001286 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1003a020 con 0x560d6baf0670 2024-09-05T12:00:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.570+0000 7f8b18c00640 1 -- 172.21.15.59:0/2770001286 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b10021a40 con 0x560d6baf0670 2024-09-05T12:00:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.570+0000 7f8b18c00640 1 -- 172.21.15.59:0/2770001286 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000f480 con 0x560d6baf0670 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- 172.21.15.59:0/2770001286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 msgr2=0x560d6bb129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 0x560d6bb129a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b100219c0 tx=0x7f8b1000b4a0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- 172.21.15.59:0/2770001286 shutdown_connections 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6bb12ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baf0670 0x560d6bb129a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 --2- 172.21.15.59:0/2770001286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baefd50 0x560d6bae7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- 172.21.15.59:0/2770001286 >> 172.21.15.59:0/2770001286 conn(0x560d6baec2e0 msgr2=0x560d6baf9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- 172.21.15.59:0/2770001286 shutdown_connections 2024-09-05T12:00:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- 172.21.15.59:0/2770001286 wait complete. 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.571+0000 7f8b2da36740 1 -- start start 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 Processor -- start 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b4010 con 0x560d6baf0670 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b5480 con 0x560d6baefd50 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241b46e0 con 0x560d6bafaf30 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b2da36740 10 client.0 _ll_register_callbacks cb 0x7f8b100143c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.572+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42548/0 (socket says 172.21.15.59:42548) 2024-09-05T12:00:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 -- 172.21.15.59:0/1835253595 learned_addr learned my addr 172.21.15.59:0/1835253595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b29000640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 -- 172.21.15.59:0/1835253595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 msgr2=0x560d6baff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 -- 172.21.15.59:0/1835253595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 msgr2=0x560d6bafeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 -- 172.21.15.59:0/1835253595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6bb74970 con 0x560d6baefd50 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23400640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b29000640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b23e00640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8b14031550 tx=0x7f8b14004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df60fcbb3376c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b0e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.573+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14035040 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b10021a40 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8b140315b0 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8b140044b0 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b2da36740 10 client.25844 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d6bb75b40 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.574+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b10021a40 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8b140039e0 con 0x560d6baefd50 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 10 client.25844 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 20 client.25844 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb75e30 0x560d6bb78220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 4 client.25844 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d6ba5de60 con 0x560d6bb75e30 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.575+0000 7f8b23400640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb75e30 0x560d6bb78220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b23400640 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb75e30 0x560d6bb78220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d8cd3e69b2abbb server_cookie=17f2623174501106 in_seq=0 out_seq=0 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b0e000640 10 client.25844 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b0e000640 1 -- 172.21.15.59:0/1835253595 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d6ba5de60 con 0x560d6bb75e30 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b0e000640 10 client.25844 handle_command_reply: tid=0 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 client.25844 shutdown 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 msgr2=0x560d6bb037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8b14031550 tx=0x7f8b14004bd0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 shutdown_connections 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d6bafaf30 0x560d6baff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560d6bb75e30 0x560d6bb78220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560d6baf0670 0x560d6bafeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 --2- 172.21.15.59:0/1835253595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d6baefd50 0x560d6bb037c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 >> 172.21.15.59:0/1835253595 conn(0x560d6baec2e0 msgr2=0x560d6baf17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b0e000640 0 client.25844 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.576+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 shutdown_connections 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b2da36740 1 -- 172.21.15.59:0/1835253595 wait complete. 2024-09-05T12:00:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b2da36740 20 client.25844 trim_cache size 0 max 16384 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "51c190f3-d99c-43c3-b601-13836bf76a84": { 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "51c190f3-d99c-43c3-b601-13836bf76a84", 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:29.582 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 msgr2=0x7f8b080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 0x7f8b080a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x560d6baed660 tx=0x7f8b100046c0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 msgr2=0x7f8b24077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 0x7f8b24077180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009fd0 tx=0x7f8b1c00ef10 comp rx=0 tx=0).stop 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.577+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 shutdown_connections 2024-09-05T12:00:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8b080a2b80 0x7f8b080a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b24108f60 0x7f8b24077180 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b241047e0 0x7f8b24076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 --2- 172.21.15.59:0/1621061402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b24103e30 0x7f8b24076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 >> 172.21.15.59:0/1621061402 conn(0x7f8b240ff910 msgr2=0x7f8b24100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 shutdown_connections 2024-09-05T12:00:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:29.578+0000 7f8b29a00640 1 -- 172.21.15.59:0/1621061402 wait complete. 2024-09-05T12:00:30.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:29 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/21081973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:29 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/21081973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:29 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/21081973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:00:30.468 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 154.1 --await-for 45.0' 2024-09-05T12:00:30.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 154.1 --await-for 45.0 2024-09-05T12:00:30.985 INFO:tasks.cephfs.filesystem:scrub status for tag:51c190f3-d99c-43c3-b601-13836bf76a84 - {'path': '/', 'tag': '51c190f3-d99c-43c3-b601-13836bf76a84', 'options': 'recursive,force'} 2024-09-05T12:00:31.085 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:00:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:30 smithi189 ceph-mon[34772]: pgmap v479: 129 pgs: 129 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:00:31.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:30 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:00:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:30 smithi089 ceph-mon[34499]: pgmap v479: 129 pgs: 129 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:00:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:30 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:00:31.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:30 smithi059 ceph-mon[30025]: pgmap v479: 129 pgs: 129 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:00:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:30 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 -- 172.21.15.59:0/2214921956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0076720 msgr2=0x7f48e0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2214921956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0076720 0x7f48e0076b00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009900 tx=0x7f48d002a420 comp rx=0 tx=0).stop 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 -- 172.21.15.59:0/2214921956 shutdown_connections 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2214921956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0112a50 0x7f48e0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2214921956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0077040 0x7f48e0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2214921956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0076720 0x7f48e0076b00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 -- 172.21.15.59:0/2214921956 >> 172.21.15.59:0/2214921956 conn(0x7f48e00ffa50 msgr2=0x7f48e0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 -- 172.21.15.59:0/2214921956 shutdown_connections 2024-09-05T12:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.684+0000 7f48e4a00640 1 -- 172.21.15.59:0/2214921956 wait complete. 2024-09-05T12:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.685+0000 7f48e4a00640 1 Processor -- start 2024-09-05T12:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.685+0000 7f48e4a00640 1 -- start start 2024-09-05T12:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0077040 0x7f48e01a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0112a50 0x7f48e019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0118620 con 0x7f48e0077040 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01184a0 con 0x7f48e0112a50 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.686+0000 7f48e4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01187a0 con 0x7f48e0076720 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34188/0 (socket says 172.21.15.59:34188) 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 -- 172.21.15.59:0/2004498267 learned_addr learned my addr 172.21.15.59:0/2004498267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dd400640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0077040 0x7f48e01a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48de800640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0112a50 0x7f48e019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 -- 172.21.15.59:0/2004498267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0112a50 msgr2=0x7f48e019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0112a50 0x7f48e019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 -- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0077040 msgr2=0x7f48e01a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0077040 0x7f48e01a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 -- 172.21.15.59:0/2004498267 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e019f0a0 con 0x7f48e0076720 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.687+0000 7f48dde00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48d0006010 tx=0x7f48d00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.688+0000 7f48d6a00640 1 -- 172.21.15.59:0/2004498267 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d003f030 con 0x7f48e0076720 2024-09-05T12:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.688+0000 7f48d6a00640 1 -- 172.21.15.59:0/2004498267 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d0003ce0 con 0x7f48e0076720 2024-09-05T12:00:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.688+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e019f330 con 0x7f48e0076720 2024-09-05T12:00:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.688+0000 7f48d6a00640 1 -- 172.21.15.59:0/2004498267 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0035070 con 0x7f48e0076720 2024-09-05T12:00:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.688+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e01ab680 con 0x7f48e0076720 2024-09-05T12:00:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.691+0000 7f48e73be740 1 Processor -- start 2024-09-05T12:00:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 -- start start 2024-09-05T12:00:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48d6a00640 1 -- 172.21.15.59:0/2004498267 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f48d0042030 con 0x7f48e0076720 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3c0590 0x56122b3c2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56122b31b530 con 0x56122b3c0590 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56122b2c2860 con 0x56122b3b53b0 2024-09-05T12:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48e73be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56122b3174f0 con 0x56122b3b5cd0 2024-09-05T12:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48de800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48de800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:39794/0 (socket says 172.21.15.59:39794) 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48de800640 1 -- 172.21.15.59:0/1188702412 learned_addr learned my addr 172.21.15.59:0/1188702412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.692+0000 7f48dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3c0590 0x56122b3c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48dde00640 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 -- 172.21.15.59:0/1188702412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 msgr2=0x56122b3c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 -- 172.21.15.59:0/1188702412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3c0590 msgr2=0x56122b3c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3c0590 0x56122b3c2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 -- 172.21.15.59:0/1188702412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122b08cb90 con 0x56122b3b53b0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.693+0000 7f48de800640 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009fd0 tx=0x7f48d8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf3e5cd4d1c5b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48cb000640 1 -- 172.21.15.59:0/1188702412 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8013790 con 0x56122b3b53b0 2024-09-05T12:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48cb000640 1 -- 172.21.15.59:0/1188702412 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d8019070 con 0x56122b3b53b0 2024-09-05T12:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48cb000640 1 -- 172.21.15.59:0/1188702412 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8014de0 con 0x56122b3b53b0 2024-09-05T12:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48d6a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 0x7f48c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48d6a00640 1 -- 172.21.15.59:0/2004498267 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f48d00ef460 con 0x7f48e0076720 2024-09-05T12:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48dde00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 0x7f48c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48e73be740 1 -- 172.21.15.59:0/1188702412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 msgr2=0x56122b3b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.694+0000 7f48e73be740 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009fd0 tx=0x7f48d8009650 comp rx=0 tx=0).stop 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 -- 172.21.15.59:0/1188702412 shutdown_connections 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3c0590 0x56122b3c2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 --2- 172.21.15.59:0/1188702412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b53b0 0x56122b3b5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 -- 172.21.15.59:0/1188702412 >> 172.21.15.59:0/1188702412 conn(0x56122b3b1940 msgr2=0x56122b3b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 -- 172.21.15.59:0/1188702412 shutdown_connections 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 -- 172.21.15.59:0/1188702412 wait complete. 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48dde00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 0x7f48c40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c3ff0 tx=0x7f48d002a420 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.695+0000 7f48e73be740 1 -- start start 2024-09-05T12:00:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 Processor -- start 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 0x56122b3be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56122b3c2fb0 con 0x56122b3b53b0 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56122b2d5e60 con 0x56122b3c0590 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56122b3c3130 con 0x56122b3b5cd0 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.696+0000 7f48e73be740 10 client.0 _ll_register_callbacks cb 0x56122b2dacc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3bd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45992/0 (socket says 172.21.15.59:45992) 2024-09-05T12:00:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48de800640 1 -- 172.21.15.59:0/473408705 learned_addr learned my addr 172.21.15.59:0/473408705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48de800640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 0x56122b3be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 -- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 msgr2=0x56122b3be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 0x56122b3be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 -- 172.21.15.59:0/473408705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 msgr2=0x56122b3bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3bd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 -- 172.21.15.59:0/473408705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122b2e1990 con 0x56122b3b5cd0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48de800640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 0x56122b3be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48dd400640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3bdc70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48e0079220 tx=0x7f48cc00bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f379af63a5fb2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.697+0000 7f48bf400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc003d90 con 0x56122b3b5cd0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d8019070 con 0x56122b3b5cd0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f48cc005020 con 0x56122b3b5cd0 2024-09-05T12:00:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f48cc024050 con 0x56122b3b5cd0 2024-09-05T12:00:31.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48e73be740 10 client.25771 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:31.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56122b3cef00 con 0x56122b3b5cd0 2024-09-05T12:00:31.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.698+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d8019070 con 0x56122b3b5cd0 2024-09-05T12:00:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f48cc01b470 con 0x56122b3b5cd0 2024-09-05T12:00:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 10 client.25771 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 10 client.25771 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:00:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 20 client.25771 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b3d0360 0x56122b3d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 4 client.25771 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122b2be310 con 0x56122b3d0360 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.699+0000 7f48de800640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b3d0360 0x56122b3d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.700+0000 7f48de800640 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b3d0360 0x56122b3d27c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ad4f9753d6a1e7 server_cookie=ea44fadf045463e6 in_seq=0 out_seq=0 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.700+0000 7f48bf400640 10 client.25771 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.700+0000 7f48bf400640 1 -- 172.21.15.59:0/473408705 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56122b2be310 con 0x56122b3d0360 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.700+0000 7f48bf400640 10 client.25771 handle_command_reply: tid=0 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 client.25771 shutdown 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 msgr2=0x56122b3bdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3bdc70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48e0079220 tx=0x7f48cc00bf30 comp rx=0 tx=0).stop 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 shutdown_connections 2024-09-05T12:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3c0590 0x56122b3be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3b5cd0 0x56122b3bdc70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b3d0360 0x56122b3d27c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 --2- 172.21.15.59:0/473408705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 >> 172.21.15.59:0/473408705 conn(0x56122b3b1940 msgr2=0x56122b3b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48bf400640 0 client.25771 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 shutdown_connections 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 1 -- 172.21.15.59:0/473408705 wait complete. 2024-09-05T12:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.701+0000 7f48e73be740 20 client.25771 trim_cache size 0 max 16384 2024-09-05T12:00:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.719+0000 7f48e73be740 1 Processor -- start 2024-09-05T12:00:31.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 -- start start 2024-09-05T12:00:31.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b3560 con 0x56122b3b53b0 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b5de0 con 0x56122b3b5cd0 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48e73be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b3bd0 con 0x56122b3c0590 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46002/0 (socket says 172.21.15.59:46002) 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48de800640 1 -- 172.21.15.59:0/1574361417 learned_addr learned my addr 172.21.15.59:0/1574361417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.720+0000 7f48dd400640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48de800640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 -- 172.21.15.59:0/1574361417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 msgr2=0x56122b3d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 -- 172.21.15.59:0/1574361417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 msgr2=0x56122b3b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 -- 172.21.15.59:0/1574361417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122b3c2cf0 con 0x56122b3b5cd0 2024-09-05T12:00:31.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48de800640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:31.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48dd400640 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3d8090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48cc01ba30 tx=0x7f48cc0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0c60e50f2c4400 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48ca600640 1 -- 172.21.15.59:0/1574361417 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00b400 con 0x56122b3b5cd0 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48ca600640 1 -- 172.21.15.59:0/1574361417 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc003d90 con 0x56122b3b5cd0 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.721+0000 7f48ca600640 1 -- 172.21.15.59:0/1574361417 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc01bab0 con 0x56122b3b5cd0 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.722+0000 7f48e73be740 1 -- 172.21.15.59:0/1574361417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 msgr2=0x56122b3d8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.722+0000 7f48e73be740 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3d8090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48cc01ba30 tx=0x7f48cc0058d0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 -- 172.21.15.59:0/1574361417 shutdown_connections 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3d8090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 --2- 172.21.15.59:0/1574361417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 -- 172.21.15.59:0/1574361417 >> 172.21.15.59:0/1574361417 conn(0x56122b3b1940 msgr2=0x56122b3b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 -- 172.21.15.59:0/1574361417 shutdown_connections 2024-09-05T12:00:31.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 -- 172.21.15.59:0/1574361417 wait complete. 2024-09-05T12:00:31.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 -- start start 2024-09-05T12:00:31.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.723+0000 7f48e73be740 1 Processor -- start 2024-09-05T12:00:31.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b42a0 con 0x56122b3b53b0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b5710 con 0x56122b3b5cd0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01b4970 con 0x56122b3c0590 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48e73be740 10 client.0 _ll_register_callbacks cb 0x7f48cc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46004/0 (socket says 172.21.15.59:46004) 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48de800640 1 -- 172.21.15.59:0/3027166466 learned_addr learned my addr 172.21.15.59:0/3027166466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.724+0000 7f48dd400640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48de800640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 -- 172.21.15.59:0/3027166466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 msgr2=0x56122b3c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 -- 172.21.15.59:0/3027166466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 msgr2=0x56122b3c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 -- 172.21.15.59:0/3027166466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122b3c58e0 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48de800640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48dd400640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3c8910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x56122b3ad1b0 tx=0x7f48cc003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45807dbd577d5d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48bf400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc01bc70 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48cc003d90 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.725+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f48cc017040 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f48cc03d050 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48e73be740 10 client.25862 fetch_fsmap learned FSMap version 21 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56122b43b7f0 con 0x56122b3b5cd0 2024-09-05T12:00:31.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc003d90 con 0x56122b3b5cd0 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f48cc028530 con 0x56122b3b5cd0 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.726+0000 7f48e73be740 10 client.25862 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48e73be740 10 client.25862 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48e73be740 20 client.25862 populate_metadata read hostname 'smithi059' 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b43bec0 0x56122b43e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48e73be740 4 client.25862 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.1, "await_for": 45.0}] 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.1, "await_for": 45.0}) -- 0x56122b328c60 con 0x56122b43bec0 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48de800640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b43bec0 0x56122b43e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:00:31.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48de800640 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b43bec0 0x56122b43e2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d557b4f98d1afff server_cookie=676598d2e09396a6 in_seq=0 out_seq=0 2024-09-05T12:00:31.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.727+0000 7f48bf400640 10 client.25862 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48bf400640 1 -- 172.21.15.59:0/3027166466 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56122b328c60 con 0x56122b43bec0 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48bf400640 10 client.25862 handle_command_reply: tid=0 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 client.25862 shutdown 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 msgr2=0x56122b3c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3c8910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x56122b3ad1b0 tx=0x7f48cc003a30 comp rx=0 tx=0).stop 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 shutdown_connections 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56122b3c0590 0x56122b3c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56122b3b5cd0 0x56122b3c8910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56122b43bec0 0x56122b43e2e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 --2- 172.21.15.59:0/3027166466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56122b3b53b0 0x56122b3c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 >> 172.21.15.59:0/3027166466 conn(0x56122b3b1940 msgr2=0x56122b3bd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48bf400640 0 client.25862 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 shutdown_connections 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.736+0000 7f48e73be740 1 -- 172.21.15.59:0/3027166466 wait complete. 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e73be740 20 client.25862 trim_cache size 0 max 16384 2024-09-05T12:00:31.739 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 14, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "bcd22a4d": { 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 14, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:00:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 154.1, 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:31.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:00:31.742 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:00:31.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 msgr2=0x7f48c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 0x7f48c40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c3ff0 tx=0x7f48d002a420 comp rx=0 tx=0).stop 2024-09-05T12:00:31.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 msgr2=0x7f48e01a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:00:31.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48d0006010 tx=0x7f48d00042f0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 shutdown_connections 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f48c40a2c00 0x7f48c40a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e0112a50 0x7f48e019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48e0077040 0x7f48e01a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 --2- 172.21.15.59:0/2004498267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48e0076720 0x7f48e01a47b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.737+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 >> 172.21.15.59:0/2004498267 conn(0x7f48e00ffa50 msgr2=0x7f48e0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.738+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 shutdown_connections 2024-09-05T12:00:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:00:31.738+0000 7f48e4a00640 1 -- 172.21.15.59:0/2004498267 wait complete. 2024-09-05T12:00:32.716 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bcd22a4d', quiesce took 6.3 seconds. Will release after: 27.8 2024-09-05T12:00:32.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:32 smithi059 ceph-mon[30025]: pgmap v480: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:00:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:32 smithi189 ceph-mon[34772]: pgmap v480: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:00:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:32 smithi089 ceph-mon[34499]: pgmap v480: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:00:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:34 smithi089 ceph-mon[34499]: pgmap v481: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:34 smithi059 ceph-mon[30025]: pgmap v481: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:34 smithi189 ceph-mon[34772]: pgmap v481: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:36 smithi089 ceph-mon[34499]: pgmap v482: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:36 smithi059 ceph-mon[30025]: pgmap v482: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:36 smithi189 ceph-mon[34772]: pgmap v482: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:00:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:38 smithi089 ceph-mon[34499]: pgmap v483: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:38 smithi059 ceph-mon[30025]: pgmap v483: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:38 smithi189 ceph-mon[34772]: pgmap v483: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:39 smithi089 ceph-mon[34499]: pgmap v484: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-09-05T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:39 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:00:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:39 smithi059 ceph-mon[30025]: pgmap v484: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-09-05T12:00:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:00:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:00:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:00:40.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:39 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:00:40.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:39 smithi189 ceph-mon[34772]: pgmap v484: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-09-05T12:00:40.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:00:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:00:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:00:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:39 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:00:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:42 smithi189 ceph-mon[34772]: pgmap v485: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 6 op/s 2024-09-05T12:00:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:42 smithi089 ceph-mon[34499]: pgmap v485: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 6 op/s 2024-09-05T12:00:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:42 smithi059 ceph-mon[30025]: pgmap v485: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 6 op/s 2024-09-05T12:00:44.281 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:44 smithi089 ceph-mon[34499]: pgmap v486: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-05T12:00:44.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:44 smithi189 ceph-mon[34772]: pgmap v486: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-05T12:00:44.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:44 smithi059 ceph-mon[30025]: pgmap v486: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-09-05T12:00:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:46 smithi059 ceph-mon[30025]: pgmap v487: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-05T12:00:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:46 smithi189 ceph-mon[34772]: pgmap v487: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-05T12:00:46.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:46 smithi089 ceph-mon[34499]: pgmap v487: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-09-05T12:00:48.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:48 smithi189 ceph-mon[34772]: pgmap v488: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-05T12:00:48.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:48 smithi089 ceph-mon[34499]: pgmap v488: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-05T12:00:48.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:48 smithi059 ceph-mon[30025]: pgmap v488: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-09-05T12:00:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:50 smithi189 ceph-mon[34772]: pgmap v489: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:50 smithi089 ceph-mon[34499]: pgmap v489: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:50 smithi059 ceph-mon[30025]: pgmap v489: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:52.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:52 smithi189 ceph-mon[34772]: pgmap v490: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:52.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:52 smithi089 ceph-mon[34499]: pgmap v490: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:52.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:52 smithi059 ceph-mon[30025]: pgmap v490: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:54.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:54 smithi189 ceph-mon[34772]: pgmap v491: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:54.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:54 smithi089 ceph-mon[34499]: pgmap v491: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:54.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:54 smithi059 ceph-mon[30025]: pgmap v491: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:56.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:56 smithi189 ceph-mon[34772]: pgmap v492: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:56 smithi089 ceph-mon[34499]: pgmap v492: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:56.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:56 smithi059 ceph-mon[30025]: pgmap v492: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:58.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:00:58 smithi189 ceph-mon[34772]: pgmap v493: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:58.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:00:58 smithi089 ceph-mon[34499]: pgmap v493: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:00:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:00:58 smithi059 ceph-mon[30025]: pgmap v493: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:01:00.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:00 smithi189 ceph-mon[34772]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:01:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:00 smithi089 ceph-mon[34499]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:01:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:00 smithi059 ceph-mon[30025]: pgmap v494: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:01:00.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:01:00.909 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:00.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:01:01.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 -- 172.21.15.59:0/2201437219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff228102360 msgr2=0x7ff22810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:01.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 --2- 172.21.15.59:0/2201437219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff228102360 0x7ff22810e890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff220009cd0 tx=0x7ff22002a8f0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 -- 172.21.15.59:0/2201437219 shutdown_connections 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 --2- 172.21.15.59:0/2201437219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff228102360 0x7ff22810e890 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 --2- 172.21.15.59:0/2201437219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281019c0 0x7ff228101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 --2- 172.21.15.59:0/2201437219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2281079c0 0x7ff228107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 -- 172.21.15.59:0/2201437219 >> 172.21.15.59:0/2201437219 conn(0x7ff2280fd740 msgr2=0x7ff2280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 -- 172.21.15.59:0/2201437219 shutdown_connections 2024-09-05T12:01:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.515+0000 7ff227200640 1 -- 172.21.15.59:0/2201437219 wait complete. 2024-09-05T12:01:01.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.516+0000 7ff227200640 1 Processor -- start 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 -- start start 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff228102360 0x7ff228074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281079c0 0x7ff228074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111e10 con 0x7ff2281019c0 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111c90 con 0x7ff228102360 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff227200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111f90 con 0x7ff2281079c0 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff225e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff225e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53406/0 (socket says 172.21.15.59:53406) 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff225e00640 1 -- 172.21.15.59:0/3086844741 learned_addr learned my addr 172.21.15.59:0/3086844741 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:01.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff21fe00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff228102360 0x7ff228074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.517+0000 7ff226800640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281079c0 0x7ff228074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 -- 172.21.15.59:0/3086844741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281079c0 msgr2=0x7ff228074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281079c0 0x7ff228074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 -- 172.21.15.59:0/3086844741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff228102360 msgr2=0x7ff228074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff228102360 0x7ff228074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 -- 172.21.15.59:0/3086844741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2281a6710 con 0x7ff2281019c0 2024-09-05T12:01:01.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.518+0000 7ff225e00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d950 tx=0x7ff21800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.519+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218017070 con 0x7ff2281019c0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.519+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff218009c30 con 0x7ff2281019c0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.519+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2281a6a00 con 0x7ff2281019c0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.519+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21800f6b0 con 0x7ff2281019c0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.519+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2281a6f90 con 0x7ff2281019c0 2024-09-05T12:01:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.520+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff228101e20 con 0x7ff2281019c0 2024-09-05T12:01:01.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.522+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2180109d0 con 0x7ff2281019c0 2024-09-05T12:01:01.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.524+0000 7ff21ea00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 0x7ff20c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:01.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.524+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff2180ceef0 con 0x7ff2281019c0 2024-09-05T12:01:01.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.525+0000 7ff21fe00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 0x7ff20c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:01.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.525+0000 7ff21fe00640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 0x7ff20c0a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff228075f80 tx=0x7ff21000b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:01.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.527+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff218009de0 con 0x7ff2281019c0 2024-09-05T12:01:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:01.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.755+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff228075a20 con 0x7ff2281019c0 2024-09-05T12:01:01.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.756+0000 7ff21ea00640 1 -- 172.21.15.59:0/3086844741 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7ff21809b410 con 0x7ff2281019c0 2024-09-05T12:01:01.758 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:01:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:01:01.758 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 msgr2=0x7ff20c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 0x7ff20c0a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff228075f80 tx=0x7ff21000b390 comp rx=0 tx=0).stop 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 msgr2=0x7ff2280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d950 tx=0x7ff21800de10 comp rx=0 tx=0).stop 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 shutdown_connections 2024-09-05T12:01:01.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff20c0a2c00 0x7ff20c0a50c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2281079c0 0x7ff228074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff228102360 0x7ff228074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 --2- 172.21.15.59:0/3086844741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2281019c0 0x7ff2280744f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.758+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 >> 172.21.15.59:0/3086844741 conn(0x7ff2280fd740 msgr2=0x7ff2280feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.759+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 shutdown_connections 2024-09-05T12:01:01.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:01.759+0000 7ff227200640 1 -- 172.21.15.59:0/3086844741 wait complete. 2024-09-05T12:01:02.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:02 smithi059 ceph-mon[30025]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:01:02.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3086844741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:02.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:02 smithi189 ceph-mon[34772]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:01:02.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3086844741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:02.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 -- 172.21.15.59:0/3677027245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c1049a0 msgr2=0x7f898c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 --2- 172.21.15.59:0/3677027245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c1049a0 0x7f898c1089f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8978009900 tx=0x7f897802a410 comp rx=0 tx=0).stop 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 -- 172.21.15.59:0/3677027245 shutdown_connections 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 --2- 172.21.15.59:0/3677027245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c109120 0x7f898c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 --2- 172.21.15.59:0/3677027245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c1049a0 0x7f898c1089f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 --2- 172.21.15.59:0/3677027245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c103ff0 0x7f898c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 -- 172.21.15.59:0/3677027245 >> 172.21.15.59:0/3677027245 conn(0x7f898c0ffa90 msgr2=0x7f898c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 -- 172.21.15.59:0/3677027245 shutdown_connections 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.248+0000 7f8991200640 1 -- 172.21.15.59:0/3677027245 wait complete. 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.249+0000 7f8991200640 1 Processor -- start 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.249+0000 7f8991200640 1 -- start start 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c103ff0 0x7f898c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c1049a0 0x7f898c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f898c114190 con 0x7f898c109120 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f898c114010 con 0x7f898c103ff0 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8991200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f898c114310 con 0x7f898c1049a0 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53432/0 (socket says 172.21.15.59:53432) 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 -- 172.21.15.59:0/880683914 learned_addr learned my addr 172.21.15.59:0/880683914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8989e00640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c103ff0 0x7f898c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f8989400640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c1049a0 0x7f898c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 -- 172.21.15.59:0/880683914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c1049a0 msgr2=0x7f898c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c1049a0 0x7f898c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 -- 172.21.15.59:0/880683914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c103ff0 msgr2=0x7f898c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c103ff0 0x7f898c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.250+0000 7f898a800640 1 -- 172.21.15.59:0/880683914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c19f5c0 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f898a800640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f898400eb10 tx=0x7f898400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f8982a00640 1 -- 172.21.15.59:0/880683914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004b20 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f8982a00640 1 -- 172.21.15.59:0/880683914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984004cc0 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f8982a00640 1 -- 172.21.15.59:0/880683914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984010760 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c19f8b0 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.251+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1a7190 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.254+0000 7f8993bdf740 1 Processor -- start 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.254+0000 7f8982a00640 1 -- 172.21.15.59:0/880683914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8984005020 con 0x7f898c109120 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 -- start start 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 0x5606257eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 0x5606257fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257fad70 0x5606257fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5606257563e0 con 0x5606257efb90 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5606256fdd80 con 0x5606257f04b0 2024-09-05T12:01:02.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8993bdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606257523a0 con 0x5606257fad70 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 0x5606257eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 0x5606257eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53438/0 (socket says 172.21.15.59:53438) 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8989400640 1 -- 172.21.15.59:0/3867972233 learned_addr learned my addr 172.21.15.59:0/3867972233 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8989e00640 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 0x5606257fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.255+0000 7f8989400640 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257fad70 0x5606257fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 -- 172.21.15.59:0/3867972233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257fad70 msgr2=0x5606257fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257fad70 0x5606257fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 -- 172.21.15.59:0/3867972233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 msgr2=0x5606257eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 0x5606257eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 -- 172.21.15.59:0/3867972233 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56062571cae0 con 0x5606257f04b0 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8982a00640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989e00640 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 0x5606257fa830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f897c009c00 tx=0x7f897c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0df43acc054408 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8982a00640 1 -- 172.21.15.59:0/880683914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f89840cf9a0 con 0x7f898c109120 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8989400640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8977400640 1 -- 172.21.15.59:0/3867972233 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c004230 con 0x5606257f04b0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8977400640 1 -- 172.21.15.59:0/3867972233 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c0043d0 con 0x5606257f04b0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.256+0000 7f8977400640 1 -- 172.21.15.59:0/3867972233 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c004d20 con 0x5606257f04b0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 -- 172.21.15.59:0/3867972233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 msgr2=0x5606257fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 0x5606257fa830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f897c009c00 tx=0x7f897c011810 comp rx=0 tx=0).stop 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8989400640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 0x7f89700a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8978031550 tx=0x7f8978041040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 -- 172.21.15.59:0/3867972233 shutdown_connections 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257fad70 0x5606257fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257f04b0 0x5606257fa830 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3867972233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257efb90 0x5606257eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 -- 172.21.15.59:0/3867972233 >> 172.21.15.59:0/3867972233 conn(0x5606257ec120 msgr2=0x5606257ec500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 -- 172.21.15.59:0/3867972233 shutdown_connections 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.257+0000 7f8993bdf740 1 -- 172.21.15.59:0/3867972233 wait complete. 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 -- start start 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 Processor -- start 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257f04b0 0x5606257f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 0x5606257f8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5606257fd9e0 con 0x5606257fad70 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5606257110f0 con 0x5606257efb90 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.258+0000 7f8993bdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606257fdb60 con 0x5606257f04b0 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8993bdf740 10 client.0 _ll_register_callbacks cb 0x5606257029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42732/0 (socket says 172.21.15.59:42732) 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 -- 172.21.15.59:0/719376628 learned_addr learned my addr 172.21.15.59:0/719376628 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f898a800640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 0x5606257f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989400640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257f04b0 0x5606257f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 -- 172.21.15.59:0/719376628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257f04b0 msgr2=0x5606257f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257f04b0 0x5606257f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 -- 172.21.15.59:0/719376628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 msgr2=0x5606257f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 0x5606257f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f8989e00640 1 -- 172.21.15.59:0/719376628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606254c73c0 con 0x5606257efb90 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.259+0000 7f898a800640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 0x5606257f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f8989e00640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f897c019ae0 tx=0x7f897c019b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31973291b495c8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f896be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:02.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c028020 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f897c024620 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897c0043d0 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f897c033050 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f8993bdf740 10 client.25880 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.260+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5606258096e0 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c0043d0 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f897c018070 con 0x5606257efb90 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 10 client.25880 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 20 client.25880 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56062580aae0 0x56062580ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 4 client.25880 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560625763950 con 0x56062580aae0 2024-09-05T12:01:02.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.261+0000 7f8989400640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56062580aae0 0x56062580ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.262+0000 7f8989400640 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56062580aae0 0x56062580ced0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb697f4c68e2234 server_cookie=17855a3c89b964c4 in_seq=0 out_seq=0 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.262+0000 7f896be00640 10 client.25880 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.262+0000 7f896be00640 1 -- 172.21.15.59:0/719376628 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560625763950 con 0x56062580aae0 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.262+0000 7f896be00640 10 client.25880 handle_command_reply: tid=0 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 client.25880 shutdown 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 msgr2=0x5606257f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f897c019ae0 tx=0x7f897c019b10 comp rx=0 tx=0).stop 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 shutdown_connections 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56062580aae0 0x56062580ced0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257fad70 0x5606257f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257f04b0 0x5606257f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 --2- 172.21.15.59:0/719376628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257efb90 0x5606257f7cb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 >> 172.21.15.59:0/719376628 conn(0x5606257ec120 msgr2=0x5606257f1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f896be00640 0 client.25880 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 shutdown_connections 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 1 -- 172.21.15.59:0/719376628 wait complete. 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.263+0000 7f8993bdf740 20 client.25880 trim_cache size 0 max 16384 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 Processor -- start 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 -- start start 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x560625812b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1af010 con 0x5606257f04b0 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1b1890 con 0x5606257fad70 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.281+0000 7f8993bdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1af680 con 0x5606257efb90 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53450/0 (socket says 172.21.15.59:53450) 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56244/0 (socket says 172.21.15.59:56244) 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f898a800640 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x560625812b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 -- 172.21.15.59:0/3199645 learned_addr learned my addr 172.21.15.59:0/3199645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 -- 172.21.15.59:0/3199645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 msgr2=0x5606257f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 -- 172.21.15.59:0/3199645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 msgr2=0x560625812b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x560625812b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 -- 172.21.15.59:0/3199645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606257ebbc0 con 0x5606257f04b0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f898a800640 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x560625812b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.282+0000 7f8989400640 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f897802aa00 tx=0x7f897802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e9428f46832dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8976a00640 1 -- 172.21.15.59:0/3199645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978058070 con 0x5606257f04b0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8976a00640 1 -- 172.21.15.59:0/3199645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8978004ba0 con 0x5606257f04b0 2024-09-05T12:01:02.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8976a00640 1 -- 172.21.15.59:0/3199645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978035040 con 0x5606257f04b0 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 -- 172.21.15.59:0/3199645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 msgr2=0x560625812640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f897802aa00 tx=0x7f897802a410 comp rx=0 tx=0).stop 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 -- 172.21.15.59:0/3199645 shutdown_connections 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x560625812b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x560625812640 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 --2- 172.21.15.59:0/3199645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.283+0000 7f8993bdf740 1 -- 172.21.15.59:0/3199645 >> 172.21.15.59:0/3199645 conn(0x5606257ec120 msgr2=0x5606257f8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 -- 172.21.15.59:0/3199645 shutdown_connections 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 -- 172.21.15.59:0/3199645 wait complete. 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 -- start start 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 Processor -- start 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.284+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8993bdf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x5606257ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8993bdf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1afd50 con 0x5606257f04b0 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8993bdf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1b11c0 con 0x5606257fad70 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8993bdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1b0420 con 0x5606257efb90 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8993bdf740 10 client.0 _ll_register_callbacks cb 0x7f897c019770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53462/0 (socket says 172.21.15.59:53462) 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 -- 172.21.15.59:0/76298826 learned_addr learned my addr 172.21.15.59:0/76298826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989e00640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f898a800640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x5606257ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 -- 172.21.15.59:0/76298826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 msgr2=0x5606257fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 -- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 msgr2=0x5606257ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x5606257ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 -- 172.21.15.59:0/76298826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560625874900 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.285+0000 7f8989400640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8978005ec0 tx=0x7f8978032690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5d92fc33478a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978061060 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f89780046c0 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8978004ba0 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8978058070 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f8993bdf740 10 client.15864 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560625875c10 con 0x5606257f04b0 2024-09-05T12:01:02.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8978004ba0 con 0x5606257f04b0 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.286+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8978003850 con 0x5606257f04b0 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 10 client.15864 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 20 client.15864 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560625876280 0x560625878670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 4 client.15864 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56062575de60 con 0x560625876280 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8989e00640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560625876280 0x560625878670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f8989e00640 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560625876280 0x560625878670 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31abba083bd6d635 server_cookie=bc0b3dc71bdfc03f in_seq=0 out_seq=0 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.287+0000 7f896be00640 10 client.15864 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f896be00640 1 -- 172.21.15.59:0/76298826 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56062575de60 con 0x560625876280 2024-09-05T12:01:02.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f896be00640 10 client.15864 handle_command_reply: tid=0 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 client.15864 shutdown 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 msgr2=0x5606257ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8978005ec0 tx=0x7f8978032690 comp rx=0 tx=0).stop 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 shutdown_connections 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5606257fad70 0x5606257ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560625876280 0x560625878670 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5606257f04b0 0x5606257ff360 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 --2- 172.21.15.59:0/76298826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606257efb90 0x5606257fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f896be00640 0 client.15864 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 >> 172.21.15.59:0/76298826 conn(0x5606257ec120 msgr2=0x5606257ea9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 shutdown_connections 2024-09-05T12:01:02.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 1 -- 172.21.15.59:0/76298826 wait complete. 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.288+0000 7f8993bdf740 20 client.15864 trim_cache size 0 max 16384 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 msgr2=0x7f89700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 0x7f89700a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8978031550 tx=0x7f8978041040 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 msgr2=0x7f898c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f898400eb10 tx=0x7f898400c2d0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 shutdown_connections 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89700a2b80 0x7f89700a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898c109120 0x7f898c19eee0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f898c1049a0 0x7f898c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 --2- 172.21.15.59:0/880683914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f898c103ff0 0x7f898c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 >> 172.21.15.59:0/880683914 conn(0x7f898c0ffa90 msgr2=0x7f898c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 shutdown_connections 2024-09-05T12:01:02.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:02.289+0000 7f8991200640 1 -- 172.21.15.59:0/880683914 wait complete. 2024-09-05T12:01:02.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:02 smithi089 ceph-mon[34499]: pgmap v495: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:01:02.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3086844741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:03.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:01:03.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:01:03.717 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id bcd22a4d --release --await' 2024-09-05T12:01:03.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id bcd22a4d --release --await 2024-09-05T12:01:03.808 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:04.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:04 smithi059 ceph-mon[30025]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:04.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:04 smithi189 ceph-mon[34772]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.362+0000 7fa858e00640 1 -- 172.21.15.59:0/3827720535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 msgr2=0x7fa854074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.362+0000 7fa858e00640 1 --2- 172.21.15.59:0/3827720535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa854074a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa844009900 tx=0x7fa84402a480 comp rx=0 tx=0).stop 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 -- 172.21.15.59:0/3827720535 shutdown_connections 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 --2- 172.21.15.59:0/3827720535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa85410e3d0 0x7fa854110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 --2- 172.21.15.59:0/3827720535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa854074f40 0x7fa85410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 --2- 172.21.15.59:0/3827720535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa854074a00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 -- 172.21.15.59:0/3827720535 >> 172.21.15.59:0/3827720535 conn(0x7fa8540fd8b0 msgr2=0x7fa8540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 -- 172.21.15.59:0/3827720535 shutdown_connections 2024-09-05T12:01:04.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.363+0000 7fa858e00640 1 -- 172.21.15.59:0/3827720535 wait complete. 2024-09-05T12:01:04.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 Processor -- start 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 -- start start 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa854074f40 0x7fa85410ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa85410e3d0 0x7fa85410b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541141b0 con 0x7fa854074620 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa854114030 con 0x7fa854074f40 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.364+0000 7fa858e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa854114330 con 0x7fa85410e3d0 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53486/0 (socket says 172.21.15.59:53486) 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 -- 172.21.15.59:0/3837854209 learned_addr learned my addr 172.21.15.59:0/3837854209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851400640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa854074f40 0x7fa85410ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa852800640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa85410e3d0 0x7fa85410b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 -- 172.21.15.59:0/3837854209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa85410e3d0 msgr2=0x7fa85410b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa85410e3d0 0x7fa85410b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 -- 172.21.15.59:0/3837854209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa854074f40 msgr2=0x7fa85410ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa854074f40 0x7fa85410ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.365+0000 7fa851e00640 1 -- 172.21.15.59:0/3837854209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85410bb30 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa851e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa844004770 tx=0x7fa844003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84403f070 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844031ca0 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8541aaf50 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844035030 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.366+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541ab450 con 0x7fa854074620 2024-09-05T12:01:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.367+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa854075cd0 con 0x7fa854074620 2024-09-05T12:01:04.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.371+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa844042070 con 0x7fa854074620 2024-09-05T12:01:04.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.372+0000 7fa84aa00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 0x7fa8380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:04.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.373+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa8440ef660 con 0x7fa854074620 2024-09-05T12:01:04.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.373+0000 7fa851400640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 0x7fa8380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:04.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.374+0000 7fa851400640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 0x7fa8380a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa85410c540 tx=0x7fa840005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:04.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.381+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8440bbbc0 con 0x7fa854074620 2024-09-05T12:01:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:04 smithi089 ceph-mon[34499]: pgmap v496: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.628+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8541ab7c0 con 0x7fa854074620 2024-09-05T12:01:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.629+0000 7fa84aa00640 1 -- 172.21.15.59:0/3837854209 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fa8440bfec0 con 0x7fa854074620 2024-09-05T12:01:04.631 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:01:04.631 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:01:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 msgr2=0x7fa8380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 0x7fa8380a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa85410c540 tx=0x7fa840005f90 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 msgr2=0x7fa85410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa844004770 tx=0x7fa844003c10 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 shutdown_connections 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8380a2c00 0x7fa8380a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa85410e3d0 0x7fa85410b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa854074f40 0x7fa85410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 --2- 172.21.15.59:0/3837854209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa854074620 0x7fa85410dcd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 >> 172.21.15.59:0/3837854209 conn(0x7fa8540fd8b0 msgr2=0x7fa8540fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 shutdown_connections 2024-09-05T12:01:04.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:04.632+0000 7fa858e00640 1 -- 172.21.15.59:0/3837854209 wait complete. 2024-09-05T12:01:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 -- 172.21.15.59:0/1140087298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c103ff0 msgr2=0x7fa80c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 --2- 172.21.15.59:0/1140087298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c103ff0 0x7fa80c1043d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa800009900 tx=0x7fa80002a410 comp rx=0 tx=0).stop 2024-09-05T12:01:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 -- 172.21.15.59:0/1140087298 shutdown_connections 2024-09-05T12:01:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 --2- 172.21.15.59:0/1140087298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c109120 0x7fa80c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 --2- 172.21.15.59:0/1140087298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 --2- 172.21.15.59:0/1140087298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c103ff0 0x7fa80c1043d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 -- 172.21.15.59:0/1140087298 >> 172.21.15.59:0/1140087298 conn(0x7fa80c0ffa90 msgr2=0x7fa80c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 -- 172.21.15.59:0/1140087298 shutdown_connections 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.177+0000 7fa813400640 1 -- 172.21.15.59:0/1140087298 wait complete. 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 Processor -- start 2024-09-05T12:01:05.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 -- start start 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c103ff0 0x7fa80c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 0x7fa80c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c114170 con 0x7fa80c109120 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c113ff0 con 0x7fa80c103ff0 2024-09-05T12:01:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.179+0000 7fa813400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1142f0 con 0x7fa80c1049a0 2024-09-05T12:01:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa811400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 0x7fa80c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56296/0 (socket says 172.21.15.59:56296) 2024-09-05T12:01:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 -- 172.21.15.59:0/3081196349 learned_addr learned my addr 172.21.15.59:0/3081196349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa810a00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c103ff0 0x7fa80c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 -- 172.21.15.59:0/3081196349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c103ff0 msgr2=0x7fa80c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c103ff0 0x7fa80c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 -- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 msgr2=0x7fa80c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 0x7fa80c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa80be00640 1 -- 172.21.15.59:0/3081196349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c19f590 con 0x7fa80c1049a0 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.180+0000 7fa811400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 0x7fa80c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa80be00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ea60 tx=0x7fa7f800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa809600640 1 -- 172.21.15.59:0/3081196349 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800cde0 con 0x7fa80c1049a0 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa809600640 1 -- 172.21.15.59:0/3081196349 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f80035f0 con 0x7fa80c1049a0 2024-09-05T12:01:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa809600640 1 -- 172.21.15.59:0/3081196349 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8010670 con 0x7fa80c1049a0 2024-09-05T12:01:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c19f880 con 0x7fa80c1049a0 2024-09-05T12:01:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.181+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c1a7160 con 0x7fa80c1049a0 2024-09-05T12:01:05.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.185+0000 7fa816048740 1 Processor -- start 2024-09-05T12:01:05.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.185+0000 7fa816048740 1 -- start start 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.185+0000 7fa809600640 1 -- 172.21.15.59:0/3081196349 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7f80037a0 con 0x7fa80c1049a0 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eb024f0 0x55eb3eb048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb3ea5d650 con 0x55eb3eaf7310 2024-09-05T12:01:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb3ea04f10 con 0x55eb3eb024f0 2024-09-05T12:01:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa816048740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb3ea59610 con 0x55eb3eaf7c30 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa811400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa811400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53508/0 (socket says 172.21.15.59:53508) 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa811400640 1 -- 172.21.15.59:0/2690783928 learned_addr learned my addr 172.21.15.59:0/2690783928 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa80be00640 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eb024f0 0x55eb3eb048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.186+0000 7fa810a00640 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 -- 172.21.15.59:0/2690783928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 msgr2=0x55eb3eb01fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb01fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 -- 172.21.15.59:0/2690783928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eb024f0 msgr2=0x55eb3eb048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eb024f0 0x55eb3eb048e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 -- 172.21.15.59:0/2690783928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3e76c580 con 0x55eb3eaf7310 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa811400640 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa804011790 tx=0x7fa804011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d14636cf6ee78e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa7fdc00640 1 -- 172.21.15.59:0/2690783928 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804003c30 con 0x55eb3eaf7310 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa7fdc00640 1 -- 172.21.15.59:0/2690783928 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80400fe80 con 0x55eb3eaf7310 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.187+0000 7fa7fdc00640 1 -- 172.21.15.59:0/2690783928 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040193e0 con 0x55eb3eaf7310 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa809600640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 0x7fa7f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa809600640 1 -- 172.21.15.59:0/3081196349 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7f80cf740 con 0x7fa80c1049a0 2024-09-05T12:01:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa810a00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 0x7fa7f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 -- 172.21.15.59:0/2690783928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 msgr2=0x55eb3eaf76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa804011790 tx=0x7fa804011c50 comp rx=0 tx=0).stop 2024-09-05T12:01:05.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 -- 172.21.15.59:0/2690783928 shutdown_connections 2024-09-05T12:01:05.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eb024f0 0x55eb3eb048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 --2- 172.21.15.59:0/2690783928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf76f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.188+0000 7fa816048740 1 -- 172.21.15.59:0/2690783928 >> 172.21.15.59:0/2690783928 conn(0x55eb3eaf38a0 msgr2=0x55eb3eaf3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.189+0000 7fa816048740 1 -- 172.21.15.59:0/2690783928 shutdown_connections 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.189+0000 7fa816048740 1 -- 172.21.15.59:0/2690783928 wait complete. 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.189+0000 7fa810a00640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 0x7fa7f40a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa800000c00 tx=0x7fa800005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.189+0000 7fa816048740 1 -- start start 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.189+0000 7fa816048740 1 Processor -- start 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eaff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eaffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb3eb050c0 con 0x55eb3eaf7c30 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eb3ea18350 con 0x55eb3eaf7310 2024-09-05T12:01:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb3eb05240 con 0x55eb3eb024f0 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa816048740 10 client.0 _ll_register_callbacks cb 0x55eb3ea3b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa810a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa811400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eaffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa811400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eaffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56304/0 (socket says 172.21.15.59:56304) 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa810a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42792/0 (socket says 172.21.15.59:42792) 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa80be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eaff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.190+0000 7fa811400640 1 -- 172.21.15.59:0/719157770 learned_addr learned my addr 172.21.15.59:0/719157770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 -- 172.21.15.59:0/719157770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 msgr2=0x55eb3eaffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eaffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 -- 172.21.15.59:0/719157770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 msgr2=0x55eb3eaff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eaff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 -- 172.21.15.59:0/719157770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3e836960 con 0x55eb3eaf7310 2024-09-05T12:01:05.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa810a00640 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa80002aa00 tx=0x7fa800009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57bc1bd61e9fee73 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.191+0000 7fa7f2000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80004e070 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb3ea6abc0 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa8000048c0 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa80005a050 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa816048740 10 client.25904 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55eb3eb10dc0 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eb3ea6abc0 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa8000035f0 con 0x55eb3eaf7310 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.192+0000 7fa816048740 10 client.25904 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:05.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa816048740 10 client.25904 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa816048740 20 client.25904 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb121d0 0x55eb3eb14600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa816048740 4 client.25904 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3ea650d0 con 0x55eb3eb121d0 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa80be00640 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb121d0 0x55eb3eb14600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa80be00640 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb121d0 0x55eb3eb14600 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2356ceb7a315646 server_cookie=3d9adfd78f50feee in_seq=0 out_seq=0 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.193+0000 7fa7f2000640 10 client.25904 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.194+0000 7fa7f2000640 1 -- 172.21.15.59:0/719157770 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb3ea650d0 con 0x55eb3eb121d0 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.194+0000 7fa7f2000640 10 client.25904 handle_command_reply: tid=0 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.194+0000 7fa816048740 1 client.25904 shutdown 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.194+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 msgr2=0x55eb3eaff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.194+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa80002aa00 tx=0x7fa800009260 comp rx=0 tx=0).stop 2024-09-05T12:01:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 shutdown_connections 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eaffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb121d0 0x55eb3eb14600 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eaff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 --2- 172.21.15.59:0/719157770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaff470 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa7f2000640 0 client.25904 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 >> 172.21.15.59:0/719157770 conn(0x55eb3eaf38a0 msgr2=0x55eb3eaf2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 shutdown_connections 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 1 -- 172.21.15.59:0/719157770 wait complete. 2024-09-05T12:01:05.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.195+0000 7fa816048740 20 client.25904 trim_cache size 0 max 16384 2024-09-05T12:01:05.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.213+0000 7fa816048740 1 Processor -- start 2024-09-05T12:01:05.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.213+0000 7fa816048740 1 -- start start 2024-09-05T12:01:05.217 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:05 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3837854209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:05.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eb1a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1af030 con 0x55eb3eaf7c30 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b18b0 con 0x55eb3eaf7310 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa816048740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1af6a0 con 0x55eb3eb024f0 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa80be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa810a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa80be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53524/0 (socket says 172.21.15.59:53524) 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa80be00640 1 -- 172.21.15.59:0/4147729573 learned_addr learned my addr 172.21.15.59:0/4147729573 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa811400640 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eb1a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa810a00640 1 -- 172.21.15.59:0/4147729573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 msgr2=0x55eb3eb1a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.214+0000 7fa810a00640 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eb1a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa810a00640 1 -- 172.21.15.59:0/4147729573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 msgr2=0x55eb3eb19ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa810a00640 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa810a00640 1 -- 172.21.15.59:0/4147729573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3eaef260 con 0x55eb3eaf7310 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa80be00640 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa810a00640 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf9580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa800005c40 tx=0x7fa800005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f18997a73db735 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa7fd200640 1 -- 172.21.15.59:0/4147729573 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80004e070 con 0x55eb3eaf7310 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa7fd200640 1 -- 172.21.15.59:0/4147729573 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa800063070 con 0x55eb3eaf7310 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.215+0000 7fa7fd200640 1 -- 172.21.15.59:0/4147729573 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800035040 con 0x55eb3eaf7310 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 -- 172.21.15.59:0/4147729573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 msgr2=0x55eb3eaf9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf9580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa800005c40 tx=0x7fa800005960 comp rx=0 tx=0).stop 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 -- 172.21.15.59:0/4147729573 shutdown_connections 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eb024f0 0x55eb3eb1a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb19ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 --2- 172.21.15.59:0/4147729573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eaf9580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 -- 172.21.15.59:0/4147729573 >> 172.21.15.59:0/4147729573 conn(0x55eb3eaf38a0 msgr2=0x55eb3eafe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 -- 172.21.15.59:0/4147729573 shutdown_connections 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.216+0000 7fa816048740 1 -- 172.21.15.59:0/4147729573 wait complete. 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 -- start start 2024-09-05T12:01:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 Processor -- start 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb06950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eb024f0 0x55eb3eb06e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1afd70 con 0x55eb3eb024f0 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b11e0 con 0x55eb3eaf7310 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1b0440 con 0x55eb3eaf7c30 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.217+0000 7fa816048740 10 client.0 _ll_register_callbacks cb 0x7fa800009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42820/0 (socket says 172.21.15.59:42820) 2024-09-05T12:01:05.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa80be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 -- 172.21.15.59:0/881747351 learned_addr learned my addr 172.21.15.59:0/881747351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa811400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eb024f0 0x55eb3eb06e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 -- 172.21.15.59:0/881747351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 msgr2=0x55eb3eb06950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb06950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 -- 172.21.15.59:0/881747351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eb024f0 msgr2=0x55eb3eb06e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eb024f0 0x55eb3eb06e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 -- 172.21.15.59:0/881747351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3eb7c0f0 con 0x55eb3eaf7310 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa80be00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa810a00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8000054c0 tx=0x7fa8000043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22e04a24dafe68e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.218+0000 7fa7f2000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80006b020 con 0x55eb3eaf7310 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa8000048c0 con 0x55eb3eaf7310 2024-09-05T12:01:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa800063070 con 0x55eb3eaf7310 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa800076050 con 0x55eb3eaf7310 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa816048740 10 client.25910 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.219+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55eb3eb7d3f0 con 0x55eb3eaf7310 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa800063070 con 0x55eb3eaf7310 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa80004a450 con 0x55eb3eaf7310 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 10 client.25910 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 10 client.25910 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 20 client.25910 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb7da40 0x55eb3eb7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 4 client.25910 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bcd22a4d", "release": true, "await": true}] 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bcd22a4d", "release": true, "await": true}) -- 0x55eb3e9e1430 con 0x55eb3eb7da40 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.220+0000 7fa80be00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb7da40 0x55eb3eb7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.221+0000 7fa80be00640 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb7da40 0x55eb3eb7fe30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380d3733f7124dd3 server_cookie=5959196debc6060b in_seq=0 out_seq=0 2024-09-05T12:01:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.221+0000 7fa7f2000640 10 client.25910 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.222+0000 7fa7f2000640 1 -- 172.21.15.59:0/881747351 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eb3e9e1430 con 0x55eb3eb7da40 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.222+0000 7fa7f2000640 10 client.25910 handle_command_reply: tid=0 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.222+0000 7fa816048740 1 client.25910 shutdown 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 msgr2=0x55eb3eb06410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8000054c0 tx=0x7fa8000043c0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 shutdown_connections 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55eb3eb7da40 0x55eb3eb7fe30 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb3eb024f0 0x55eb3eb06e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3eaf7c30 0x55eb3eb06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 --2- 172.21.15.59:0/881747351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb3eaf7310 0x55eb3eb06410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 >> 172.21.15.59:0/881747351 conn(0x55eb3eaf38a0 msgr2=0x55eb3eaf2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa7f2000640 0 client.25910 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 shutdown_connections 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 1 -- 172.21.15.59:0/881747351 wait complete. 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.223+0000 7fa816048740 20 client.25910 trim_cache size 0 max 16384 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 16, 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "bcd22a4d": { 2024-09-05T12:01:05.228 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 16, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 154.1, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:01:05.229 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:05.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 msgr2=0x7fa7f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 0x7fa7f40a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa800000c00 tx=0x7fa800005b50 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 msgr2=0x7fa80c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ea60 tx=0x7fa7f800ef20 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 shutdown_connections 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7f40a2c00 0x7fa7f40a50c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa80c109120 0x7fa80c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa80c1049a0 0x7fa80c19ab20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 --2- 172.21.15.59:0/3081196349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa80c103ff0 0x7fa80c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:05.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 >> 172.21.15.59:0/3081196349 conn(0x7fa80c0ffa90 msgr2=0x7fa80c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:05.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 shutdown_connections 2024-09-05T12:01:05.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:05.224+0000 7fa813400640 1 -- 172.21.15.59:0/3081196349 wait complete. 2024-09-05T12:01:05.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:05 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3837854209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:05.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:05 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3837854209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:06.126 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:01:06.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:06 smithi189 ceph-mon[34772]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:06 smithi089 ceph-mon[34499]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:06.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:06 smithi059 ceph-mon[30025]: pgmap v497: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:01:06.614 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bcd22a4d', total seconds elapsed: 40.2 2024-09-05T12:01:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:07.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 -- 172.21.15.59:0/3462749117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 msgr2=0x7fa7c410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3462749117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c410df50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02a410 comp rx=0 tx=0).stop 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 -- 172.21.15.59:0/3462749117 shutdown_connections 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3462749117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c410e490 0x7fa7c4110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3462749117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c410df50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3462749117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c41008e0 0x7fa7c4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 -- 172.21.15.59:0/3462749117 >> 172.21.15.59:0/3462749117 conn(0x7fa7c40fc7d0 msgr2=0x7fa7c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 -- 172.21.15.59:0/3462749117 shutdown_connections 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.306+0000 7fa7cb400640 1 -- 172.21.15.59:0/3462749117 wait complete. 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.307+0000 7fa7cb400640 1 Processor -- start 2024-09-05T12:01:07.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.307+0000 7fa7cb400640 1 -- start start 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c41008e0 0x7fa7c419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c410e490 0x7fa7c419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41141c0 con 0x7fa7c4101290 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4114040 con 0x7fa7c41008e0 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7cb400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4114340 con 0x7fa7c410e490 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53554/0 (socket says 172.21.15.59:53554) 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 -- 172.21.15.59:0/3645825686 learned_addr learned my addr 172.21.15.59:0/3645825686 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c41008e0 0x7fa7c419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7c9400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c410e490 0x7fa7c419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 -- 172.21.15.59:0/3645825686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c410e490 msgr2=0x7fa7c419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c410e490 0x7fa7c419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 -- 172.21.15.59:0/3645825686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c41008e0 msgr2=0x7fa7c419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c41008e0 0x7fa7c419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.308+0000 7fa7bbe00640 1 -- 172.21.15.59:0/3645825686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419f660 con 0x7fa7c4101290 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006010 tx=0x7fa7ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7c2000640 1 -- 172.21.15.59:0/3645825686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac03f030 con 0x7fa7c4101290 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7c2000640 1 -- 172.21.15.59:0/3645825686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac003ce0 con 0x7fa7c4101290 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c419f8f0 con 0x7fa7c4101290 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7c2000640 1 -- 172.21.15.59:0/3645825686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac035070 con 0x7fa7c4101290 2024-09-05T12:01:07.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.309+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41a7230 con 0x7fa7c4101290 2024-09-05T12:01:07.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.312+0000 7fa7c2000640 1 -- 172.21.15.59:0/3645825686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7ac042030 con 0x7fa7c4101290 2024-09-05T12:01:07.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 Processor -- start 2024-09-05T12:01:07.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 -- start start 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca82240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b3c9db3e0 con 0x564b3ca75590 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564b3c982d80 con 0x564b3ca74c70 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7cde96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b3c9d73a0 con 0x564b3ca7fe50 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53560/0 (socket says 172.21.15.59:53560) 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca82240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7c8a00640 1 -- 172.21.15.59:0/2594871238 learned_addr learned my addr 172.21.15.59:0/2594871238 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:07.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.313+0000 7fa7c9400640 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca75050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 -- 172.21.15.59:0/2594871238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 msgr2=0x564b3ca82240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca82240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 -- 172.21.15.59:0/2594871238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 msgr2=0x564b3ca75050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca75050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 -- 172.21.15.59:0/2594871238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3c9a1ae0 con 0x564b3ca75590 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009d20 tx=0x7fa7b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe7f88e05425793 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7c2000640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 0x7fa7a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.314+0000 7fa7b9c00640 1 -- 172.21.15.59:0/2594871238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b401e070 con 0x564b3ca75590 2024-09-05T12:01:07.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7c2000640 1 -- 172.21.15.59:0/3645825686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7ac0ef960 con 0x7fa7c4101290 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7b9c00640 1 -- 172.21.15.59:0/2594871238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4019b10 con 0x564b3ca75590 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7b9c00640 1 -- 172.21.15.59:0/2594871238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ebb0 con 0x564b3ca75590 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7c9400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 0x7fa7a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 -- 172.21.15.59:0/2594871238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 msgr2=0x564b3ca7f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009d20 tx=0x7fa7b4011810 comp rx=0 tx=0).stop 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 -- 172.21.15.59:0/2594871238 shutdown_connections 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca82240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca7f910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2594871238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.315+0000 7fa7cde96740 1 -- 172.21.15.59:0/2594871238 >> 172.21.15.59:0/2594871238 conn(0x564b3ca71200 msgr2=0x564b3ca715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.316+0000 7fa7c9400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 0x7fa7a80a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7c406a160 tx=0x7fa7bc009600 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.316+0000 7fa7cde96740 1 -- 172.21.15.59:0/2594871238 shutdown_connections 2024-09-05T12:01:07.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.316+0000 7fa7cde96740 1 -- 172.21.15.59:0/2594871238 wait complete. 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.316+0000 7fa7cde96740 1 -- start start 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.316+0000 7fa7cde96740 1 Processor -- start 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca75590 0x564b3ca7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b3ca82a20 con 0x564b3ca74c70 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564b3c9960f0 con 0x564b3ca75590 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b3ca82ba0 con 0x564b3ca7fe50 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7cde96740 10 client.0 _ll_register_callbacks cb 0x564b3c97dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:07.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53568/0 (socket says 172.21.15.59:53568) 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7c8a00640 1 -- 172.21.15.59:0/569384337 learned_addr learned my addr 172.21.15.59:0/569384337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7c9400640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.317+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca75590 0x564b3ca7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 -- 172.21.15.59:0/569384337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 msgr2=0x564b3ca7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 -- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca75590 msgr2=0x564b3ca7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca75590 0x564b3ca7d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 -- 172.21.15.59:0/569384337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3c74c3c0 con 0x564b3ca74c70 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005890 tx=0x7fa7b4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3850a0c98999260 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7b2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4027020 con 0x564b3ca74c70 2024-09-05T12:01:07.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7b400eaf0 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.318+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4019b10 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa7b4032050 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 10 client.15900 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564b3ca8e9f0 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4019b10 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa7b40046c0 con 0x564b3ca74c70 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 10 client.15900 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 20 client.15900 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3ca8fdf0 0x564b3ca921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 4 client.15900 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.319+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b3c9e8950 con 0x564b3ca8fdf0 2024-09-05T12:01:07.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.320+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3ca8fdf0 0x564b3ca921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.320+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3ca8fdf0 0x564b3ca921e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe1abbed22123d0 server_cookie=8c30d9098530f497 in_seq=0 out_seq=0 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.320+0000 7fa7b2000640 10 client.15900 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7b2000640 1 -- 172.21.15.59:0/569384337 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b3c9e8950 con 0x564b3ca8fdf0 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7b2000640 10 client.15900 handle_command_reply: tid=0 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 client.15900 shutdown 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 msgr2=0x564b3ca7cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005890 tx=0x7fa7b4019660 comp rx=0 tx=0).stop 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 shutdown_connections 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca75590 0x564b3ca7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3ca8fdf0 0x564b3ca921e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 --2- 172.21.15.59:0/569384337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca74c70 0x564b3ca7cce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7b2000640 0 client.15900 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 >> 172.21.15.59:0/569384337 conn(0x564b3ca71200 msgr2=0x564b3ca70380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 shutdown_connections 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.321+0000 7fa7cde96740 1 -- 172.21.15.59:0/569384337 wait complete. 2024-09-05T12:01:07.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.322+0000 7fa7cde96740 20 client.15900 trim_cache size 0 max 16384 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.335+0000 7fa7cde96740 1 Processor -- start 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 -- start start 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca75590 0x564b3ca7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41af0c0 con 0x564b3ca7fe50 2024-09-05T12:01:07.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b1940 con 0x564b3ca74c70 2024-09-05T12:01:07.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7cde96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41af730 con 0x564b3ca75590 2024-09-05T12:01:07.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53576/0 (socket says 172.21.15.59:53576) 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42874/0 (socket says 172.21.15.59:42874) 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7c9400640 1 -- 172.21.15.59:0/1789317461 learned_addr learned my addr 172.21.15.59:0/1789317461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.336+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca75590 0x564b3ca7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 -- 172.21.15.59:0/1789317461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca75590 msgr2=0x564b3ca7b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca75590 0x564b3ca7b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 -- 172.21.15.59:0/1789317461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 msgr2=0x564b3ca776d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 -- 172.21.15.59:0/1789317461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3ca72390 con 0x564b3ca7fe50 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7c9400640 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc011720 tx=0x7fa7bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bad3d87a690bf5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7b9200640 1 -- 172.21.15.59:0/1789317461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011dd0 con 0x564b3ca7fe50 2024-09-05T12:01:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7b9200640 1 -- 172.21.15.59:0/1789317461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc01d910 con 0x564b3ca7fe50 2024-09-05T12:01:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.337+0000 7fa7b9200640 1 -- 172.21.15.59:0/1789317461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc020ac0 con 0x564b3ca7fe50 2024-09-05T12:01:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 -- 172.21.15.59:0/1789317461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 msgr2=0x564b3ca97970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc011720 tx=0x7fa7bc016000 comp rx=0 tx=0).stop 2024-09-05T12:01:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 -- 172.21.15.59:0/1789317461 shutdown_connections 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca7fe50 0x564b3ca97970 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca75590 0x564b3ca7b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 --2- 172.21.15.59:0/1789317461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 -- 172.21.15.59:0/1789317461 >> 172.21.15.59:0/1789317461 conn(0x564b3ca71200 msgr2=0x564b3ca7ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 -- 172.21.15.59:0/1789317461 shutdown_connections 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.338+0000 7fa7cde96740 1 -- 172.21.15.59:0/1789317461 wait complete. 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 -- start start 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 Processor -- start 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca83ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca84530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41afe00 con 0x564b3ca75590 2024-09-05T12:01:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b1270 con 0x564b3ca74c70 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41b04d0 con 0x564b3ca7fe50 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7cde96740 10 client.0 _ll_register_callbacks cb 0x7fa7b401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.339+0000 7fa7bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7c8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca83ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53592/0 (socket says 172.21.15.59:53592) 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 -- 172.21.15.59:0/2302211094 learned_addr learned my addr 172.21.15.59:0/2302211094 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7c9400640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca84530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 -- 172.21.15.59:0/2302211094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 msgr2=0x564b3ca84530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca84530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 -- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 msgr2=0x564b3ca83ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca83ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 -- 172.21.15.59:0/2302211094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3caf9940 con 0x564b3ca75590 2024-09-05T12:01:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca83ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7bbe00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac03b610 tx=0x7fa7ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2b87ca9713a598 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7b2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.340+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac049040 con 0x564b3ca75590 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7ac0b8530 con 0x564b3ca75590 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7bc01d910 con 0x564b3ca75590 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa7ac0bba00 con 0x564b3ca75590 2024-09-05T12:01:07.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 10 client.15906 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564b3cafab80 con 0x564b3ca75590 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc01d910 con 0x564b3ca75590 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa7ac0bc120 con 0x564b3ca75590 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 10 client.15906 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 20 client.15906 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3cafb2b0 0x564b3cafd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 4 client.15906 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.341+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b3c9e2e60 con 0x564b3cafb2b0 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.342+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3cafb2b0 0x564b3cafd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.342+0000 7fa7c8a00640 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3cafb2b0 0x564b3cafd6d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365170536007b94 server_cookie=997585f07f8531f9 in_seq=0 out_seq=0 2024-09-05T12:01:07.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.342+0000 7fa7b2000640 10 client.15906 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7b2000640 1 -- 172.21.15.59:0/2302211094 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b3c9e2e60 con 0x564b3cafb2b0 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7b2000640 10 client.15906 handle_command_reply: tid=0 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 client.15906 shutdown 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 msgr2=0x564b3ca83ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac03b610 tx=0x7fa7ac004270 comp rx=0 tx=0).stop 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 shutdown_connections 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b3ca7fe50 0x564b3ca84530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564b3cafb2b0 0x564b3cafd6d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b3ca75590 0x564b3ca83ff0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 --2- 172.21.15.59:0/2302211094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564b3ca74c70 0x564b3ca83ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 >> 172.21.15.59:0/2302211094 conn(0x564b3ca71200 msgr2=0x564b3ca6fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7b2000640 0 client.15906 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:07.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 shutdown_connections 2024-09-05T12:01:07.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 1 -- 172.21.15.59:0/2302211094 wait complete. 2024-09-05T12:01:07.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.343+0000 7fa7cde96740 20 client.15906 trim_cache size 0 max 16384 2024-09-05T12:01:07.349 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 msgr2=0x7fa7a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 0x7fa7a80a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7c406a160 tx=0x7fa7bc009600 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 msgr2=0x7fa7c419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006010 tx=0x7fa7ac0042f0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 shutdown_connections 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7a80a2c00 0x7fa7a80a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7c410e490 0x7fa7c419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7c4101290 0x7fa7c419abf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 --2- 172.21.15.59:0/3645825686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7c41008e0 0x7fa7c419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 >> 172.21.15.59:0/3645825686 conn(0x7fa7c40fc7d0 msgr2=0x7fa7c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 shutdown_connections 2024-09-05T12:01:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:07.344+0000 7fa7cb400640 1 -- 172.21.15.59:0/3645825686 wait complete. 2024-09-05T12:01:08.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:08 smithi189 ceph-mon[34772]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T12:01:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:08 smithi089 ceph-mon[34499]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T12:01:08.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:08 smithi059 ceph-mon[30025]: pgmap v498: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-05T12:01:09.196 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:01:09.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:01:09.589 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 -- 172.21.15.59:0/575697590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928101620 msgr2=0x7ff928101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 --2- 172.21.15.59:0/575697590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928101620 0x7ff928101a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff910009900 tx=0x7ff91002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 -- 172.21.15.59:0/575697590 shutdown_connections 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 --2- 172.21.15.59:0/575697590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101fc0 0x7ff92810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 --2- 172.21.15.59:0/575697590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928101620 0x7ff928101a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 --2- 172.21.15.59:0/575697590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928105aa0 0x7ff928105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 -- 172.21.15.59:0/575697590 >> 172.21.15.59:0/575697590 conn(0x7ff9280fba90 msgr2=0x7ff9280fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 -- 172.21.15.59:0/575697590 shutdown_connections 2024-09-05T12:01:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.190+0000 7ff92e400640 1 -- 172.21.15.59:0/575697590 wait complete. 2024-09-05T12:01:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 Processor -- start 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 -- start start 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 0x7ff92819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928105aa0 0x7ff92819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff928110ec0 con 0x7ff928101620 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff928110d40 con 0x7ff928105aa0 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.191+0000 7ff92e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff928111040 con 0x7ff928101fc0 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 0x7ff92819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53610/0 (socket says 172.21.15.59:53610) 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 -- 172.21.15.59:0/383124495 learned_addr learned my addr 172.21.15.59:0/383124495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927e00640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928105aa0 0x7ff92819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 -- 172.21.15.59:0/383124495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 msgr2=0x7ff92819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 0x7ff92819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 -- 172.21.15.59:0/383124495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928105aa0 msgr2=0x7ff92819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928105aa0 0x7ff92819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff927400640 1 -- 172.21.15.59:0/383124495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92819f650 con 0x7ff928101620 2024-09-05T12:01:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.192+0000 7ff91fe00640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 0x7ff92819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff927400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9f0 tx=0x7ff91800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff924c00640 1 -- 172.21.15.59:0/383124495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91800cd80 con 0x7ff928101620 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92819f940 con 0x7ff928101620 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff924c00640 1 -- 172.21.15.59:0/383124495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff918005500 con 0x7ff928101620 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff924c00640 1 -- 172.21.15.59:0/383124495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918010690 con 0x7ff928101620 2024-09-05T12:01:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.193+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9281a7220 con 0x7ff928101620 2024-09-05T12:01:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.196+0000 7ff93105e740 1 Processor -- start 2024-09-05T12:01:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.196+0000 7ff924c00640 1 -- 172.21.15.59:0/383124495 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9180046c0 con 0x7ff928101620 2024-09-05T12:01:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.196+0000 7ff93105e740 1 -- start start 2024-09-05T12:01:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.196+0000 7ff93105e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.196+0000 7ff93105e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890aea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff93105e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890aeaca0 0x561890aed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff93105e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561890a46290 con 0x561890adfac0 2024-09-05T12:01:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff93105e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5618909edb40 con 0x561890aeaca0 2024-09-05T12:01:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff93105e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561890a42250 con 0x561890ae03e0 2024-09-05T12:01:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53624/0 (socket says 172.21.15.59:53624) 2024-09-05T12:01:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 -- 172.21.15.59:0/902100931 learned_addr learned my addr 172.21.15.59:0/902100931 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927400640 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890aea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 -- 172.21.15.59:0/902100931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 msgr2=0x561890aea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890aea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 -- 172.21.15.59:0/902100931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890aeaca0 msgr2=0x561890aed090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890aeaca0 0x561890aed090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.197+0000 7ff927e00640 1 -- 172.21.15.59:0/902100931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618907b76a0 con 0x561890adfac0 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff924c00640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 0x7ff90c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff927e00640 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff920010370 tx=0x7ff920010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19799e7d9942648f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff924c00640 1 -- 172.21.15.59:0/383124495 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff9180cf830 con 0x7ff928101620 2024-09-05T12:01:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff927400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 0x7ff90c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff91cc00640 1 -- 172.21.15.59:0/902100931 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920016070 con 0x561890adfac0 2024-09-05T12:01:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff91cc00640 1 -- 172.21.15.59:0/902100931 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff920004d20 con 0x561890adfac0 2024-09-05T12:01:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.198+0000 7ff91cc00640 1 -- 172.21.15.59:0/902100931 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920005480 con 0x561890adfac0 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 -- 172.21.15.59:0/902100931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 msgr2=0x561890adfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff927400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 0x7ff90c0a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9180a4360 tx=0x7ff9180d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff920010370 tx=0x7ff920010830 comp rx=0 tx=0).stop 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 -- 172.21.15.59:0/902100931 shutdown_connections 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890aeaca0 0x561890aed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890aea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 --2- 172.21.15.59:0/902100931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890adfea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 -- 172.21.15.59:0/902100931 >> 172.21.15.59:0/902100931 conn(0x561890adc050 msgr2=0x561890adc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 -- 172.21.15.59:0/902100931 shutdown_connections 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.199+0000 7ff93105e740 1 -- 172.21.15.59:0/902100931 wait complete. 2024-09-05T12:01:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 -- start start 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 Processor -- start 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 0x561890aea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890ae8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561890aed910 con 0x561890aeaca0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561890a00fa0 con 0x561890adfac0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561890aeda90 con 0x561890ae03e0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.200+0000 7ff93105e740 10 client.0 _ll_register_callbacks cb 0x561890754ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff927e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 0x561890aea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff927e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 0x561890aea710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42916/0 (socket says 172.21.15.59:42916) 2024-09-05T12:01:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff927e00640 1 -- 172.21.15.59:0/3887549732 learned_addr learned my addr 172.21.15.59:0/3887549732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56418/0 (socket says 172.21.15.59:56418) 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff927e00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890ae8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 -- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 msgr2=0x561890aea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 0x561890aea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 -- 172.21.15.59:0/3887549732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 msgr2=0x561890ae8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890ae8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.201+0000 7ff91fe00640 1 -- 172.21.15.59:0/3887549732 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561890af6d80 con 0x561890ae03e0 2024-09-05T12:01:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff910005940 tx=0x7ff9100317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ce4e4ca1b46e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff915600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:01:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910035040 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff920004d20 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff910004250 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff91004f050 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff93105e740 10 client.25855 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.202+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561890af99b0 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff920004d20 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff91003c440 con 0x561890ae03e0 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 10 client.25855 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 20 client.25855 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890afadc0 0x561890afd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 4 client.25855 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561890a53800 con 0x561890afadc0 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.203+0000 7ff927e00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890afadc0 0x561890afd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.204+0000 7ff927e00640 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890afadc0 0x561890afd1b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce9f8c115ed4bbf server_cookie=bffed585e6f01cdd in_seq=0 out_seq=0 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.204+0000 7ff915600640 10 client.25855 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff915600640 1 -- 172.21.15.59:0/3887549732 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561890a53800 con 0x561890afadc0 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff915600640 10 client.25855 handle_command_reply: tid=0 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 client.25855 shutdown 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 msgr2=0x561890ae79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff910005940 tx=0x7ff9100317e0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 shutdown_connections 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890afadc0 0x561890afd1b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890ae8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890ae03e0 0x561890ae79c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 --2- 172.21.15.59:0/3887549732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890adfac0 0x561890aea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff915600640 0 client.25855 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 >> 172.21.15.59:0/3887549732 conn(0x561890adc050 msgr2=0x561890ae1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 shutdown_connections 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 1 -- 172.21.15.59:0/3887549732 wait complete. 2024-09-05T12:01:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.205+0000 7ff93105e740 20 client.25855 trim_cache size 0 max 16384 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.223+0000 7ff93105e740 1 Processor -- start 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.223+0000 7ff93105e740 1 -- start start 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890aeaca0 0x561890ae4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281af110 con 0x561890adfac0 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281b1990 con 0x561890ae03e0 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff93105e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281af780 con 0x561890aeaca0 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff927e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff927e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53646/0 (socket says 172.21.15.59:53646) 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff927e00640 1 -- 172.21.15.59:0/3762890634 learned_addr learned my addr 172.21.15.59:0/3762890634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.224+0000 7ff927e00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890aeaca0 0x561890ae4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 -- 172.21.15.59:0/3762890634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890aeaca0 msgr2=0x561890ae4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890aeaca0 0x561890ae4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 -- 172.21.15.59:0/3762890634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 msgr2=0x561890b02a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 -- 172.21.15.59:0/3762890634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561890ae9d60 con 0x561890ae03e0 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff927e00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff91fe00640 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b02fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x561890ae8f70 tx=0x7ff910000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374b29d4f222d296 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff917e00640 1 -- 172.21.15.59:0/3762890634 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910058070 con 0x561890ae03e0 2024-09-05T12:01:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff917e00640 1 -- 172.21.15.59:0/3762890634 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff910005550 con 0x561890ae03e0 2024-09-05T12:01:10.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.225+0000 7ff917e00640 1 -- 172.21.15.59:0/3762890634 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91004c040 con 0x561890ae03e0 2024-09-05T12:01:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 -- 172.21.15.59:0/3762890634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 msgr2=0x561890b02fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b02fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x561890ae8f70 tx=0x7ff910000bc0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 -- 172.21.15.59:0/3762890634 shutdown_connections 2024-09-05T12:01:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890aeaca0 0x561890ae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b02fc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 --2- 172.21.15.59:0/3762890634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890adfac0 0x561890b02a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 -- 172.21.15.59:0/3762890634 >> 172.21.15.59:0/3762890634 conn(0x561890adc050 msgr2=0x561890ae2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 -- 172.21.15.59:0/3762890634 shutdown_connections 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.226+0000 7ff93105e740 1 -- 172.21.15.59:0/3762890634 wait complete. 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 -- start start 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 Processor -- start 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890adfac0 0x561890b5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890aef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281afe50 con 0x561890aeaca0 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281b12c0 con 0x561890ae03e0 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9281b0520 con 0x561890adfac0 2024-09-05T12:01:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.227+0000 7ff93105e740 10 client.0 _ll_register_callbacks cb 0x7ff910009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42942/0 (socket says 172.21.15.59:42942) 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 -- 172.21.15.59:0/2957154154 learned_addr learned my addr 172.21.15.59:0/2957154154 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff927e00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890adfac0 0x561890b5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff927e00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890aef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 -- 172.21.15.59:0/2957154154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890adfac0 msgr2=0x561890b5ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890adfac0 0x561890b5ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 -- 172.21.15.59:0/2957154154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 msgr2=0x561890aef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890aef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 -- 172.21.15.59:0/2957154154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561890afd520 con 0x561890ae03e0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff91fe00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff91003c930 tx=0x7ff91003c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70379322e09af9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.228+0000 7ff915600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910058040 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff910005550 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff9100319c0 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff910069050 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 10 client.25943 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561890b663c0 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff910005550 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff910064030 con 0x561890ae03e0 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 10 client.25943 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.229+0000 7ff93105e740 20 client.25943 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890b66b30 0x561890b68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff93105e740 4 client.25943 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:01:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561890a4dd10 con 0x561890b66b30 2024-09-05T12:01:10.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff927e00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890b66b30 0x561890b68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff927e00640 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890b66b30 0x561890b68f50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b1dabce85b6dbb server_cookie=17c20a54296093f3 in_seq=0 out_seq=0 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.230+0000 7ff915600640 10 client.25943 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.231+0000 7ff915600640 1 -- 172.21.15.59:0/2957154154 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561890a4dd10 con 0x561890b66b30 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.231+0000 7ff915600640 10 client.25943 handle_command_reply: tid=0 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.231+0000 7ff93105e740 1 client.25943 shutdown 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.231+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 msgr2=0x561890b5f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.231+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff91003c930 tx=0x7ff91003c6f0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 shutdown_connections 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561890b66b30 0x561890b68f50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff915600640 0 client.25943 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561890aeaca0 0x561890aef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561890ae03e0 0x561890b5f220 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 --2- 172.21.15.59:0/2957154154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561890adfac0 0x561890b5ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 >> 172.21.15.59:0/2957154154 conn(0x561890adc050 msgr2=0x561890ada9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 shutdown_connections 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 1 -- 172.21.15.59:0/2957154154 wait complete. 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.232+0000 7ff93105e740 20 client.25943 trim_cache size 0 max 16384 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:01:10.237 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "f7f18483-2536-49c9-8ad4-2a9e2eab2a52", 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 msgr2=0x7ff90c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 0x7ff90c0a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9180a4360 tx=0x7ff9180d5530 comp rx=0 tx=0).stop 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 msgr2=0x7ff92819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9f0 tx=0x7ff91800edf0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 shutdown_connections 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff90c0a2c00 0x7ff90c0a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928105aa0 0x7ff92819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff928101fc0 0x7ff92819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 --2- 172.21.15.59:0/383124495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff928101620 0x7ff92819a6a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 >> 172.21.15.59:0/383124495 conn(0x7ff9280fba90 msgr2=0x7ff928100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 shutdown_connections 2024-09-05T12:01:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:10.233+0000 7ff92e400640 1 -- 172.21.15.59:0/383124495 wait complete. 2024-09-05T12:01:10.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:10 smithi189 ceph-mon[34772]: pgmap v499: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T12:01:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:10 smithi089 ceph-mon[34499]: pgmap v499: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T12:01:10.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:10 smithi059 ceph-mon[30025]: pgmap v499: 129 pgs: 129 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-05T12:01:11.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:01:11.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:11 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:01:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:11 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:11 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:01:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:11 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:11 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:01:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:11 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:11 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:01:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:11 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:11 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:01:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:11 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:11 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:01:11.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:11 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:11.501 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 -- 172.21.15.59:0/689860955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 msgr2=0x7f4888110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 --2- 172.21.15.59:0/689860955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f4888110ca0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4880009ca0 tx=0x7f488002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 -- 172.21.15.59:0/689860955 shutdown_connections 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 --2- 172.21.15.59:0/689860955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f4888110ca0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 --2- 172.21.15.59:0/689860955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48881049a0 0x7f48881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 --2- 172.21.15.59:0/689860955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4888103ff0 0x7f48881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 -- 172.21.15.59:0/689860955 >> 172.21.15.59:0/689860955 conn(0x7f48880ffa90 msgr2=0x7f4888101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 -- 172.21.15.59:0/689860955 shutdown_connections 2024-09-05T12:01:12.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.057+0000 7f488ec00640 1 -- 172.21.15.59:0/689860955 wait complete. 2024-09-05T12:01:12.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.058+0000 7f488ec00640 1 Processor -- start 2024-09-05T12:01:12.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 -- start start 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 0x7f488819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f488819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4888114130 con 0x7f4888109120 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4888113fb0 con 0x7f48881049a0 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48881142b0 con 0x7f4888103ff0 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f4887e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f488cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f488819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f4887e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34498/0 (socket says 172.21.15.59:34498) 2024-09-05T12:01:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f4887e00640 1 -- 172.21.15.59:0/1372062491 learned_addr learned my addr 172.21.15.59:0/1372062491 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.059+0000 7f487fe00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 0x7f488819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 -- 172.21.15.59:0/1372062491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 msgr2=0x7f488819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 0x7f488819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 -- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 msgr2=0x7f488819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f488819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 -- 172.21.15.59:0/1372062491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488819f540 con 0x7f4888103ff0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f487fe00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 0x7f488819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4887e00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9f0 tx=0x7f487800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.060+0000 7f4885600640 1 -- 172.21.15.59:0/1372062491 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800cd80 con 0x7f4888103ff0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.061+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488819f830 con 0x7f4888103ff0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.061+0000 7f4885600640 1 -- 172.21.15.59:0/1372062491 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4878010910 con 0x7f4888103ff0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.061+0000 7f4885600640 1 -- 172.21.15.59:0/1372062491 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780188f0 con 0x7f4888103ff0 2024-09-05T12:01:12.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.061+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a7110 con 0x7f4888103ff0 2024-09-05T12:01:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 Processor -- start 2024-09-05T12:01:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 -- start start 2024-09-05T12:01:12.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb89f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4885600640 1 -- 172.21.15.59:0/1372062491 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4878010ac0 con 0x7f4888103ff0 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a18baf03e0 con 0x55a18bb8a4b0 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a18ba97d80 con 0x55a18bb89b90 2024-09-05T12:01:12.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4891681740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a18baec3a0 con 0x55a18bb94d70 2024-09-05T12:01:12.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.064+0000 7f4887e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f488cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb89f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f488cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb89f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:34316/0 (socket says 172.21.15.59:34316) 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34504/0 (socket says 172.21.15.59:34504) 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f488cc00640 1 -- 172.21.15.59:0/1345810479 learned_addr learned my addr 172.21.15.59:0/1345810479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 -- 172.21.15.59:0/1345810479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 msgr2=0x55a18bb89f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb89f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 -- 172.21.15.59:0/1345810479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 msgr2=0x55a18bb94830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb94830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 -- 172.21.15.59:0/1345810479 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18bab6ae0 con 0x55a18bb94d70 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.065+0000 7f487fe00640 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4870009f20 tx=0x7f48700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5bb30d3f21bf498 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f487d200640 1 -- 172.21.15.59:0/1345810479 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870004030 con 0x55a18bb94d70 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f487d200640 1 -- 172.21.15.59:0/1345810479 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4870005500 con 0x55a18bb94d70 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f487d200640 1 -- 172.21.15.59:0/1345810479 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870004db0 con 0x55a18bb94d70 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f4885600640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 0x7f486c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f4885600640 1 -- 172.21.15.59:0/1372062491 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f48780cf850 con 0x7f4888103ff0 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f488cc00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 0x7f486c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f4891681740 1 -- 172.21.15.59:0/1345810479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 msgr2=0x55a18bb97160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.066+0000 7f4891681740 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4870009f20 tx=0x7f48700118d0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 -- 172.21.15.59:0/1345810479 shutdown_connections 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb97160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 --2- 172.21.15.59:0/1345810479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb89f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 -- 172.21.15.59:0/1345810479 >> 172.21.15.59:0/1345810479 conn(0x55a18bb86120 msgr2=0x55a18bb86500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f488cc00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 0x7f486c0a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4880004af0 tx=0x7f4880004a80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 -- 172.21.15.59:0/1345810479 shutdown_connections 2024-09-05T12:01:12.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 -- 172.21.15.59:0/1345810479 wait complete. 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.067+0000 7f4891681740 1 -- start start 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 Processor -- start 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb92130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a18bb979e0 con 0x55a18bb8a4b0 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a18baab0f0 con 0x55a18bb89b90 2024-09-05T12:01:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a18bb97b60 con 0x55a18bb94d70 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.068+0000 7f4891681740 10 client.0 _ll_register_callbacks cb 0x55a18ba9c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f487fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb92130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f488cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f488cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb92670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34512/0 (socket says 172.21.15.59:34512) 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f488cc00640 1 -- 172.21.15.59:0/615444022 learned_addr learned my addr 172.21.15.59:0/615444022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f487fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb92130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48524/0 (socket says 172.21.15.59:48524) 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 -- 172.21.15.59:0/615444022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 msgr2=0x55a18bb92670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb92670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 -- 172.21.15.59:0/615444022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 msgr2=0x55a18bb92130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb92130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 -- 172.21.15.59:0/615444022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18b8613c0 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4887e00640 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb91bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48780ccf90 tx=0x7f487809bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98badeed461a615c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.069+0000 7f4875600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487809b3a0 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4870005500 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4878010430 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f487809bd50 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 10 client.25955 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a18bba3670 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4870005500 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4878018bb0 con 0x55a18bb89b90 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 10 client.25955 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:12.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.070+0000 7f4891681740 20 client.25955 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.071+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bba4a80 0x55a18bba6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.071+0000 7f4891681740 4 client.25955 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.071+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18bafd950 con 0x55a18bba4a80 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.071+0000 7f487fe00640 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bba4a80 0x55a18bba6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.072+0000 7f487fe00640 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bba4a80 0x55a18bba6ed0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae387cf0c445eeea server_cookie=17227d4bd616755a in_seq=0 out_seq=0 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.072+0000 7f4875600640 10 client.25955 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4875600640 1 -- 172.21.15.59:0/615444022 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a18bafd950 con 0x55a18bba4a80 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4875600640 10 client.25955 handle_command_reply: tid=0 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 client.25955 shutdown 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 msgr2=0x55a18bb91bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb91bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48780ccf90 tx=0x7f487809bb20 comp rx=0 tx=0).stop 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 shutdown_connections 2024-09-05T12:01:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb94d70 0x55a18bb92670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bba4a80 0x55a18bba6ed0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb92130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 --2- 172.21.15.59:0/615444022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb91bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 >> 172.21.15.59:0/615444022 conn(0x55a18bb86120 msgr2=0x55a18bb8be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4875600640 0 client.25955 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 shutdown_connections 2024-09-05T12:01:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.073+0000 7f4891681740 1 -- 172.21.15.59:0/615444022 wait complete. 2024-09-05T12:01:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.074+0000 7f4891681740 20 client.25955 trim_cache size 0 max 16384 2024-09-05T12:01:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 Processor -- start 2024-09-05T12:01:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 -- start start 2024-09-05T12:01:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb8bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb94d70 0x55a18bbacd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48881af180 con 0x55a18bb94d70 2024-09-05T12:01:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48881b1a00 con 0x55a18bb89b90 2024-09-05T12:01:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.094+0000 7f4891681740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48881af7f0 con 0x55a18bb8a4b0 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f487fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f488cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb94d70 0x55a18bbacd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb8bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f487fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34520/0 (socket says 172.21.15.59:34520) 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f487fe00640 1 -- 172.21.15.59:0/2201399395 learned_addr learned my addr 172.21.15.59:0/2201399395 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 -- 172.21.15.59:0/2201399395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 msgr2=0x55a18bbac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 -- 172.21.15.59:0/2201399395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb94d70 msgr2=0x55a18bbacd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb94d70 0x55a18bbacd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 -- 172.21.15.59:0/2201399395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18bba7520 con 0x55a18bb89b90 2024-09-05T12:01:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f487fe00640 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.095+0000 7f4887e00640 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb8bcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48780981b0 tx=0x7f487809bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaee74d6eeba87ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4877e00640 1 -- 172.21.15.59:0/2201399395 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780b8030 con 0x55a18bb89b90 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4877e00640 1 -- 172.21.15.59:0/2201399395 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48780a4910 con 0x55a18bb89b90 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4877e00640 1 -- 172.21.15.59:0/2201399395 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487809b1a0 con 0x55a18bb89b90 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 -- 172.21.15.59:0/2201399395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 msgr2=0x55a18bb8bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb8bcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48780981b0 tx=0x7f487809bc50 comp rx=0 tx=0).stop 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 -- 172.21.15.59:0/2201399395 shutdown_connections 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb94d70 0x55a18bbacd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb8a4b0 0x55a18bbac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.096+0000 7f4891681740 1 --2- 172.21.15.59:0/2201399395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb89b90 0x55a18bb8bcf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.097+0000 7f4891681740 1 -- 172.21.15.59:0/2201399395 >> 172.21.15.59:0/2201399395 conn(0x55a18bb86120 msgr2=0x55a18bb8b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.097+0000 7f4891681740 1 -- 172.21.15.59:0/2201399395 shutdown_connections 2024-09-05T12:01:12.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.097+0000 7f4891681740 1 -- 172.21.15.59:0/2201399395 wait complete. 2024-09-05T12:01:12.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.097+0000 7f4891681740 1 -- start start 2024-09-05T12:01:12.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.097+0000 7f4891681740 1 Processor -- start 2024-09-05T12:01:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb94d70 0x55a18bb99720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48881afec0 con 0x55a18bb8a4b0 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48881b1330 con 0x55a18bb94d70 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48881b0590 con 0x55a18bb89b90 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4891681740 10 client.0 _ll_register_callbacks cb 0x7f4878009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f487fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4887e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4887e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34524/0 (socket says 172.21.15.59:34524) 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f4887e00640 1 -- 172.21.15.59:0/1824693168 learned_addr learned my addr 172.21.15.59:0/1824693168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.098+0000 7f488cc00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb94d70 0x55a18bb99720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 -- 172.21.15.59:0/1824693168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 msgr2=0x55a18bb98ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 -- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb94d70 msgr2=0x55a18bb99720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb94d70 0x55a18bb99720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 -- 172.21.15.59:0/1824693168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18bc0e930 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f4887e00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f487fe00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb991e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4870000c00 tx=0x7f48700049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880ebe655fb30f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f4875600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870011dd0 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4870004250 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.099+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a18baf7e60 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f487003a050 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 10 client.15942 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a18bc0fc20 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a18baf7e60 con 0x55a18bb8a4b0 2024-09-05T12:01:12.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f487000f540 con 0x55a18bb8a4b0 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 10 client.15942 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 20 client.15942 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bc10150 0x55a18bc12570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 4 client.15942 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.100+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a18ba740c0 con 0x55a18bc10150 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4887e00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bc10150 0x55a18bc12570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4887e00640 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bc10150 0x55a18bc12570 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05d19f79f36f0bb server_cookie=9b962c59d21d4b27 in_seq=0 out_seq=0 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4875600640 10 client.15942 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:12.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4875600640 1 -- 172.21.15.59:0/1824693168 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a18ba740c0 con 0x55a18bc10150 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4875600640 10 client.15942 handle_command_reply: tid=0 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4891681740 1 client.15942 shutdown 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 msgr2=0x55a18bb991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.101+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb991e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4870000c00 tx=0x7f48700049e0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 shutdown_connections 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a18bb94d70 0x55a18bb99720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a18bc10150 0x55a18bc12570 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a18bb8a4b0 0x55a18bb991e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 --2- 172.21.15.59:0/1824693168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18bb89b90 0x55a18bb98ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 >> 172.21.15.59:0/1824693168 conn(0x55a18bb86120 msgr2=0x55a18bb8be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4875600640 0 client.15942 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 shutdown_connections 2024-09-05T12:01:12.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 1 -- 172.21.15.59:0/1824693168 wait complete. 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f4891681740 20 client.15942 trim_cache size 0 max 16384 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "f7f18483-2536-49c9-8ad4-2a9e2eab2a52": { 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "f7f18483-2536-49c9-8ad4-2a9e2eab2a52", 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:12.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:12.113 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 msgr2=0x7f486c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 0x7f486c0a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4880004af0 tx=0x7f4880004a80 comp rx=0 tx=0).stop 2024-09-05T12:01:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 msgr2=0x7f488819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:12.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.102+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9f0 tx=0x7f487800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 shutdown_connections 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f486c0a2b80 0x7f486c0a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4888109120 0x7f488819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48881049a0 0x7f488819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 --2- 172.21.15.59:0/1372062491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4888103ff0 0x7f488819a590 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 >> 172.21.15.59:0/1372062491 conn(0x7f48880ffa90 msgr2=0x7f4888100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 shutdown_connections 2024-09-05T12:01:12.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:12.103+0000 7f488ec00640 1 -- 172.21.15.59:0/1372062491 wait complete. 2024-09-05T12:01:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:12 smithi059 ceph-mon[30025]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-09-05T12:01:12.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:12 smithi189 ceph-mon[34772]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-09-05T12:01:12.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:12 smithi089 ceph-mon[34499]: pgmap v500: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-09-05T12:01:13.114 INFO:tasks.cephfs.filesystem:scrub status for tag:f7f18483-2536-49c9-8ad4-2a9e2eab2a52 - {'path': '/', 'tag': 'f7f18483-2536-49c9-8ad4-2a9e2eab2a52', 'options': 'recursive,force'} 2024-09-05T12:01:14.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:14 smithi189 ceph-mon[34772]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.6 KiB/s wr, 3 op/s 2024-09-05T12:01:14.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:14 smithi059 ceph-mon[30025]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.6 KiB/s wr, 3 op/s 2024-09-05T12:01:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:14 smithi089 ceph-mon[34499]: pgmap v501: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.6 KiB/s wr, 3 op/s 2024-09-05T12:01:15.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:15 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:01:15.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:15 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:01:15.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:15 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:01:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:16 smithi089 ceph-mon[34499]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-09-05T12:01:16.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:16 smithi059 ceph-mon[30025]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-09-05T12:01:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:16 smithi189 ceph-mon[34772]: pgmap v502: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 4 op/s 2024-09-05T12:01:18.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:18 smithi089 ceph-mon[34499]: pgmap v503: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:18 smithi059 ceph-mon[30025]: pgmap v503: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:18.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:18 smithi189 ceph-mon[34772]: pgmap v503: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:20 smithi089 ceph-mon[34499]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:20 smithi059 ceph-mon[30025]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:20 smithi189 ceph-mon[34772]: pgmap v504: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:22.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:22 smithi089 ceph-mon[34499]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:01:22.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:22 smithi059 ceph-mon[30025]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:01:22.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:22 smithi189 ceph-mon[34772]: pgmap v505: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:01:24.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:24 smithi089 ceph-mon[34499]: pgmap v506: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:24.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:24 smithi059 ceph-mon[30025]: pgmap v506: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:24 smithi189 ceph-mon[34772]: pgmap v506: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:26.392 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:26 smithi189 ceph-mon[34772]: pgmap v507: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:01:26.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:26 smithi089 ceph-mon[34499]: pgmap v507: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:01:26.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:26 smithi059 ceph-mon[30025]: pgmap v507: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:01:28.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:28 smithi089 ceph-mon[34499]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:28 smithi059 ceph-mon[30025]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:28.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:28 smithi189 ceph-mon[34772]: pgmap v508: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:30.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:30 smithi089 ceph-mon[34499]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:30.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:30 smithi059 ceph-mon[30025]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:30.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:30 smithi189 ceph-mon[34772]: pgmap v509: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:32.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:32 smithi089 ceph-mon[34499]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:32.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:32 smithi059 ceph-mon[30025]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:32.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:32 smithi189 ceph-mon[34772]: pgmap v510: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:34.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:34 smithi089 ceph-mon[34499]: pgmap v511: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:34.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:34 smithi059 ceph-mon[30025]: pgmap v511: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:34 smithi189 ceph-mon[34772]: pgmap v511: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:01:36.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:36 smithi089 ceph-mon[34499]: pgmap v512: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:36.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:36 smithi059 ceph-mon[30025]: pgmap v512: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:36.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:36 smithi189 ceph-mon[34772]: pgmap v512: 129 pgs: 129 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:38 smithi089 ceph-mon[34499]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:01:38.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:38 smithi059 ceph-mon[30025]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:01:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:38 smithi189 ceph-mon[34772]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:01:40.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:40 smithi089 ceph-mon[34499]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:01:40.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:40 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:01:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:40 smithi059 ceph-mon[30025]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:01:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:40 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:01:40.491 INFO:tasks.workunit.client.0.smithi059.stdout:64/600: open fd = 3 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:65/600: open fd = 3 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:66/600: open fd = 3 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.492 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:67/600: open fd = 3 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:68/600: open fd = 3 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:69/600: open fd = 3 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.493 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:70/600: open fd = 3 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:71/600: open fd = 3 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:72/600: open fd = 3 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.494 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:73/600: open fd = 3 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:74/600: open fd = 3 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:75/600: open fd = 3 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.495 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:76/600: open fd = 3 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:77/600: open fd = 3 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:78/600: open fd = 3 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.496 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:79/600: open fd = 3 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:80/600: open fd = 3 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:81/600: open fd = 3 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.497 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:82/600: open fd = 3 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:83/600: open fd = 3 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:84/600: open fd = 3 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:85/600: open fd = 3 2024-09-05T12:01:40.498 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:86/600: open fd = 3 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:87/600: open fd = 3 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:88/600: open fd = 3 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.499 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:89/600: open fd = 3 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:90/600: open fd = 3 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:91/600: open fd = 3 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.500 INFO:tasks.workunit.client.0.smithi059.stdout:92/600: open fd = 3 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:93/600: open fd = 3 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:94/600: open fd = 3 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.501 INFO:tasks.workunit.client.0.smithi059.stdout:95/600: open fd = 3 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:96/600: open fd = 3 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:97/600: open fd = 3 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.502 INFO:tasks.workunit.client.0.smithi059.stdout:98/600: open fd = 3 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:99/600: open fd = 3 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:100/600: open fd = 3 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:101/600: open fd = 3 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.503 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:102/600: open fd = 3 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:103/600: open fd = 3 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:104/600: open fd = 3 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.504 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:105/600: open fd = 3 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:106/600: open fd = 3 2024-09-05T12:01:40.505 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:107/600: open fd = 3 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:108/600: open fd = 3 2024-09-05T12:01:40.508 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:109/600: open fd = 3 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:110/600: open fd = 3 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:111/600: open fd = 3 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.509 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:112/600: open fd = 3 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:113/600: open fd = 3 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:114/600: open fd = 3 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.510 INFO:tasks.workunit.client.0.smithi059.stdout:115/600: open fd = 3 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:116/600: open fd = 3 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:117/600: open fd = 3 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:118/600: open fd = 3 2024-09-05T12:01:40.511 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:119/600: open fd = 3 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:120/600: open fd = 3 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.512 INFO:tasks.workunit.client.0.smithi059.stdout:121/600: open fd = 3 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:122/600: open fd = 3 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:123/600: open fd = 3 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:124/600: open fd = 3 2024-09-05T12:01:40.513 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:125/600: open fd = 3 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:126/600: open fd = 3 2024-09-05T12:01:40.514 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:01:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:40 smithi189 ceph-mon[34772]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:01:40.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:40 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:01:41.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:01:41.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:01:41.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:01:41.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:01:41.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:01:41.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:01:41.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:01:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:01:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:01:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:42 smithi189 ceph-mon[34772]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:43.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:01:43.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:42 smithi059 ceph-mon[30025]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:42 smithi089 ceph-mon[34499]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:01:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 -- 172.21.15.59:0/2736811523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144109120 msgr2=0x7f7144110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 --2- 172.21.15.59:0/2736811523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144109120 0x7f7144110ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ca0 tx=0x7f713c02a8b0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 -- 172.21.15.59:0/2736811523 shutdown_connections 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 --2- 172.21.15.59:0/2736811523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144109120 0x7f7144110ca0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 --2- 172.21.15.59:0/2736811523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71441049a0 0x7f71441089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 --2- 172.21.15.59:0/2736811523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7144103ff0 0x7f71441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 -- 172.21.15.59:0/2736811523 >> 172.21.15.59:0/2736811523 conn(0x7f71440ffa90 msgr2=0x7f7144101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 -- 172.21.15.59:0/2736811523 shutdown_connections 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.067+0000 7f7149400640 1 -- 172.21.15.59:0/2736811523 wait complete. 2024-09-05T12:01:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.068+0000 7f7149400640 1 Processor -- start 2024-09-05T12:01:44.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.068+0000 7f7149400640 1 -- start start 2024-09-05T12:01:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71441049a0 0x7f714419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7144109120 0x7f714419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7144114180 con 0x7f7144103ff0 2024-09-05T12:01:44.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7144114000 con 0x7f71441049a0 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7149400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7144114300 con 0x7f7144109120 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43256/0 (socket says 172.21.15.59:43256) 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7142800640 1 -- 172.21.15.59:0/2292312883 learned_addr learned my addr 172.21.15.59:0/2292312883 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7139e00640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71441049a0 0x7f714419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.069+0000 7f7143200640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7144109120 0x7f714419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 -- 172.21.15.59:0/2292312883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7144109120 msgr2=0x7f714419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7144109120 0x7f714419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 -- 172.21.15.59:0/2292312883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71441049a0 msgr2=0x7f714419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71441049a0 0x7f714419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 -- 172.21.15.59:0/2292312883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714419f570 con 0x7f7144103ff0 2024-09-05T12:01:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7142800640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9f0 tx=0x7f713400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f713be00640 1 -- 172.21.15.59:0/2292312883 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713400cd80 con 0x7f7144103ff0 2024-09-05T12:01:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f713be00640 1 -- 172.21.15.59:0/2292312883 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7134010910 con 0x7f7144103ff0 2024-09-05T12:01:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f713be00640 1 -- 172.21.15.59:0/2292312883 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134018990 con 0x7f7144103ff0 2024-09-05T12:01:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.070+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714419f860 con 0x7f7144103ff0 2024-09-05T12:01:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.071+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a7140 con 0x7f7144103ff0 2024-09-05T12:01:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.073+0000 7f713be00640 1 -- 172.21.15.59:0/2292312883 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7134010ac0 con 0x7f7144103ff0 2024-09-05T12:01:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 Processor -- start 2024-09-05T12:01:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 -- start start 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x562164090160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562163fe93e0 con 0x5621640834b0 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562163f90d80 con 0x562164082b90 2024-09-05T12:01:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.074+0000 7f714bfe3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562163fe53a0 con 0x56216408dd70 2024-09-05T12:01:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:41482/0 (socket says 172.21.15.59:41482) 2024-09-05T12:01:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 -- 172.21.15.59:0/3928752511 learned_addr learned my addr 172.21.15.59:0/3928752511 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7139e00640 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x562164090160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 -- 172.21.15.59:0/3928752511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 msgr2=0x562164090160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x562164090160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 -- 172.21.15.59:0/3928752511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 msgr2=0x56216408d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f7143200640 1 -- 172.21.15.59:0/3928752511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562163fafae0 con 0x562164082b90 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.075+0000 7f713be00640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 0x7f71280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f7143200640 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ca0 tx=0x7f713c030450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae61daf47b936ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f713be00640 1 -- 172.21.15.59:0/2292312883 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71340cf720 con 0x7f7144103ff0 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f7142800640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 0x7f71280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f7133400640 1 -- 172.21.15.59:0/3928752511 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c003970 con 0x562164082b90 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f7133400640 1 -- 172.21.15.59:0/3928752511 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c036030 con 0x562164082b90 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.076+0000 7f7133400640 1 -- 172.21.15.59:0/3928752511 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c004460 con 0x562164082b90 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- 172.21.15.59:0/3928752511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 msgr2=0x562164082f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f713c009ca0 tx=0x7f713c030450 comp rx=0 tx=0).stop 2024-09-05T12:01:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f7142800640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 0x7f71280a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f71340a4250 tx=0x7f71340d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- 172.21.15.59:0/3928752511 shutdown_connections 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x562164090160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3928752511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562164082b90 0x562164082f70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- 172.21.15.59:0/3928752511 >> 172.21.15.59:0/3928752511 conn(0x56216407f120 msgr2=0x56216407f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- 172.21.15.59:0/3928752511 shutdown_connections 2024-09-05T12:01:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- 172.21.15.59:0/3928752511 wait complete. 2024-09-05T12:01:44.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.077+0000 7f714bfe3740 1 -- start start 2024-09-05T12:01:44.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 Processor -- start 2024-09-05T12:01:44.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562164082b90 0x56216408acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x56216408b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562164090980 con 0x5621640834b0 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562163fa40f0 con 0x56216408dd70 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562164090b00 con 0x562164082b90 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f714bfe3740 10 client.0 _ll_register_callbacks cb 0x562163f959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f7139e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f7139e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43286/0 (socket says 172.21.15.59:43286) 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f7139e00640 1 -- 172.21.15.59:0/1881795972 learned_addr learned my addr 172.21.15.59:0/1881795972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.078+0000 7f7143200640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562164082b90 0x56216408acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7143200640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x56216408b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 -- 172.21.15.59:0/1881795972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562164082b90 msgr2=0x56216408acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562164082b90 0x56216408acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 -- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 msgr2=0x56216408b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x56216408b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 -- 172.21.15.59:0/1881795972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562163d5a3c0 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7139e00640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f712c00ba20 tx=0x7f712c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878f14c0e6fd1962 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7127e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c011440 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f712c0115e0 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.079+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c036030 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f712c0117e0 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 10 client.15954 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56216409c730 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c036030 con 0x5621640834b0 2024-09-05T12:01:44.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f712c01ba20 con 0x5621640834b0 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 10 client.15954 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 20 client.15954 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56216409dc90 0x5621640a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 4 client.15954 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.080+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562163ff6950 con 0x56216409dc90 2024-09-05T12:01:44.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.081+0000 7f7143200640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56216409dc90 0x5621640a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.081+0000 7f7143200640 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56216409dc90 0x5621640a00b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee411d8230efd41a server_cookie=480d155385b47d7 in_seq=0 out_seq=0 2024-09-05T12:01:44.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.081+0000 7f7127e00640 10 client.15954 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:44.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.082+0000 7f7127e00640 1 -- 172.21.15.59:0/1881795972 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562163ff6950 con 0x56216409dc90 2024-09-05T12:01:44.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.082+0000 7f7127e00640 10 client.15954 handle_command_reply: tid=0 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.082+0000 7f714bfe3740 1 client.15954 shutdown 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 msgr2=0x56216408b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f712c00ba20 tx=0x7f712c00bee0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 shutdown_connections 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x56216408b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56216409dc90 0x5621640a00b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5621640834b0 0x56216408b1f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 --2- 172.21.15.59:0/1881795972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562164082b90 0x56216408acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f7127e00640 0 client.15954 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 >> 172.21.15.59:0/1881795972 conn(0x56216407f120 msgr2=0x562164084e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 shutdown_connections 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 1 -- 172.21.15.59:0/1881795972 wait complete. 2024-09-05T12:01:44.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.083+0000 7f714bfe3740 20 client.15954 trim_cache size 0 max 16384 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.101+0000 7f714bfe3740 1 Processor -- start 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.101+0000 7f714bfe3740 1 -- start start 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.101+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 0x5621640a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x5621640a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f714bfe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71441aeff0 con 0x562164082b90 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f714bfe3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b1870 con 0x5621640834b0 2024-09-05T12:01:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f714bfe3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71441af660 con 0x56216408dd70 2024-09-05T12:01:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43302/0 (socket says 172.21.15.59:43302) 2024-09-05T12:01:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 -- 172.21.15.59:0/466358470 learned_addr learned my addr 172.21.15.59:0/466358470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7139e00640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 0x5621640a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x5621640a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 -- 172.21.15.59:0/466358470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 msgr2=0x5621640a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x5621640a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 -- 172.21.15.59:0/466358470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 msgr2=0x5621640a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 0x5621640a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7143200640 1 -- 172.21.15.59:0/466358470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621640832e0 con 0x562164082b90 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.102+0000 7f7139e00640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 0x5621640a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.103+0000 7f7143200640 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f713c045000 tx=0x7f713c030450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5c75942ce81beb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.103+0000 7f7132a00640 1 -- 172.21.15.59:0/466358470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c009540 con 0x562164082b90 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.103+0000 7f7132a00640 1 -- 172.21.15.59:0/466358470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c036070 con 0x562164082b90 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.103+0000 7f7132a00640 1 -- 172.21.15.59:0/466358470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c004320 con 0x562164082b90 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- 172.21.15.59:0/466358470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 msgr2=0x562164084cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f713c045000 tx=0x7f713c030450 comp rx=0 tx=0).stop 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- 172.21.15.59:0/466358470 shutdown_connections 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56216408dd70 0x5621640a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5621640834b0 0x5621640a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 --2- 172.21.15.59:0/466358470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164084cf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- 172.21.15.59:0/466358470 >> 172.21.15.59:0/466358470 conn(0x56216407f120 msgr2=0x5621640845e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- 172.21.15.59:0/466358470 shutdown_connections 2024-09-05T12:01:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- 172.21.15.59:0/466358470 wait complete. 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 -- start start 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.104+0000 7f714bfe3740 1 Processor -- start 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 0x562164092360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x562164107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71441afd30 con 0x562164082b90 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b11a0 con 0x56216408dd70 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b0400 con 0x5621640834b0 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f714bfe3740 10 client.0 _ll_register_callbacks cb 0x7f712c0044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f7143200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f7143200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43304/0 (socket says 172.21.15.59:43304) 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f7143200640 1 -- 172.21.15.59:0/3063770944 learned_addr learned my addr 172.21.15.59:0/3063770944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f7139e00640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 0x562164092360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.105+0000 7f7143200640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x562164107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 -- 172.21.15.59:0/3063770944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 msgr2=0x562164092360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 0x562164092360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 -- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 msgr2=0x562164107910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x562164107910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 -- 172.21.15.59:0/3063770944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562164092c40 con 0x562164082b90 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7139e00640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 0x562164092360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7143200640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f713c009410 tx=0x7f713c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9cddb660cd2a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7127e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:44.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c003b80 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f713c004b40 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c036070 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f713c004d40 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.106+0000 7f714bfe3740 10 client.15960 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562164108f70 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c036070 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f713c043070 con 0x562164082b90 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 10 client.15960 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 20 client.15960 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5621641091a0 0x56216410b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 4 client.15960 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:01:44.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.107+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562163ff0e60 con 0x5621641091a0 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f7139e00640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5621641091a0 0x56216410b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f7139e00640 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5621641091a0 0x56216410b590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7eb5fa3fc08936 server_cookie=155c76b8fd7b540a in_seq=0 out_seq=0 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f7127e00640 10 client.15960 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f7127e00640 1 -- 172.21.15.59:0/3063770944 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562163ff0e60 con 0x5621641091a0 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f7127e00640 10 client.15960 handle_command_reply: tid=0 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f714bfe3740 1 client.15960 shutdown 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.108+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 msgr2=0x562164091e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f713c009410 tx=0x7f713c003e30 comp rx=0 tx=0).stop 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 shutdown_connections 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56216408dd70 0x562164107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621640834b0 0x562164092360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5621641091a0 0x56216410b590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 --2- 172.21.15.59:0/3063770944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562164082b90 0x562164091e20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 >> 172.21.15.59:0/3063770944 conn(0x56216407f120 msgr2=0x562164084e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f7127e00640 0 client.15960 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 shutdown_connections 2024-09-05T12:01:44.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 1 -- 172.21.15.59:0/3063770944 wait complete. 2024-09-05T12:01:44.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.109+0000 7f714bfe3740 20 client.15960 trim_cache size 0 max 16384 2024-09-05T12:01:44.115 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:01:44.115 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:01:44.115 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 msgr2=0x7f71280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 0x7f71280a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f71340a4250 tx=0x7f71340d5530 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 msgr2=0x7f714419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f713400e9f0 tx=0x7f713400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 shutdown_connections 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71280a2c00 0x7f71280a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7144109120 0x7f714419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71441049a0 0x7f714419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 --2- 172.21.15.59:0/2292312883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7144103ff0 0x7f714419a5c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 >> 172.21.15.59:0/2292312883 conn(0x7f71440ffa90 msgr2=0x7f71441004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 shutdown_connections 2024-09-05T12:01:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:44.110+0000 7f7149400640 1 -- 172.21.15.59:0/2292312883 wait complete. 2024-09-05T12:01:44.892 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:44 smithi059 ceph-mon[30025]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:01:44.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:01:44.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:01:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:44 smithi189 ceph-mon[34772]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:01:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:44 smithi089 ceph-mon[34499]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:01:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 -- 172.21.15.59:0/4251245243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93981049a0 msgr2=0x7f93981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 --2- 172.21.15.59:0/4251245243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93981049a0 0x7f93981089f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9380009900 tx=0x7f938002a410 comp rx=0 tx=0).stop 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 -- 172.21.15.59:0/4251245243 shutdown_connections 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 --2- 172.21.15.59:0/4251245243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9398109120 0x7f9398110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 --2- 172.21.15.59:0/4251245243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93981049a0 0x7f93981089f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 --2- 172.21.15.59:0/4251245243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398103ff0 0x7f93981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 -- 172.21.15.59:0/4251245243 >> 172.21.15.59:0/4251245243 conn(0x7f93980ffa90 msgr2=0x7f9398101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 -- 172.21.15.59:0/4251245243 shutdown_connections 2024-09-05T12:01:45.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.857+0000 7f939e200640 1 -- 172.21.15.59:0/4251245243 wait complete. 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.858+0000 7f939e200640 1 Processor -- start 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.858+0000 7f939e200640 1 -- start start 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 0x7f939819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398109120 0x7f939819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9398114190 con 0x7f9398103ff0 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9398114010 con 0x7f9398109120 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f939e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9398114310 con 0x7f93981049a0 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f9397400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f938fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 0x7f939819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f9397400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43316/0 (socket says 172.21.15.59:43316) 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f9397400640 1 -- 172.21.15.59:0/3352828637 learned_addr learned my addr 172.21.15.59:0/3352828637 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.859+0000 7f9397e00640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398109120 0x7f939819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 -- 172.21.15.59:0/3352828637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 msgr2=0x7f939819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 0x7f939819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 -- 172.21.15.59:0/3352828637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398109120 msgr2=0x7f939819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398109120 0x7f939819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 -- 172.21.15.59:0/3352828637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939819f5a0 con 0x7f9398103ff0 2024-09-05T12:01:45.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f938fe00640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 0x7f939819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:45.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9397400640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f938800e9f0 tx=0x7f938800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:45.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800cd80 con 0x7f9398103ff0 2024-09-05T12:01:45.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388010910 con 0x7f9398103ff0 2024-09-05T12:01:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939819f890 con 0x7f9398103ff0 2024-09-05T12:01:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388018990 con 0x7f9398103ff0 2024-09-05T12:01:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.860+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a7170 con 0x7f9398103ff0 2024-09-05T12:01:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.862+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93981043d0 con 0x7f9398103ff0 2024-09-05T12:01:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.863+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9388010ac0 con 0x7f9398103ff0 2024-09-05T12:01:45.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.866+0000 7f9394c00640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 0x7f937c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:45.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.866+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f93880cf7c0 con 0x7f9398103ff0 2024-09-05T12:01:45.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.866+0000 7f938fe00640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 0x7f937c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:45.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.867+0000 7f938fe00640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 0x7f937c0a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9380000c00 tx=0x7f9380031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:45.870+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f938809bc60 con 0x7f9398103ff0 2024-09-05T12:01:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.093+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93981a73e0 con 0x7f9398103ff0 2024-09-05T12:01:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.094+0000 7f9394c00640 1 -- 172.21.15.59:0/3352828637 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f938809ff60 con 0x7f9398103ff0 2024-09-05T12:01:46.095 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:01:46.096 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:01:46.096 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 msgr2=0x7f937c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 0x7f937c0a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9380000c00 tx=0x7f9380031510 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 msgr2=0x7f939819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f938800e9f0 tx=0x7f938800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 shutdown_connections 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f937c0a2b80 0x7f937c0a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9398109120 0x7f939819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93981049a0 0x7f939819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 --2- 172.21.15.59:0/3352828637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9398103ff0 0x7f939819a5f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 >> 172.21.15.59:0/3352828637 conn(0x7f93980ffa90 msgr2=0x7f9398101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 shutdown_connections 2024-09-05T12:01:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:46.096+0000 7f939e200640 1 -- 172.21.15.59:0/3352828637 wait complete. 2024-09-05T12:01:46.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:01:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:46 smithi189 ceph-mon[34772]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3352828637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:46 smithi089 ceph-mon[34499]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:47.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3352828637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:46 smithi059 ceph-mon[30025]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3352828637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:01:47.313 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 -- 172.21.15.59:0/2300552140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78076720 msgr2=0x7f0e78076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/2300552140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78076720 0x7f0e78076b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009900 tx=0x7f0e6c02a410 comp rx=0 tx=0).stop 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 -- 172.21.15.59:0/2300552140 shutdown_connections 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/2300552140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78112870 0x7f0e78114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/2300552140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 0x7f0e78112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/2300552140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78076720 0x7f0e78076b00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 -- 172.21.15.59:0/2300552140 >> 172.21.15.59:0/2300552140 conn(0x7f0e780ff8d0 msgr2=0x7f0e78101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 -- 172.21.15.59:0/2300552140 shutdown_connections 2024-09-05T12:01:47.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.913+0000 7f0e7ec00640 1 -- 172.21.15.59:0/2300552140 wait complete. 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.914+0000 7f0e7ec00640 1 Processor -- start 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.914+0000 7f0e7ec00640 1 -- start start 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 0x7f0e781a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 0x7f0e781a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78118400 con 0x7f0e78112870 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78118280 con 0x7f0e78077040 2024-09-05T12:01:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78118580 con 0x7f0e78076720 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 0x7f0e781a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43334/0 (socket says 172.21.15.59:43334) 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7e200640 1 -- 172.21.15.59:0/3515631212 learned_addr learned my addr 172.21.15.59:0/3515631212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 0x7f0e781a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 -- 172.21.15.59:0/3515631212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 msgr2=0x7f0e781a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 0x7f0e781a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 -- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 msgr2=0x7f0e7819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7d800640 1 -- 172.21.15.59:0/3515631212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7819efc0 con 0x7f0e78076720 2024-09-05T12:01:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.915+0000 7f0e7e200640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e7d800640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 0x7f0e781a4680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c005940 tx=0x7f0e6c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e76000640 1 -- 172.21.15.59:0/3515631212 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03f030 con 0x7f0e78076720 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e76000640 1 -- 172.21.15.59:0/3515631212 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c0044c0 con 0x7f0e78076720 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7819f250 con 0x7f0e78076720 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e76000640 1 -- 172.21.15.59:0/3515631212 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c035070 con 0x7f0e78076720 2024-09-05T12:01:47.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.916+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e781ab430 con 0x7f0e78076720 2024-09-05T12:01:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.919+0000 7f0e82e65740 1 Processor -- start 2024-09-05T12:01:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.919+0000 7f0e76000640 1 -- 172.21.15.59:0/3515631212 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e6c042030 con 0x7f0e78076720 2024-09-05T12:01:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.919+0000 7f0e82e65740 1 -- start start 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e66aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e66d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55872e5c63e0 con 0x55872e65fe40 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55872e56dd80 con 0x55872e660760 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e82e65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55872e5c23a0 con 0x55872e66b020 2024-09-05T12:01:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43340/0 (socket says 172.21.15.59:43340) 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e7e200640 1 -- 172.21.15.59:0/4173246483 learned_addr learned my addr 172.21.15.59:0/4173246483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e7d800640 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e66aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.920+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e66d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 -- 172.21.15.59:0/4173246483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 msgr2=0x55872e66d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e66d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 -- 172.21.15.59:0/4173246483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 msgr2=0x55872e66aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e66aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 -- 172.21.15.59:0/4173246483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872e58cae0 con 0x55872e65fe40 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e7e200640 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e700117f0 tx=0x7f0e70011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee672276b076606e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e6aa00640 1 -- 172.21.15.59:0/4173246483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700112f0 con 0x55872e65fe40 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e6aa00640 1 -- 172.21.15.59:0/4173246483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7000fe80 con 0x55872e65fe40 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e76000640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 0x7f0e600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.921+0000 7f0e6aa00640 1 -- 172.21.15.59:0/4173246483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700193e0 con 0x55872e65fe40 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e76000640 1 -- 172.21.15.59:0/3515631212 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0e6c0ef730 con 0x7f0e78076720 2024-09-05T12:01:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e7d800640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 0x7f0e600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 -- 172.21.15.59:0/4173246483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 msgr2=0x55872e660220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e700117f0 tx=0x7f0e70011cb0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 -- 172.21.15.59:0/4173246483 shutdown_connections 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e66d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e66aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 --2- 172.21.15.59:0/4173246483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e660220 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 -- 172.21.15.59:0/4173246483 >> 172.21.15.59:0/4173246483 conn(0x55872e65c3d0 msgr2=0x55872e65c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e7d800640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 0x7f0e600a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02a930 tx=0x7f0e6c0f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.922+0000 7f0e82e65740 1 -- 172.21.15.59:0/4173246483 shutdown_connections 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.923+0000 7f0e82e65740 1 -- 172.21.15.59:0/4173246483 wait complete. 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.923+0000 7f0e82e65740 1 -- start start 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.923+0000 7f0e82e65740 1 Processor -- start 2024-09-05T12:01:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e660760 0x55872e668330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e668870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55872e66dc40 con 0x55872e660760 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55872e5810f0 con 0x55872e65fe40 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55872e66ddc0 con 0x55872e66b020 2024-09-05T12:01:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e82e65740 10 client.0 _ll_register_callbacks cb 0x55872e568bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e660760 0x55872e668330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:41540/0 (socket says 172.21.15.59:41540) 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 -- 172.21.15.59:0/208879294 learned_addr learned my addr 172.21.15.59:0/208879294 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e668870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 -- 172.21.15.59:0/208879294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 msgr2=0x55872e668870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e668870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 -- 172.21.15.59:0/208879294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e660760 msgr2=0x55872e668330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e660760 0x55872e668330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.924+0000 7f0e7e200640 1 -- 172.21.15.59:0/208879294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872e3373c0 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e7e200640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e700094b0 tx=0x7f0e70010ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd4ad7e80449f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e5f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011470 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55872e5d3950 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0e70010640 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e70036050 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e82e65740 10 client.25997 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.925+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55872e679b90 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55872e5d3950 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0e70025040 con 0x55872e65fe40 2024-09-05T12:01:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 10 client.25997 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 20 client.25997 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e67af40 0x55872e67d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 4 client.25997 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.926+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55872e5cde60 con 0x55872e67af40 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.927+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e67af40 0x55872e67d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.927+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e67af40 0x55872e67d330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e64b5cc92263e27 server_cookie=f9e3c54b49d07cc9 in_seq=0 out_seq=0 2024-09-05T12:01:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.927+0000 7f0e5f400640 10 client.25997 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e5f400640 1 -- 172.21.15.59:0/208879294 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55872e5cde60 con 0x55872e67af40 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e5f400640 10 client.25997 handle_command_reply: tid=0 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 client.25997 shutdown 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 msgr2=0x55872e667df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e700094b0 tx=0x7f0e70010ba0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 shutdown_connections 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e66b020 0x55872e668870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e67af40 0x55872e67d330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e660760 0x55872e668330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 --2- 172.21.15.59:0/208879294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e65fe40 0x55872e667df0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e5f400640 0 client.25997 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 >> 172.21.15.59:0/208879294 conn(0x55872e65c3d0 msgr2=0x55872e662100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 shutdown_connections 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.928+0000 7f0e82e65740 1 -- 172.21.15.59:0/208879294 wait complete. 2024-09-05T12:01:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.929+0000 7f0e82e65740 20 client.25997 trim_cache size 0 max 16384 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.941+0000 7f0e82e65740 1 Processor -- start 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.941+0000 7f0e82e65740 1 -- start start 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e65fe40 0x55872e682af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e683030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b32f0 con 0x55872e66b020 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b5b70 con 0x55872e660760 2024-09-05T12:01:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e82e65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b3960 con 0x55872e65fe40 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43350/0 (socket says 172.21.15.59:43350) 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 -- 172.21.15.59:0/3718433286 learned_addr learned my addr 172.21.15.59:0/3718433286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e683030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e65fe40 0x55872e682af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 -- 172.21.15.59:0/3718433286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e65fe40 msgr2=0x55872e682af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e65fe40 0x55872e682af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 -- 172.21.15.59:0/3718433286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 msgr2=0x55872e683030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e683030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7e200640 1 -- 172.21.15.59:0/3718433286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872e65bc80 con 0x55872e66b020 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.942+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e683030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e7e200640 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009560 tx=0x7f0e70011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb3a174978eaba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e6a000640 1 -- 172.21.15.59:0/3718433286 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011300 con 0x55872e66b020 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e6a000640 1 -- 172.21.15.59:0/3718433286 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7002d070 con 0x55872e66b020 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e6a000640 1 -- 172.21.15.59:0/3718433286 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70010640 con 0x55872e66b020 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e82e65740 1 -- 172.21.15.59:0/3718433286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 msgr2=0x55872e667db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.943+0000 7f0e82e65740 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009560 tx=0x7f0e70011cb0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 -- 172.21.15.59:0/3718433286 shutdown_connections 2024-09-05T12:01:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e66b020 0x55872e667db0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e660760 0x55872e683030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 --2- 172.21.15.59:0/3718433286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e65fe40 0x55872e682af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 -- 172.21.15.59:0/3718433286 >> 172.21.15.59:0/3718433286 conn(0x55872e65c3d0 msgr2=0x55872e6620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 -- 172.21.15.59:0/3718433286 shutdown_connections 2024-09-05T12:01:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 -- 172.21.15.59:0/3718433286 wait complete. 2024-09-05T12:01:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 -- start start 2024-09-05T12:01:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.944+0000 7f0e82e65740 1 Processor -- start 2024-09-05T12:01:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 0x55872e664d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e66b020 0x55872e672010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b4030 con 0x55872e65fe40 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b54a0 con 0x55872e66b020 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b4700 con 0x55872e660760 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e82e65740 10 client.0 _ll_register_callbacks cb 0x7f0e700116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43354/0 (socket says 172.21.15.59:43354) 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 -- 172.21.15.59:0/1754725198 learned_addr learned my addr 172.21.15.59:0/1754725198 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 0x55872e664d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e66b020 0x55872e672010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 -- 172.21.15.59:0/1754725198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 msgr2=0x55872e664d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.945+0000 7f0e7e200640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 0x55872e664d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e7e200640 1 -- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e66b020 msgr2=0x55872e672010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e7e200640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e66b020 0x55872e672010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e7e200640 1 -- 172.21.15.59:0/1754725198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872e672650 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 0x55872e664d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e7e200640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e70002230 tx=0x7f0e70026860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88df5dffddd7d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e5f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7001f040 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7002d070 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0e70010640 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e7001ad20 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e82e65740 10 client.15984 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.946+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55872e673910 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7002d070 con 0x55872e65fe40 2024-09-05T12:01:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0e70018070 con 0x55872e65fe40 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 10 client.15984 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 20 client.15984 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e6e5c80 0x55872e6e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 4 client.15984 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55872e54a010 con 0x55872e6e5c80 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.947+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e6e5c80 0x55872e6e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e7ce00640 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e6e5c80 0x55872e6e80a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920a899e9b7a44a9 server_cookie=bbf7f156362dd481 in_seq=0 out_seq=0 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e5f400640 10 client.15984 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e5f400640 1 -- 172.21.15.59:0/1754725198 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55872e54a010 con 0x55872e6e5c80 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e5f400640 10 client.15984 handle_command_reply: tid=0 2024-09-05T12:01:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 client.15984 shutdown 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 msgr2=0x55872e664800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e70002230 tx=0x7f0e70026860 comp rx=0 tx=0).stop 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 shutdown_connections 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55872e66b020 0x55872e672010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55872e660760 0x55872e664d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55872e6e5c80 0x55872e6e80a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 --2- 172.21.15.59:0/1754725198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55872e65fe40 0x55872e664800 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 >> 172.21.15.59:0/1754725198 conn(0x55872e65c3d0 msgr2=0x55872e65ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.948+0000 7f0e5f400640 0 client.15984 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 shutdown_connections 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e82e65740 1 -- 172.21.15.59:0/1754725198 wait complete. 2024-09-05T12:01:47.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e82e65740 20 client.15984 trim_cache size 0 max 16384 2024-09-05T12:01:47.956 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 msgr2=0x7f0e600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 0x7f0e600a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02a930 tx=0x7f0e6c0f5530 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 msgr2=0x7f0e781a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 0x7f0e781a4680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c005940 tx=0x7f0e6c0040c0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.949+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 shutdown_connections 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0e600a2c00 0x7f0e600a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e78112870 0x7f0e7819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e78077040 0x7f0e781a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 --2- 172.21.15.59:0/3515631212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e78076720 0x7f0e781a4680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 >> 172.21.15.59:0/3515631212 conn(0x7f0e780ff8d0 msgr2=0x7f0e78101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 shutdown_connections 2024-09-05T12:01:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:47.950+0000 7f0e7ec00640 1 -- 172.21.15.59:0/3515631212 wait complete. 2024-09-05T12:01:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:48 smithi189 ceph-mon[34772]: pgmap v518: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:01:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:48 smithi089 ceph-mon[34499]: pgmap v518: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:01:49.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:48 smithi059 ceph-mon[30025]: pgmap v518: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:01:49.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:01:49.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:01:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 -- 172.21.15.59:0/1046754660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 msgr2=0x7f6c4c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 --2- 172.21.15.59:0/1046754660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c1043d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009860 tx=0x7f6c3c02a340 comp rx=0 tx=0).stop 2024-09-05T12:01:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 -- 172.21.15.59:0/1046754660 shutdown_connections 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 --2- 172.21.15.59:0/1046754660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c109120 0x7f6c4c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 --2- 172.21.15.59:0/1046754660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c1049a0 0x7f6c4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 --2- 172.21.15.59:0/1046754660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c1043d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 -- 172.21.15.59:0/1046754660 >> 172.21.15.59:0/1046754660 conn(0x7f6c4c0ffa90 msgr2=0x7f6c4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 -- 172.21.15.59:0/1046754660 shutdown_connections 2024-09-05T12:01:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.850+0000 7f6c51000640 1 -- 172.21.15.59:0/1046754660 wait complete. 2024-09-05T12:01:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.851+0000 7f6c51000640 1 Processor -- start 2024-09-05T12:01:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.851+0000 7f6c51000640 1 -- start start 2024-09-05T12:01:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c1049a0 0x7f6c4c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c109120 0x7f6c4c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c114150 con 0x7f6c4c103ff0 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113fd0 con 0x7f6c4c109120 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c51000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1142d0 con 0x7f6c4c1049a0 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60930/0 (socket says 172.21.15.59:60930) 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c49e00640 1 -- 172.21.15.59:0/3473000355 learned_addr learned my addr 172.21.15.59:0/3473000355 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c43e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c1049a0 0x7f6c4c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.852+0000 7f6c4a800640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c109120 0x7f6c4c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 -- 172.21.15.59:0/3473000355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c1049a0 msgr2=0x7f6c4c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c1049a0 0x7f6c4c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 -- 172.21.15.59:0/3473000355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c109120 msgr2=0x7f6c4c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c109120 0x7f6c4c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 -- 172.21.15.59:0/3473000355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c19f5a0 con 0x7f6c4c103ff0 2024-09-05T12:01:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c49e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c035040 tx=0x7f6c3c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c42a00640 1 -- 172.21.15.59:0/3473000355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c037040 con 0x7f6c4c103ff0 2024-09-05T12:01:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c42a00640 1 -- 172.21.15.59:0/3473000355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c040070 con 0x7f6c4c103ff0 2024-09-05T12:01:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c42a00640 1 -- 172.21.15.59:0/3473000355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0039b0 con 0x7f6c4c103ff0 2024-09-05T12:01:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.853+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c19f830 con 0x7f6c4c103ff0 2024-09-05T12:01:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.854+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c19fcc0 con 0x7f6c4c103ff0 2024-09-05T12:01:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.856+0000 7f6c539d5740 1 Processor -- start 2024-09-05T12:01:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 -- start start 2024-09-05T12:01:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c42a00640 1 -- 172.21.15.59:0/3473000355 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c3c0305e0 con 0x7f6c4c103ff0 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x563753840760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x563753843090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56375379c290 con 0x563753835ac0 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563753743b40 con 0x563753840ca0 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c539d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563753798250 con 0x5637538363e0 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60944/0 (socket says 172.21.15.59:60944) 2024-09-05T12:01:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c4a800640 1 -- 172.21.15.59:0/4277732354 learned_addr learned my addr 172.21.15.59:0/4277732354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c43e00640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x563753843090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.857+0000 7f6c49e00640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x563753840760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 -- 172.21.15.59:0/4277732354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 msgr2=0x563753840760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x563753840760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 -- 172.21.15.59:0/4277732354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 msgr2=0x563753843090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x563753843090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 -- 172.21.15.59:0/4277732354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375350d6a0 con 0x563753835ac0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c43e00640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x563753843090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.858+0000 7f6c4a800640 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009f50 tx=0x7f6c440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d777251242c4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c42a00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 0x7f6c300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c3b400640 1 -- 172.21.15.59:0/4277732354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44004d20 con 0x563753835ac0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c3b400640 1 -- 172.21.15.59:0/4277732354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44004050 con 0x563753835ac0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c3b400640 1 -- 172.21.15.59:0/4277732354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44005280 con 0x563753835ac0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c42a00640 1 -- 172.21.15.59:0/3473000355 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c3c0ef470 con 0x7f6c4c103ff0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c49e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 0x7f6c300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c539d5740 1 -- 172.21.15.59:0/4277732354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 msgr2=0x563753835ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.859+0000 7f6c539d5740 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009f50 tx=0x7f6c440118d0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c49e00640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 0x7f6c300a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0bb7f0 tx=0x7f6c3c0f2eb0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 -- 172.21.15.59:0/4277732354 shutdown_connections 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x563753843090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x563753840760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 --2- 172.21.15.59:0/4277732354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753835ea0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 -- 172.21.15.59:0/4277732354 >> 172.21.15.59:0/4277732354 conn(0x563753832050 msgr2=0x563753832430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 -- 172.21.15.59:0/4277732354 shutdown_connections 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 -- 172.21.15.59:0/4277732354 wait complete. 2024-09-05T12:01:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 -- start start 2024-09-05T12:01:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.860+0000 7f6c539d5740 1 Processor -- start 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x563753840650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x56375383dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563753843910 con 0x5637538363e0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563753756fa0 con 0x563753840ca0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563753843a90 con 0x563753835ac0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c539d5740 10 client.0 _ll_register_callbacks cb 0x5637534aaba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x563753840650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x563753840650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46518/0 (socket says 172.21.15.59:46518) 2024-09-05T12:01:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c4a800640 1 -- 172.21.15.59:0/2213913478 learned_addr learned my addr 172.21.15.59:0/2213913478 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60950/0 (socket says 172.21.15.59:60950) 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.861+0000 7f6c4a800640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x56375383dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 -- 172.21.15.59:0/2213913478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 msgr2=0x563753840650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x563753840650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 -- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 msgr2=0x56375383dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x56375383dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 -- 172.21.15.59:0/2213913478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375383e6d0 con 0x5637538363e0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c10aaf0 tx=0x7f6c34011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131cd7aec8b53ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.862+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3401e020 con 0x5637538363e0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c3400fbc0 con 0x5637538363e0 2024-09-05T12:01:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44004050 con 0x5637538363e0 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c34026050 con 0x5637538363e0 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c539d5740 10 client.15996 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56375384f920 con 0x5637538363e0 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44004050 con 0x5637538363e0 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.863+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c34019070 con 0x5637538363e0 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 10 client.15996 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 20 client.15996 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753850e30 0x563753853250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 4 client.15996 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637537a9800 con 0x563753850e30 2024-09-05T12:01:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.864+0000 7f6c4a800640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753850e30 0x563753853250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.865+0000 7f6c4a800640 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753850e30 0x563753853250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c1dcfd1e2108d server_cookie=b10d9305202ad13f in_seq=0 out_seq=0 2024-09-05T12:01:50.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.865+0000 7f6c2fe00640 10 client.15996 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.865+0000 7f6c2fe00640 1 -- 172.21.15.59:0/2213913478 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637537a9800 con 0x563753850e30 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.865+0000 7f6c2fe00640 10 client.15996 handle_command_reply: tid=0 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.865+0000 7f6c539d5740 1 client.15996 shutdown 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 msgr2=0x56375383d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c10aaf0 tx=0x7f6c34011b30 comp rx=0 tx=0).stop 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 shutdown_connections 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753840ca0 0x56375383dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753850e30 0x563753853250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x56375383d900 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2213913478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x563753840650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c2fe00640 0 client.15996 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 >> 172.21.15.59:0/2213913478 conn(0x563753832050 msgr2=0x5637538311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 shutdown_connections 2024-09-05T12:01:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 1 -- 172.21.15.59:0/2213913478 wait complete. 2024-09-05T12:01:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.866+0000 7f6c539d5740 20 client.15996 trim_cache size 0 max 16384 2024-09-05T12:01:50.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 Processor -- start 2024-09-05T12:01:50.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 -- start start 2024-09-05T12:01:50.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753858b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x56375383aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1af090 con 0x563753835ac0 2024-09-05T12:01:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b1910 con 0x563753832050 2024-09-05T12:01:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c539d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1af700 con 0x5637538363e0 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60958/0 (socket says 172.21.15.59:60958) 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c43e00640 1 -- 172.21.15.59:0/2791864862 learned_addr learned my addr 172.21.15.59:0/2791864862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c4a800640 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753858b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.885+0000 7f6c4a800640 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x56375383aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 -- 172.21.15.59:0/2791864862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 msgr2=0x56375383aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x56375383aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 -- 172.21.15.59:0/2791864862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 msgr2=0x563753858b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753858b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 -- 172.21.15.59:0/2791864862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375383e8a0 con 0x563753835ac0 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c43e00640 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009340 tx=0x7f6c3400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280494f5e98e20d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c3aa00640 1 -- 172.21.15.59:0/2791864862 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3401e040 con 0x563753835ac0 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c3aa00640 1 -- 172.21.15.59:0/2791864862 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3402f070 con 0x563753835ac0 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.886+0000 7f6c3aa00640 1 -- 172.21.15.59:0/2791864862 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34004a20 con 0x563753835ac0 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 -- 172.21.15.59:0/2791864862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 msgr2=0x563753859060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009340 tx=0x7f6c3400f7b0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 -- 172.21.15.59:0/2791864862 shutdown_connections 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637538363e0 0x56375383aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563753835ac0 0x563753859060 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 --2- 172.21.15.59:0/2791864862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753858b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 -- 172.21.15.59:0/2791864862 >> 172.21.15.59:0/2791864862 conn(0x563753839330 msgr2=0x56375383f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 -- 172.21.15.59:0/2791864862 shutdown_connections 2024-09-05T12:01:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.887+0000 7f6c539d5740 1 -- 172.21.15.59:0/2791864862 wait complete. 2024-09-05T12:01:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 -- start start 2024-09-05T12:01:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 Processor -- start 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x5637538495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x563753845f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1afdd0 con 0x5637538363e0 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b1240 con 0x563753832050 2024-09-05T12:01:50.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b04a0 con 0x563753835ac0 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.888+0000 7f6c539d5740 10 client.0 _ll_register_callbacks cb 0x7f6c34009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45082/0 (socket says 172.21.15.59:45082) 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 -- 172.21.15.59:0/3337362874 learned_addr learned my addr 172.21.15.59:0/3337362874 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x563753845f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c43e00640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x5637538495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 -- 172.21.15.59:0/3337362874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 msgr2=0x5637538495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x5637538495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 -- 172.21.15.59:0/3337362874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 msgr2=0x563753845f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x563753845f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 -- 172.21.15.59:0/3337362874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563753846760 con 0x563753832050 2024-09-05T12:01:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c4a800640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c44004990 tx=0x7f6c4402ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c3f4d512ef0fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.889+0000 7f6c2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44003e40 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3402f070 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c44004050 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c44044050 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 10 client.26018 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5637538bc830 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3402f070 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c44027cc0 con 0x563753832050 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 10 client.26018 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 20 client.26018 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.890+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753840ca0 0x5637538bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.891+0000 7f6c539d5740 4 client.26018 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.891+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637537a3d10 con 0x563753840ca0 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.891+0000 7f6c43e00640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753840ca0 0x5637538bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.891+0000 7f6c43e00640 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753840ca0 0x5637538bee10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d102b6a2ce678f0 server_cookie=7482ed642d548aa9 in_seq=0 out_seq=0 2024-09-05T12:01:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.891+0000 7f6c2fe00640 10 client.26018 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:50.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.892+0000 7f6c2fe00640 1 -- 172.21.15.59:0/3337362874 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637537a3d10 con 0x563753840ca0 2024-09-05T12:01:50.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.892+0000 7f6c2fe00640 10 client.26018 handle_command_reply: tid=0 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.892+0000 7f6c539d5740 1 client.26018 shutdown 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.892+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 msgr2=0x563753849080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c44004990 tx=0x7f6c4402ee80 comp rx=0 tx=0).stop 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 shutdown_connections 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563753840ca0 0x5637538bee10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637538363e0 0x563753845f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563753835ac0 0x5637538495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 --2- 172.21.15.59:0/3337362874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563753832050 0x563753849080 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 >> 172.21.15.59:0/3337362874 conn(0x563753839330 msgr2=0x5637538309a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c2fe00640 0 client.26018 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 shutdown_connections 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 1 -- 172.21.15.59:0/3337362874 wait complete. 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.893+0000 7f6c539d5740 20 client.26018 trim_cache size 0 max 16384 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:01:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "d56e4a19-23f3-4969-aa89-ad1ae6b55084", 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 msgr2=0x7f6c300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 0x7f6c300a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0bb7f0 tx=0x7f6c3c0f2eb0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 msgr2=0x7f6c4c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:50.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c035040 tx=0x7f6c3c033b60 comp rx=0 tx=0).stop 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 shutdown_connections 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c300a2b80 0x7f6c300a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4c109120 0x7f6c4c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c4c1049a0 0x7f6c4c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 --2- 172.21.15.59:0/3473000355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c4c103ff0 0x7f6c4c19a5f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 >> 172.21.15.59:0/3473000355 conn(0x7f6c4c0ffa90 msgr2=0x7f6c4c100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 shutdown_connections 2024-09-05T12:01:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:50.894+0000 7f6c51000640 1 -- 172.21.15.59:0/3473000355 wait complete. 2024-09-05T12:01:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:50 smithi189 ceph-mon[34772]: pgmap v519: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:50 smithi089 ceph-mon[34499]: pgmap v519: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:50 smithi059 ceph-mon[30025]: pgmap v519: 129 pgs: 129 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:51.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:01:52.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:51 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:01:52.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:51 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:51 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:01:52.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:51 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.154 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:01:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:51 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:01:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:51 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:51 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:01:52.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:51 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:51 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:01:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:51 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:51 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:01:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:51 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 -- 172.21.15.59:0/928318943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc81049a0 msgr2=0x7f1fc81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 --2- 172.21.15.59:0/928318943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc81049a0 0x7f1fc81089f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009900 tx=0x7f1fb402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 -- 172.21.15.59:0/928318943 shutdown_connections 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 --2- 172.21.15.59:0/928318943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc8109120 0x7f1fc8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 --2- 172.21.15.59:0/928318943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc81049a0 0x7f1fc81089f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 --2- 172.21.15.59:0/928318943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 0x7f1fc81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 -- 172.21.15.59:0/928318943 >> 172.21.15.59:0/928318943 conn(0x7f1fc80ffa90 msgr2=0x7f1fc8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 -- 172.21.15.59:0/928318943 shutdown_connections 2024-09-05T12:01:52.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.712+0000 7f1fce000640 1 -- 172.21.15.59:0/928318943 wait complete. 2024-09-05T12:01:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 Processor -- start 2024-09-05T12:01:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 -- start start 2024-09-05T12:01:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 0x7f1fc819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc81049a0 0x7f1fc819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8114170 con 0x7f1fc8109120 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8113ff0 con 0x7f1fc81049a0 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.713+0000 7f1fce000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81142f0 con 0x7f1fc8103ff0 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60980/0 (socket says 172.21.15.59:60980) 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 -- 172.21.15.59:0/1329484732 learned_addr learned my addr 172.21.15.59:0/1329484732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc81049a0 0x7f1fc819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7400640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 0x7f1fc819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 -- 172.21.15.59:0/1329484732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 msgr2=0x7f1fc819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 0x7f1fc819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 -- 172.21.15.59:0/1329484732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc81049a0 msgr2=0x7f1fc819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc81049a0 0x7f1fc819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 -- 172.21.15.59:0/1329484732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc819f600 con 0x7f1fc8109120 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.714+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000eb10 tx=0x7f1fc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.715+0000 7f1fbfe00640 1 -- 172.21.15.59:0/1329484732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0004b20 con 0x7f1fc8109120 2024-09-05T12:01:52.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.715+0000 7f1fbfe00640 1 -- 172.21.15.59:0/1329484732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0004cc0 con 0x7f1fc8109120 2024-09-05T12:01:52.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.715+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc819f8f0 con 0x7f1fc8109120 2024-09-05T12:01:52.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.715+0000 7f1fbfe00640 1 -- 172.21.15.59:0/1329484732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0010760 con 0x7f1fc8109120 2024-09-05T12:01:52.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.715+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a71d0 con 0x7f1fc8109120 2024-09-05T12:01:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fbfe00640 1 -- 172.21.15.59:0/1329484732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1fc0005020 con 0x7f1fc8109120 2024-09-05T12:01:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 Processor -- start 2024-09-05T12:01:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 -- start start 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d4516af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d45178160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d450d13e0 con 0x559d45175d70 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559d45078d80 con 0x559d4516ab90 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.718+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d450cd3a0 con 0x559d4516b4b0 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46560/0 (socket says 172.21.15.59:46560) 2024-09-05T12:01:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 -- 172.21.15.59:0/2317608357 learned_addr learned my addr 172.21.15.59:0/2317608357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d45178160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d4516af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 -- 172.21.15.59:0/2317608357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 msgr2=0x559d4516af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d4516af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 -- 172.21.15.59:0/2317608357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 msgr2=0x559d45178160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d45178160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc7400640 1 -- 172.21.15.59:0/2317608357 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d45097ae0 con 0x559d4516b4b0 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.719+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d4516af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:01:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800bd10 tx=0x7f1fb80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e9ef461f136a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fb3e00640 1 -- 172.21.15.59:0/2317608357 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8004030 con 0x559d4516b4b0 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fb3e00640 1 -- 172.21.15.59:0/2317608357 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80041d0 con 0x559d4516b4b0 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fb3e00640 1 -- 172.21.15.59:0/2317608357 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80117e0 con 0x559d4516b4b0 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fbfe00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 0x7f1fac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fbfe00640 1 -- 172.21.15.59:0/1329484732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1fc00cf940 con 0x7f1fc8109120 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.720+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 0x7f1fac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2317608357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 msgr2=0x559d45175830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800bd10 tx=0x7f1fb80107a0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2317608357 shutdown_connections 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d45178160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d45175830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2317608357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d4516af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2317608357 >> 172.21.15.59:0/2317608357 conn(0x559d45167120 msgr2=0x559d45167500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2317608357 shutdown_connections 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 0x7f1fac0a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4005bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.721+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2317608357 wait complete. 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 -- start start 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 Processor -- start 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516b4b0 0x559d45173160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d451736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d451789e0 con 0x559d45175d70 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559d4508c0f0 con 0x559d4516b4b0 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d45178b60 con 0x559d4516ab90 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.722+0000 7f1fd0c25740 10 client.0 _ll_register_callbacks cb 0x559d4507d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516b4b0 0x559d45173160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46576/0 (socket says 172.21.15.59:46576) 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 -- 172.21.15.59:0/50786568 learned_addr learned my addr 172.21.15.59:0/50786568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d451736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 -- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516b4b0 msgr2=0x559d45173160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516b4b0 0x559d45173160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 -- 172.21.15.59:0/50786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 msgr2=0x559d451736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d451736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 -- 172.21.15.59:0/50786568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d44e423c0 con 0x559d4516ab90 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fc7400640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8010ca0 tx=0x7f1fb800b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23918cf2c3337d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.723+0000 7f1fb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:01:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8029060 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1fb80035f0 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb80041d0 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fb8030050 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fd0c25740 10 client.25951 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.724+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559d451845d0 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80041d0 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1fb80046c0 con 0x559d4516ab90 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 10 client.25951 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:52.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 20 client.25951 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451859d0 0x559d45187dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 4 client.25951 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d450de950 con 0x559d451859d0 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.725+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451859d0 0x559d45187dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.726+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451859d0 0x559d45187dc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f3c9a926ac2614 server_cookie=36cafc7335dccf64 in_seq=0 out_seq=0 2024-09-05T12:01:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.726+0000 7f1fb0c00640 10 client.25951 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fb0c00640 1 -- 172.21.15.59:0/50786568 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d450de950 con 0x559d451859d0 2024-09-05T12:01:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fb0c00640 10 client.25951 handle_command_reply: tid=0 2024-09-05T12:01:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 client.25951 shutdown 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 msgr2=0x559d45172c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8010ca0 tx=0x7f1fb800b950 comp rx=0 tx=0).stop 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 shutdown_connections 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451859d0 0x559d45187dc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d451736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516b4b0 0x559d45173160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/50786568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d45172c20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fb0c00640 0 client.25951 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 >> 172.21.15.59:0/50786568 conn(0x559d45167120 msgr2=0x559d4516ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.727+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 shutdown_connections 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.728+0000 7f1fd0c25740 1 -- 172.21.15.59:0/50786568 wait complete. 2024-09-05T12:01:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.728+0000 7f1fd0c25740 20 client.25951 trim_cache size 0 max 16384 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 Processor -- start 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 -- start start 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d4518d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d4518d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81af0c0 con 0x559d45175d70 2024-09-05T12:01:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81b1940 con 0x559d4516ab90 2024-09-05T12:01:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.746+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81af730 con 0x559d4516b4b0 2024-09-05T12:01:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45120/0 (socket says 172.21.15.59:45120) 2024-09-05T12:01:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 -- 172.21.15.59:0/1166487416 learned_addr learned my addr 172.21.15.59:0/1166487416 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d4518d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d4518d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 -- 172.21.15.59:0/1166487416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 msgr2=0x559d4518d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d4518d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 -- 172.21.15.59:0/1166487416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 msgr2=0x559d4518d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d4518d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 -- 172.21.15.59:0/1166487416 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d45166f50 con 0x559d4516ab90 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d4518d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:52.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.747+0000 7f1fc7400640 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ba40 tx=0x7f1fb8005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733c37ac1fe6da2a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.749+0000 7f1fb3400640 1 -- 172.21.15.59:0/1166487416 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8029070 con 0x559d4516ab90 2024-09-05T12:01:52.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fb3400640 1 -- 172.21.15.59:0/1166487416 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8003ef0 con 0x559d4516ab90 2024-09-05T12:01:52.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fb3400640 1 -- 172.21.15.59:0/1166487416 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80053e0 con 0x559d4516ab90 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 -- 172.21.15.59:0/1166487416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 msgr2=0x559d45171e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ba40 tx=0x7f1fb8005cc0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 -- 172.21.15.59:0/1166487416 shutdown_connections 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d45175d70 0x559d4518d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516b4b0 0x559d4518d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/1166487416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d4516ab90 0x559d45171e90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 -- 172.21.15.59:0/1166487416 >> 172.21.15.59:0/1166487416 conn(0x559d45167120 msgr2=0x559d45173400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.750+0000 7f1fd0c25740 1 -- 172.21.15.59:0/1166487416 shutdown_connections 2024-09-05T12:01:52.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.751+0000 7f1fd0c25740 1 -- 172.21.15.59:0/1166487416 wait complete. 2024-09-05T12:01:52.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.751+0000 7f1fd0c25740 1 -- start start 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.751+0000 7f1fd0c25740 1 Processor -- start 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.751+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.751+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 0x559d45179920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fd0c25740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 0x559d45179ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81afe00 con 0x559d4516b4b0 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81b1270 con 0x559d45175d70 2024-09-05T12:01:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fd0c25740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81b04d0 con 0x559d4516ab90 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fd0c25740 10 client.0 _ll_register_callbacks cb 0x7f1fb8003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:46590/0 (socket says 172.21.15.59:46590) 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 -- 172.21.15.59:0/2774379478 learned_addr learned my addr 172.21.15.59:0/2774379478 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 0x559d45179920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 0x559d45179ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 -- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 msgr2=0x559d45179ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 0x559d45179ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 -- 172.21.15.59:0/2774379478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 msgr2=0x559d45179920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 0x559d45179920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7400640 1 -- 172.21.15.59:0/2774379478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d45167db0 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc7e00640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 0x559d45179ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.752+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 0x559d45179920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fc7400640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80107e0 tx=0x7f1fb80042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc87148f9ea6e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8029040 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb8003ef0 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1fb8004590 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fb8004790 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fd0c25740 10 client.25957 fetch_fsmap learned FSMap version 21 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559d451f0270 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.753+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8003ef0 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1fb8011440 con 0x559d4516ab90 2024-09-05T12:01:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 10 client.25957 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:01:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:01:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 20 client.25957 populate_metadata read hostname 'smithi059' 2024-09-05T12:01:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451f08c0 0x559d451f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:01:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 4 client.25957 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:01:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d450d8e60 con 0x559d451f08c0 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.754+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451f08c0 0x559d451f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fc6a00640 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451f08c0 0x559d451f2cb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6092376cbcced50 server_cookie=e45ca243e9d32e63 in_seq=0 out_seq=0 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fb0c00640 10 client.25957 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fb0c00640 1 -- 172.21.15.59:0/2774379478 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559d450d8e60 con 0x559d451f08c0 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fb0c00640 10 client.25957 handle_command_reply: tid=0 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 client.25957 shutdown 2024-09-05T12:01:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 msgr2=0x559d4517e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80107e0 tx=0x7f1fb80042d0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 shutdown_connections 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d45175d70 0x559d45179ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559d451f08c0 0x559d451f2cb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d4516b4b0 0x559d45179920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 --2- 172.21.15.59:0/2774379478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d4516ab90 0x559d4517e5a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fb0c00640 0 client.25957 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 >> 172.21.15.59:0/2774379478 conn(0x559d45167120 msgr2=0x559d451659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 shutdown_connections 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 1 -- 172.21.15.59:0/2774379478 wait complete. 2024-09-05T12:01:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.755+0000 7f1fd0c25740 20 client.25957 trim_cache size 0 max 16384 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "d56e4a19-23f3-4969-aa89-ad1ae6b55084": { 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "d56e4a19-23f3-4969-aa89-ad1ae6b55084", 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:01:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 msgr2=0x7f1fac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 0x7f1fac0a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4000c00 tx=0x7f1fb4005bc0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 msgr2=0x7f1fc819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000eb10 tx=0x7f1fc000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 shutdown_connections 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1fac0a2c00 0x7f1fac0a50c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc8109120 0x7f1fc819ef20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc81049a0 0x7f1fc819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 --2- 172.21.15.59:0/1329484732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fc8103ff0 0x7f1fc819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:01:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 >> 172.21.15.59:0/1329484732 conn(0x7f1fc80ffa90 msgr2=0x7f1fc8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:01:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.756+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 shutdown_connections 2024-09-05T12:01:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:01:52.757+0000 7f1fce000640 1 -- 172.21.15.59:0/1329484732 wait complete. 2024-09-05T12:01:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:52 smithi059 ceph-mon[30025]: pgmap v520: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:01:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:52 smithi189 ceph-mon[34772]: pgmap v520: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:01:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:52 smithi089 ceph-mon[34499]: pgmap v520: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:01:53.509 INFO:tasks.cephfs.filesystem:scrub status for tag:d56e4a19-23f3-4969-aa89-ad1ae6b55084 - {'path': '/', 'tag': 'd56e4a19-23f3-4969-aa89-ad1ae6b55084', 'options': 'recursive,force'} 2024-09-05T12:01:55.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:54 smithi189 ceph-mon[34772]: pgmap v521: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:54 smithi089 ceph-mon[34499]: pgmap v521: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:55.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:54 smithi059 ceph-mon[30025]: pgmap v521: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:56.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:55 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:01:56.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:55 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:01:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:55 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:01:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:56 smithi189 ceph-mon[34772]: pgmap v522: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:57.127 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:56 smithi059 ceph-mon[30025]: pgmap v522: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:56 smithi089 ceph-mon[34499]: pgmap v522: 129 pgs: 129 active+clean; 2.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:01:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:01:58 smithi189 ceph-mon[34772]: pgmap v523: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:01:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:01:58 smithi089 ceph-mon[34499]: pgmap v523: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:01:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:01:58 smithi059 ceph-mon[30025]: pgmap v523: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:02:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:00 smithi189 ceph-mon[34772]: pgmap v524: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:00 smithi089 ceph-mon[34499]: pgmap v524: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:00 smithi059 ceph-mon[30025]: pgmap v524: 129 pgs: 129 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:02 smithi189 ceph-mon[34772]: pgmap v525: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:02 smithi089 ceph-mon[34499]: pgmap v525: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:03.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:02 smithi059 ceph-mon[30025]: pgmap v525: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:04 smithi189 ceph-mon[34772]: pgmap v526: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:02:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:04 smithi089 ceph-mon[34499]: pgmap v526: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:02:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:04 smithi059 ceph-mon[30025]: pgmap v526: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:02:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:06 smithi189 ceph-mon[34772]: pgmap v527: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:06 smithi089 ceph-mon[34499]: pgmap v527: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:06 smithi059 ceph-mon[30025]: pgmap v527: 129 pgs: 129 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:02:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:08 smithi189 ceph-mon[34772]: pgmap v528: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:02:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:08 smithi089 ceph-mon[34499]: pgmap v528: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:02:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:08 smithi059 ceph-mon[30025]: pgmap v528: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:02:09.943 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:02:09.944 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:02:09.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:02:10.334 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 -- 172.21.15.59:0/3099196242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1079c0 msgr2=0x7f359c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 --2- 172.21.15.59:0/3099196242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1079c0 0x7f359c107da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f358c009930 tx=0x7f358c02a420 comp rx=0 tx=0).stop 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 -- 172.21.15.59:0/3099196242 shutdown_connections 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 --2- 172.21.15.59:0/3099196242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 --2- 172.21.15.59:0/3099196242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1019c0 0x7f359c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 --2- 172.21.15.59:0/3099196242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1079c0 0x7f359c107da0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 -- 172.21.15.59:0/3099196242 >> 172.21.15.59:0/3099196242 conn(0x7f359c0fd6e0 msgr2=0x7f359c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 -- 172.21.15.59:0/3099196242 shutdown_connections 2024-09-05T12:02:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.903+0000 7f35a1400640 1 -- 172.21.15.59:0/3099196242 wait complete. 2024-09-05T12:02:10.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.904+0000 7f35a1400640 1 Processor -- start 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.904+0000 7f35a1400640 1 -- start start 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.904+0000 7f35a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1019c0 0x7f359c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.904+0000 7f35a1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f35a1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1079c0 0x7f359c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f35a1400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f359c111e10 con 0x7f359c1019c0 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f35a1400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f359c111c90 con 0x7f359c102360 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f35a1400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f359c111f90 con 0x7f359c1079c0 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36290/0 (socket says 172.21.15.59:36290) 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f359be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1019c0 0x7f359c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 -- 172.21.15.59:0/1535945177 learned_addr learned my addr 172.21.15.59:0/1535945177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f35a0a00640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1079c0 0x7f359c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 -- 172.21.15.59:0/1535945177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1079c0 msgr2=0x7f359c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1079c0 0x7f359c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 -- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1019c0 msgr2=0x7f359c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1019c0 0x7f359c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.905+0000 7f3593400640 1 -- 172.21.15.59:0/1535945177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c19d270 con 0x7f359c102360 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f3593400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f358400e9f0 tx=0x7f358400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358400cd80 con 0x7f359c102360 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3584005500 con 0x7f359c102360 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c19d560 con 0x7f359c102360 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584010690 con 0x7f359c102360 2024-09-05T12:02:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.906+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a4e40 con 0x7f359c102360 2024-09-05T12:02:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.907+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c101e20 con 0x7f359c102360 2024-09-05T12:02:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.909+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35840046c0 con 0x7f359c102360 2024-09-05T12:02:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.910+0000 7f3599600640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 0x7f35800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.910+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f35840cf830 con 0x7f359c102360 2024-09-05T12:02:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.911+0000 7f359be00640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 0x7f35800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.911+0000 7f359be00640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 0x7f35800a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:10.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:10.915+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f358409c320 con 0x7f359c102360 2024-09-05T12:02:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:10 smithi189 ceph-mon[34772]: pgmap v529: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.136+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f359c02d7c0 con 0x7f359c102360 2024-09-05T12:02:11.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:10 smithi059 ceph-mon[30025]: pgmap v529: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.137+0000 7f3599600640 1 -- 172.21.15.59:0/1535945177 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f3584086020 con 0x7f359c102360 2024-09-05T12:02:11.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:02:11.139 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:02:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:02:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 msgr2=0x7f35800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 0x7f35800a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 msgr2=0x7f359c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f358400e9f0 tx=0x7f358400edf0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 shutdown_connections 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35800a2c00 0x7f35800a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f359c1079c0 0x7f359c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f359c102360 0x7f359c198800 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 --2- 172.21.15.59:0/1535945177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f359c1019c0 0x7f359c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.139+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 >> 172.21.15.59:0/1535945177 conn(0x7f359c0fd6e0 msgr2=0x7f359c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.140+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 shutdown_connections 2024-09-05T12:02:11.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:11.140+0000 7f35a1400640 1 -- 172.21.15.59:0/1535945177 wait complete. 2024-09-05T12:02:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:10 smithi089 ceph-mon[34499]: pgmap v529: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:11.976 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:02:11.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:02:12.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:11 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1535945177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1535945177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:12.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1535945177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 -- 172.21.15.59:0/594744609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 msgr2=0x7fc7b4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 --2- 172.21.15.59:0/594744609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b4108a10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009900 tx=0x7fc79c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 -- 172.21.15.59:0/594744609 shutdown_connections 2024-09-05T12:02:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 --2- 172.21.15.59:0/594744609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4109140 0x7fc7b4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 --2- 172.21.15.59:0/594744609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b4108a10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 --2- 172.21.15.59:0/594744609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4104010 0x7fc7b41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 -- 172.21.15.59:0/594744609 >> 172.21.15.59:0/594744609 conn(0x7fc7b40ffa50 msgr2=0x7fc7b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 -- 172.21.15.59:0/594744609 shutdown_connections 2024-09-05T12:02:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.998+0000 7fc7b9600640 1 -- 172.21.15.59:0/594744609 wait complete. 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:12.999+0000 7fc7b9600640 1 Processor -- start 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 -- start start 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 0x7fc7b419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 0x7fc7b419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4114160 con 0x7fc7b41049c0 2024-09-05T12:02:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4113fe0 con 0x7fc7b4104010 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41142e0 con 0x7fc7b4109140 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50620/0 (socket says 172.21.15.59:50620) 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7aa000640 1 -- 172.21.15.59:0/754090685 learned_addr learned my addr 172.21.15.59:0/754090685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 0x7fc7b419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.000+0000 7fc7b3400640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 0x7fc7b419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 -- 172.21.15.59:0/754090685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 msgr2=0x7fc7b419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 0x7fc7b419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 -- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 msgr2=0x7fc7b419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 -- 172.21.15.59:0/754090685 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419f640 con 0x7fc7b4109140 2024-09-05T12:02:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 0x7fc7b419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7aa000640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b3400640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 0x7fc7b419ef60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00d6b0 tx=0x7fc7ac00db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7abe00640 1 -- 172.21.15.59:0/754090685 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac017070 con 0x7fc7b4109140 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7abe00640 1 -- 172.21.15.59:0/754090685 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ac009d00 con 0x7fc7b4109140 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b419f930 con 0x7fc7b4109140 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.001+0000 7fc7abe00640 1 -- 172.21.15.59:0/754090685 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac003d60 con 0x7fc7b4109140 2024-09-05T12:02:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.002+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a7210 con 0x7fc7b4109140 2024-09-05T12:02:13.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.004+0000 7fc7bc0c4740 1 Processor -- start 2024-09-05T12:02:13.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7abe00640 1 -- 172.21.15.59:0/754090685 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc7ac009820 con 0x7fc7b4109140 2024-09-05T12:02:13.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 -- start start 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x559283632050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x559283634980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55928358d530 con 0x559283632590 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559283534860 con 0x5592836273b0 2024-09-05T12:02:13.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5592835894f0 con 0x559283627cd0 2024-09-05T12:02:13.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36318/0 (socket says 172.21.15.59:36318) 2024-09-05T12:02:13.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.005+0000 7fc7aa000640 1 -- 172.21.15.59:0/3362009811 learned_addr learned my addr 172.21.15.59:0/3362009811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:13.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x559283634980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x559283632050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 -- 172.21.15.59:0/3362009811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 msgr2=0x559283632050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x559283632050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 -- 172.21.15.59:0/3362009811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 msgr2=0x559283634980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x559283634980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 -- 172.21.15.59:0/3362009811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592832feb90 con 0x5592836273b0 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7abe00640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 0x7fc7980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7aa000640 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0096c0 tx=0x7fc79c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d3f52a8fb9f6da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.006+0000 7fc7abe00640 1 -- 172.21.15.59:0/754090685 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc7ac0cec60 con 0x7fc7b4109140 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7a3400640 1 -- 172.21.15.59:0/3362009811 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c035070 con 0x5592836273b0 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 0x7fc7980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7a3400640 1 -- 172.21.15.59:0/3362009811 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c0042f0 con 0x5592836273b0 2024-09-05T12:02:13.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7a3400640 1 -- 172.21.15.59:0/3362009811 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c03d6b0 con 0x5592836273b0 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/3362009811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 msgr2=0x559283627790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 0x7fc7980a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009fd0 tx=0x7fc7a4009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0096c0 tx=0x7fc79c009260 comp rx=0 tx=0).stop 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/3362009811 shutdown_connections 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x559283634980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x559283632050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/3362009811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x559283627790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.007+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/3362009811 >> 172.21.15.59:0/3362009811 conn(0x559283623940 msgr2=0x559283623d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.008+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/3362009811 shutdown_connections 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.008+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/3362009811 wait complete. 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.008+0000 7fc7bc0c4740 1 -- start start 2024-09-05T12:02:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.008+0000 7fc7bc0c4740 1 Processor -- start 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928362fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x5592836301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559283634fb0 con 0x559283632590 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559283547e60 con 0x5592836273b0 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559283635130 con 0x559283627cd0 2024-09-05T12:02:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7bc0c4740 10 client.0 _ll_register_callbacks cb 0x55928354ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x5592836301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36326/0 (socket says 172.21.15.59:36326) 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7b2a00640 1 -- 172.21.15.59:0/4096248926 learned_addr learned my addr 172.21.15.59:0/4096248926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.009+0000 7fc7aa000640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928362fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 -- 172.21.15.59:0/4096248926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 msgr2=0x55928362fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928362fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 -- 172.21.15.59:0/4096248926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 msgr2=0x5592836301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x5592836301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 -- 172.21.15.59:0/4096248926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559283553990 con 0x5592836273b0 2024-09-05T12:02:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b3400640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x5592836301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40124a0 tx=0x7fc7a4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e129a98d6a194a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.010+0000 7fc797e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4003e00 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c0042f0 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc7a4018030 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7a40046e0 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc7bc0c4740 10 client.26051 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559283640ed0 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.011+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c0042f0 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7a4028540 con 0x5592836273b0 2024-09-05T12:02:13.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 10 client.26051 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 10 client.26051 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 20 client.26051 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559283642390 0x559283644800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 4 client.26051 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559283530310 con 0x559283642390 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.012+0000 7fc7aa000640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559283642390 0x559283644800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.013+0000 7fc7aa000640 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559283642390 0x559283644800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5fbe2d11fe4e5a server_cookie=5e4ac6660524f2cd in_seq=0 out_seq=0 2024-09-05T12:02:13.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.013+0000 7fc797e00640 10 client.26051 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc797e00640 1 -- 172.21.15.59:0/4096248926 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559283530310 con 0x559283642390 2024-09-05T12:02:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc797e00640 10 client.26051 handle_command_reply: tid=0 2024-09-05T12:02:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 client.26051 shutdown 2024-09-05T12:02:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 msgr2=0x55928362f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40124a0 tx=0x7fc7a4004ab0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 shutdown_connections 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559283642390 0x559283644800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559283632590 0x5592836301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928362fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/4096248926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5592836273b0 0x55928362f760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc797e00640 0 client.26051 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 >> 172.21.15.59:0/4096248926 conn(0x559283623940 msgr2=0x559283622c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 shutdown_connections 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/4096248926 wait complete. 2024-09-05T12:02:13.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.014+0000 7fc7bc0c4740 20 client.26051 trim_cache size 0 max 16384 2024-09-05T12:02:13.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 Processor -- start 2024-09-05T12:02:13.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 -- start start 2024-09-05T12:02:13.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928364a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283632590 0x55928364a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41af000 con 0x5592836273b0 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41b1880 con 0x559283632590 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.032+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41af670 con 0x559283627cd0 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50654/0 (socket says 172.21.15.59:50654) 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 -- 172.21.15.59:0/975011168 learned_addr learned my addr 172.21.15.59:0/975011168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7aa000640 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928364a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b3400640 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283632590 0x55928364a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 -- 172.21.15.59:0/975011168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 msgr2=0x55928364a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928364a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 -- 172.21.15.59:0/975011168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283632590 msgr2=0x55928364a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283632590 0x55928364a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 -- 172.21.15.59:0/975011168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592836249f0 con 0x5592836273b0 2024-09-05T12:02:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.033+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400ffd0 tx=0x7fc7a40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801c2483eed42776 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7a2a00640 1 -- 172.21.15.59:0/975011168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400fd30 con 0x5592836273b0 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7a2a00640 1 -- 172.21.15.59:0/975011168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4004880 con 0x5592836273b0 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7a2a00640 1 -- 172.21.15.59:0/975011168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4011e30 con 0x5592836273b0 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/975011168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 msgr2=0x55928362fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400ffd0 tx=0x7fc7a40095f0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/975011168 shutdown_connections 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283632590 0x55928364a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283627cd0 0x55928364a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/975011168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928362fe30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.034+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/975011168 >> 172.21.15.59:0/975011168 conn(0x559283623940 msgr2=0x55928362f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.035+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/975011168 shutdown_connections 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.035+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/975011168 wait complete. 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.035+0000 7fc7bc0c4740 1 -- start start 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.035+0000 7fc7bc0c4740 1 Processor -- start 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283627cd0 0x55928363a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283632590 0x559283637350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41afd40 con 0x5592836273b0 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41b11b0 con 0x559283627cd0 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41b0410 con 0x559283632590 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7bc0c4740 10 client.0 _ll_register_callbacks cb 0x7fc7a4012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50668/0 (socket says 172.21.15.59:50668) 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7b2a00640 1 -- 172.21.15.59:0/482687728 learned_addr learned my addr 172.21.15.59:0/482687728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7b3400640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283632590 0x559283637350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.036+0000 7fc7aa000640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283627cd0 0x55928363a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 -- 172.21.15.59:0/482687728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283632590 msgr2=0x559283637350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283632590 0x559283637350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 -- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283627cd0 msgr2=0x55928363a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283627cd0 0x55928363a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 -- 172.21.15.59:0/482687728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559283637910 con 0x5592836273b0 2024-09-05T12:02:13.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7b2a00640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40116f0 tx=0x7fc7a4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e83bf141d1c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc797e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400fd30 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc7a4011cc0 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4004880 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.037+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7a404d050 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 10 client.16044 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5592836ad950 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4004880 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7a403f820 con 0x5592836273b0 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 10 client.16044 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 10 client.16044 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:13.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 20 client.16044 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5592836ae020 0x5592836b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 4 client.16044 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55928359ac60 con 0x5592836ae020 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.038+0000 7fc7aa000640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5592836ae020 0x5592836b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.039+0000 7fc7aa000640 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5592836ae020 0x5592836b0440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78afcb32d80b671 server_cookie=8647712e5f939693 in_seq=0 out_seq=0 2024-09-05T12:02:13.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.039+0000 7fc797e00640 10 client.16044 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc797e00640 1 -- 172.21.15.59:0/482687728 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55928359ac60 con 0x5592836ae020 2024-09-05T12:02:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc797e00640 10 client.16044 handle_command_reply: tid=0 2024-09-05T12:02:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 client.16044 shutdown 2024-09-05T12:02:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 msgr2=0x55928363a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40116f0 tx=0x7fc7a4011720 comp rx=0 tx=0).stop 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 shutdown_connections 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559283632590 0x559283637350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559283627cd0 0x55928363a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5592836ae020 0x5592836b0440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 --2- 172.21.15.59:0/482687728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592836273b0 0x55928363a380 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 >> 172.21.15.59:0/482687728 conn(0x559283623940 msgr2=0x55928362ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc797e00640 0 client.16044 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 shutdown_connections 2024-09-05T12:02:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 1 -- 172.21.15.59:0/482687728 wait complete. 2024-09-05T12:02:13.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:12 smithi059 ceph-mon[30025]: pgmap v530: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.048+0000 7fc7bc0c4740 20 client.16044 trim_cache size 0 max 16384 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 18, 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "4cea4795": { 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 18, 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:02:13.052 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:13.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 msgr2=0x7fc7980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 0x7fc7980a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009fd0 tx=0x7fc7a4009500 comp rx=0 tx=0).stop 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 msgr2=0x7fc7b419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 0x7fc7b419ef60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00d6b0 tx=0x7fc7ac00db70 comp rx=0 tx=0).stop 2024-09-05T12:02:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 shutdown_connections 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc7980a2c00 0x7fc7980a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7b4109140 0x7fc7b419ef60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7b41049c0 0x7fc7b419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 --2- 172.21.15.59:0/754090685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7b4104010 0x7fc7b419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 >> 172.21.15.59:0/754090685 conn(0x7fc7b40ffa50 msgr2=0x7fc7b4100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 shutdown_connections 2024-09-05T12:02:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:13.049+0000 7fc7b9600640 1 -- 172.21.15.59:0/754090685 wait complete. 2024-09-05T12:02:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:12 smithi189 ceph-mon[34772]: pgmap v530: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:12 smithi089 ceph-mon[34499]: pgmap v530: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:13.907 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4cea4795', quiesce took 4.0 seconds. Will release after: 26.0 2024-09-05T12:02:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:14 smithi189 ceph-mon[34772]: pgmap v531: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:02:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:14 smithi089 ceph-mon[34499]: pgmap v531: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:02:15.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:14 smithi059 ceph-mon[30025]: pgmap v531: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:02:17.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:16 smithi189 ceph-mon[34772]: pgmap v532: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:16 smithi089 ceph-mon[34499]: pgmap v532: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:16 smithi059 ceph-mon[30025]: pgmap v532: 129 pgs: 129 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:02:19.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:18 smithi189 ceph-mon[34772]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:18 smithi089 ceph-mon[34499]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:19.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:18 smithi059 ceph-mon[30025]: pgmap v533: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:02:21.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:20 smithi189 ceph-mon[34772]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:02:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:20 smithi089 ceph-mon[34499]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:02:21.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:20 smithi059 ceph-mon[30025]: pgmap v534: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:02:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:22 smithi189 ceph-mon[34772]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:02:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:22 smithi089 ceph-mon[34499]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:02:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:22 smithi059 ceph-mon[30025]: pgmap v535: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:02:23.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:02:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.607+0000 7fa7a4c00640 1 -- 172.21.15.59:0/1324569158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 msgr2=0x7fa7a0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.607+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/1324569158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a0108a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009900 tx=0x7fa78c02a410 comp rx=0 tx=0).stop 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 -- 172.21.15.59:0/1324569158 shutdown_connections 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/1324569158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 0x7fa7a0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/1324569158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a0108a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/1324569158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 -- 172.21.15.59:0/1324569158 >> 172.21.15.59:0/1324569158 conn(0x7fa7a00ffa50 msgr2=0x7fa7a0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 -- 172.21.15.59:0/1324569158 shutdown_connections 2024-09-05T12:02:24.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.608+0000 7fa7a4c00640 1 -- 172.21.15.59:0/1324569158 wait complete. 2024-09-05T12:02:24.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 Processor -- start 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 -- start start 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 0x7fa7a019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0114160 con 0x7fa7a01049c0 2024-09-05T12:02:24.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0113fe0 con 0x7fa7a0109140 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.609+0000 7fa7a4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01142e0 con 0x7fa7a0104010 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35262/0 (socket says 172.21.15.59:35262) 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 -- 172.21.15.59:0/3299084314 learned_addr learned my addr 172.21.15.59:0/3299084314 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79d400640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79e800640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 0x7fa7a019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 -- 172.21.15.59:0/3299084314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 msgr2=0x7fa7a019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 0x7fa7a019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 -- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 msgr2=0x7fa7a019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 -- 172.21.15.59:0/3299084314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a019f670 con 0x7fa7a0104010 2024-09-05T12:02:24.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.610+0000 7fa79de00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e9f0 tx=0x7fa79000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.611+0000 7fa796a00640 1 -- 172.21.15.59:0/3299084314 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000cd80 con 0x7fa7a0104010 2024-09-05T12:02:24.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.611+0000 7fa796a00640 1 -- 172.21.15.59:0/3299084314 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790005500 con 0x7fa7a0104010 2024-09-05T12:02:24.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.611+0000 7fa796a00640 1 -- 172.21.15.59:0/3299084314 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790010690 con 0x7fa7a0104010 2024-09-05T12:02:24.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.611+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a019f960 con 0x7fa7a0104010 2024-09-05T12:02:24.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.611+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a01a7240 con 0x7fa7a0104010 2024-09-05T12:02:24.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa7a782e740 1 Processor -- start 2024-09-05T12:02:24.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa7a782e740 1 -- start start 2024-09-05T12:02:24.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa796a00640 1 -- 172.21.15.59:0/3299084314 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7900046c0 con 0x7fa7a0104010 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.614+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6282160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa7a782e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddf61db3e0 con 0x55ddf627fd70 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa7a782e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6182d80 con 0x55ddf6274b90 2024-09-05T12:02:24.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa7a782e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ddf61d73a0 con 0x55ddf62754b0 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45766/0 (socket says 172.21.15.59:45766) 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35264/0 (socket says 172.21.15.59:35264) 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79e800640 1 -- 172.21.15.59:0/4249008098 learned_addr learned my addr 172.21.15.59:0/4249008098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79d400640 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6282160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 -- 172.21.15.59:0/4249008098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 msgr2=0x55ddf6274f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 -- 172.21.15.59:0/4249008098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 msgr2=0x55ddf6282160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6282160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79de00640 1 -- 172.21.15.59:0/4249008098 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf61a1ae0 con 0x55ddf62754b0 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.615+0000 7fa79e800640 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa796a00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 0x7fa7840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa79de00640 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7900cce60 tx=0x7fa7900d5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8845a9670d5025 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa796a00640 1 -- 172.21.15.59:0/3299084314 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7900cf830 con 0x7fa7a0104010 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa79e800640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 0x7fa7840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa78b000640 1 -- 172.21.15.59:0/4249008098 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790097ec0 con 0x55ddf62754b0 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa78b000640 1 -- 172.21.15.59:0/4249008098 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79009c320 con 0x55ddf62754b0 2024-09-05T12:02:24.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.616+0000 7fa78b000640 1 -- 172.21.15.59:0/4249008098 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79009b580 con 0x55ddf62754b0 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 -- 172.21.15.59:0/4249008098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 msgr2=0x55ddf627f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa79e800640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 0x7fa7840a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa798009fd0 tx=0x7fa798009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7900cce60 tx=0x7fa7900d5b20 comp rx=0 tx=0).stop 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 -- 172.21.15.59:0/4249008098 shutdown_connections 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6282160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627f830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 --2- 172.21.15.59:0/4249008098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf6274f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 -- 172.21.15.59:0/4249008098 >> 172.21.15.59:0/4249008098 conn(0x55ddf6271120 msgr2=0x55ddf6271500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 -- 172.21.15.59:0/4249008098 shutdown_connections 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.617+0000 7fa7a782e740 1 -- 172.21.15.59:0/4249008098 wait complete. 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 -- start start 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 Processor -- start 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf627d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddf62829e0 con 0x55ddf627fd70 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddf61960f0 con 0x55ddf6274b90 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6282b60 con 0x55ddf62754b0 2024-09-05T12:02:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.618+0000 7fa7a782e740 10 client.0 _ll_register_callbacks cb 0x55ddf61879e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45782/0 (socket says 172.21.15.59:45782) 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 -- 172.21.15.59:0/3412608751 learned_addr learned my addr 172.21.15.59:0/3412608751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79e800640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf627d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79d400640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 -- 172.21.15.59:0/3412608751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 msgr2=0x55ddf627d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 -- 172.21.15.59:0/3412608751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 msgr2=0x55ddf627d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf627d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 -- 172.21.15.59:0/3412608751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf5f4c3c0 con 0x55ddf6274b90 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa79de00640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7900049d0 tx=0x7fa79000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dbd8c7a37adbb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.619+0000 7fa77f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79009b760 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7900cf430 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79009c320 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa79009b9f0 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 10 client.26069 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ddf628e750 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79009c320 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa79009be80 con 0x55ddf6274b90 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 10 client.26069 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 20 client.26069 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf628faa0 0x55ddf6291e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 4 client.26069 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:24.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.620+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf61e8950 con 0x55ddf628faa0 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.621+0000 7fa79d400640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf628faa0 0x55ddf6291e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.621+0000 7fa79d400640 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf628faa0 0x55ddf6291e90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d480e60f6318361 server_cookie=b274f94d71657d25 in_seq=0 out_seq=0 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.621+0000 7fa77f400640 10 client.26069 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa77f400640 1 -- 172.21.15.59:0/3412608751 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddf61e8950 con 0x55ddf628faa0 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa77f400640 10 client.26069 handle_command_reply: tid=0 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa7a782e740 1 client.26069 shutdown 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 msgr2=0x55ddf627cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa7900049d0 tx=0x7fa79000cf90 comp rx=0 tx=0).stop 2024-09-05T12:02:24.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 shutdown_connections 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.622+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf628faa0 0x55ddf6291e90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf627d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf62754b0 0x55ddf627d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa77f400640 0 client.26069 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3412608751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf6274b90 0x55ddf627cd30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 >> 172.21.15.59:0/3412608751 conn(0x55ddf6271120 msgr2=0x55ddf6276e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 shutdown_connections 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 1 -- 172.21.15.59:0/3412608751 wait complete. 2024-09-05T12:02:24.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.623+0000 7fa7a782e740 20 client.26069 trim_cache size 0 max 16384 2024-09-05T12:02:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.641+0000 7fa7a782e740 1 Processor -- start 2024-09-05T12:02:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 -- start start 2024-09-05T12:02:24.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf6274b90 0x55ddf62775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6294eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01af0b0 con 0x55ddf627fd70 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b1930 con 0x55ddf62754b0 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa7a782e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01af720 con 0x55ddf6274b90 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6294eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45790/0 (socket says 172.21.15.59:45790) 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79d400640 1 -- 172.21.15.59:0/3346056451 learned_addr learned my addr 172.21.15.59:0/3346056451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6294eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35570/0 (socket says 172.21.15.59:35570) 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.642+0000 7fa79de00640 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf6274b90 0x55ddf62775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 -- 172.21.15.59:0/3346056451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf6274b90 msgr2=0x55ddf62775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf6274b90 0x55ddf62775f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 -- 172.21.15.59:0/3346056451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 msgr2=0x55ddf6294eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6294eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 -- 172.21.15.59:0/3346056451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf6276110 con 0x55ddf62754b0 2024-09-05T12:02:24.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa79d400640 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009900 tx=0x7fa78c0319a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccce96056a7f90a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa78a600640 1 -- 172.21.15.59:0/3346056451 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c035040 con 0x55ddf62754b0 2024-09-05T12:02:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa78a600640 1 -- 172.21.15.59:0/3346056451 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005500 con 0x55ddf62754b0 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.643+0000 7fa78a600640 1 -- 172.21.15.59:0/3346056451 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c004c30 con 0x55ddf62754b0 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 -- 172.21.15.59:0/3346056451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 msgr2=0x55ddf627ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009900 tx=0x7fa78c0319a0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 -- 172.21.15.59:0/3346056451 shutdown_connections 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf627fd70 0x55ddf6294eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf627ccf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 --2- 172.21.15.59:0/3346056451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf6274b90 0x55ddf62775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 -- 172.21.15.59:0/3346056451 >> 172.21.15.59:0/3346056451 conn(0x55ddf6271120 msgr2=0x55ddf628b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.644+0000 7fa7a782e740 1 -- 172.21.15.59:0/3346056451 shutdown_connections 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.645+0000 7fa7a782e740 1 -- 172.21.15.59:0/3346056451 wait complete. 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.645+0000 7fa7a782e740 1 -- start start 2024-09-05T12:02:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.645+0000 7fa7a782e740 1 Processor -- start 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf6292750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf627fd70 0x55ddf6292c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01afdf0 con 0x55ddf6274b90 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b1260 con 0x55ddf62754b0 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01b04c0 con 0x55ddf627fd70 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa7a782e740 10 client.0 _ll_register_callbacks cb 0x7fa790005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35572/0 (socket says 172.21.15.59:35572) 2024-09-05T12:02:24.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 -- 172.21.15.59:0/1542329472 learned_addr learned my addr 172.21.15.59:0/1542329472 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:24.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79d400640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf6292750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79e800640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf627fd70 0x55ddf6292c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 -- 172.21.15.59:0/1542329472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf627fd70 msgr2=0x55ddf6292c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf627fd70 0x55ddf6292c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 -- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 msgr2=0x55ddf6292750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf6292750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.646+0000 7fa79de00640 1 -- 172.21.15.59:0/1542329472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf6293250 con 0x55ddf6274b90 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa79de00640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7900d5210 tx=0x7fa7900cd1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf399a49609d9acc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa77f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79009b760 con 0x55ddf6274b90 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa790099f20 con 0x55ddf6274b90 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c005500 con 0x55ddf6274b90 2024-09-05T12:02:24.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa790097ec0 con 0x55ddf6274b90 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa7a782e740 10 client.16062 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.647+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ddf6293250 con 0x55ddf6274b90 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005500 con 0x55ddf6274b90 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa7900cf430 con 0x55ddf6274b90 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 10 client.16062 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 20 client.16062 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf62fa530 0x55ddf62fc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 4 client.16062 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddf61e2e60 con 0x55ddf62fa530 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.648+0000 7fa79d400640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf62fa530 0x55ddf62fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa79d400640 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf62fa530 0x55ddf62fc920 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844151c127b9b7fe server_cookie=acc4674671a1ac49 in_seq=0 out_seq=0 2024-09-05T12:02:24.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa77f400640 10 client.16062 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa77f400640 1 -- 172.21.15.59:0/1542329472 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ddf61e2e60 con 0x55ddf62fa530 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa77f400640 10 client.16062 handle_command_reply: tid=0 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 client.16062 shutdown 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 msgr2=0x55ddf62953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7900d5210 tx=0x7fa7900cd1e0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 shutdown_connections 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddf627fd70 0x55ddf6292c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddf62754b0 0x55ddf6292750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ddf62fa530 0x55ddf62fc920 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 --2- 172.21.15.59:0/1542329472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf6274b90 0x55ddf62953b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 >> 172.21.15.59:0/1542329472 conn(0x55ddf6271120 msgr2=0x55ddf627b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.649+0000 7fa77f400640 0 client.16062 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 shutdown_connections 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a782e740 1 -- 172.21.15.59:0/1542329472 wait complete. 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a782e740 20 client.16062 trim_cache size 0 max 16384 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:02:24.655 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 msgr2=0x7fa7840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 0x7fa7840a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa798009fd0 tx=0x7fa798009500 comp rx=0 tx=0).stop 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 msgr2=0x7fa7a019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.650+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e9f0 tx=0x7fa79000edf0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 shutdown_connections 2024-09-05T12:02:24.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa7840a2c00 0x7fa7840a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7a0109140 0x7fa7a019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7a01049c0 0x7fa7a019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 --2- 172.21.15.59:0/3299084314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7a0104010 0x7fa7a019a6c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 >> 172.21.15.59:0/3299084314 conn(0x7fa7a00ffa50 msgr2=0x7fa7a0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 shutdown_connections 2024-09-05T12:02:24.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:24.651+0000 7fa7a4c00640 1 -- 172.21.15.59:0/3299084314 wait complete. 2024-09-05T12:02:24.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:24 smithi059 ceph-mon[30025]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:24 smithi189 ceph-mon[34772]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:24 smithi089 ceph-mon[34499]: pgmap v536: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:25.444 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:02:25.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:02:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:26.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.414+0000 7f084dc00640 1 -- 172.21.15.59:0/570601558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848074590 msgr2=0x7f0848074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.414+0000 7f084dc00640 1 --2- 172.21.15.59:0/570601558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848074590 0x7f0848074970 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0838009900 tx=0x7f083802a480 comp rx=0 tx=0).stop 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.414+0000 7f084dc00640 1 -- 172.21.15.59:0/570601558 shutdown_connections 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.414+0000 7f084dc00640 1 --2- 172.21.15.59:0/570601558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848112b90 0x7f0848114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.415+0000 7f084dc00640 1 --2- 172.21.15.59:0/570601558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f0848112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.415+0000 7f084dc00640 1 --2- 172.21.15.59:0/570601558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848074590 0x7f0848074970 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.415+0000 7f084dc00640 1 -- 172.21.15.59:0/570601558 >> 172.21.15.59:0/570601558 conn(0x7f08480fd8c0 msgr2=0x7f08480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.415+0000 7f084dc00640 1 -- 172.21.15.59:0/570601558 shutdown_connections 2024-09-05T12:02:26.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.415+0000 7f084dc00640 1 -- 172.21.15.59:0/570601558 wait complete. 2024-09-05T12:02:26.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 Processor -- start 2024-09-05T12:02:26.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 -- start start 2024-09-05T12:02:26.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848074590 0x7f08481a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848112b90 0x7f084819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08481185a0 con 0x7f0848112b90 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0848118420 con 0x7f0848074eb0 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.416+0000 7f084dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0848118720 con 0x7f0848074590 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0847400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848112b90 0x7f084819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848074590 0x7f08481a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45816/0 (socket says 172.21.15.59:45816) 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 -- 172.21.15.59:0/2546856355 learned_addr learned my addr 172.21.15.59:0/2546856355 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 -- 172.21.15.59:0/2546856355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848074590 msgr2=0x7f08481a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848074590 0x7f08481a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 -- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848112b90 msgr2=0x7f084819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:26.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848112b90 0x7f084819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 -- 172.21.15.59:0/2546856355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084819f000 con 0x7f0848074eb0 2024-09-05T12:02:26.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.417+0000 7f0846000640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f083400e9f0 tx=0x7f083400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.418+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083400cd80 con 0x7f0848074eb0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.418+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0834010910 con 0x7f0848074eb0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.418+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084819f2f0 con 0x7f0848074eb0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.418+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834018990 con 0x7f0848074eb0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.418+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08481ab5e0 con 0x7f0848074eb0 2024-09-05T12:02:26.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.419+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0848074970 con 0x7f0848074eb0 2024-09-05T12:02:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.421+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0834010ac0 con 0x7f0848074eb0 2024-09-05T12:02:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.423+0000 7f083f400640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 0x7f082c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.423+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f08340cfb40 con 0x7f0848074eb0 2024-09-05T12:02:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.423+0000 7f0846a00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 0x7f082c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:26.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.424+0000 7f0846a00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 0x7f082c0a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0838006010 tx=0x7f0838000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:26.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.424+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f083409bfe0 con 0x7f0848074eb0 2024-09-05T12:02:26.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.648+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f084802d7c0 con 0x7f0848074eb0 2024-09-05T12:02:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.649+0000 7f083f400640 1 -- 172.21.15.59:0/2546856355 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f08340a02e0 con 0x7f0848074eb0 2024-09-05T12:02:26.650 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:02:26.651 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:02:26.651 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 msgr2=0x7f082c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 0x7f082c0a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0838006010 tx=0x7f0838000bc0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 msgr2=0x7f08481a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f083400e9f0 tx=0x7f083400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 shutdown_connections 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f082c0a2c00 0x7f082c0a50c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0848112b90 0x7f084819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0848074eb0 0x7f08481a4c50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 --2- 172.21.15.59:0/2546856355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0848074590 0x7f08481a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 >> 172.21.15.59:0/2546856355 conn(0x7f08480fd8c0 msgr2=0x7f08480ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 shutdown_connections 2024-09-05T12:02:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:26.652+0000 7f084dc00640 1 -- 172.21.15.59:0/2546856355 wait complete. 2024-09-05T12:02:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:26 smithi059 ceph-mon[30025]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:26 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2546856355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:26 smithi189 ceph-mon[34772]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:27.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:26 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2546856355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:26 smithi089 ceph-mon[34499]: pgmap v537: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:26 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2546856355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:27.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:02:27.796 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 -- 172.21.15.59:0/927029709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 msgr2=0x7f08a8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 --2- 172.21.15.59:0/927029709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a8108a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0890009900 tx=0x7f089002a410 comp rx=0 tx=0).stop 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 -- 172.21.15.59:0/927029709 shutdown_connections 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 --2- 172.21.15.59:0/927029709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 --2- 172.21.15.59:0/927029709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a8108a10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 --2- 172.21.15.59:0/927029709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 -- 172.21.15.59:0/927029709 >> 172.21.15.59:0/927029709 conn(0x7f08a80ffa50 msgr2=0x7f08a8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 -- 172.21.15.59:0/927029709 shutdown_connections 2024-09-05T12:02:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.356+0000 7f08ade00640 1 -- 172.21.15.59:0/927029709 wait complete. 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 Processor -- start 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 -- start start 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8114160 con 0x7f08a81049c0 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8113fe0 con 0x7f08a8104010 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.357+0000 7f08ade00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81142e0 con 0x7f08a8109140 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35616/0 (socket says 172.21.15.59:35616) 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 -- 172.21.15.59:0/804959300 learned_addr learned my addr 172.21.15.59:0/804959300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f08a6a00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f08a7400640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 -- 172.21.15.59:0/804959300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 msgr2=0x7f08a819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 -- 172.21.15.59:0/804959300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 msgr2=0x7f08a819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f089e000640 1 -- 172.21.15.59:0/804959300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a819f610 con 0x7f08a81049c0 2024-09-05T12:02:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f08a7400640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:28.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.358+0000 7f08a6a00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f089e000640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0890006010 tx=0x7f0890004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f08aca00640 1 -- 172.21.15.59:0/804959300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089003f030 con 0x7f08a81049c0 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f08aca00640 1 -- 172.21.15.59:0/804959300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0890004440 con 0x7f08a81049c0 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f08aca00640 1 -- 172.21.15.59:0/804959300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890035070 con 0x7f08a81049c0 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a819f8a0 con 0x7f08a81049c0 2024-09-05T12:02:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.359+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a819fd30 con 0x7f08a81049c0 2024-09-05T12:02:28.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.362+0000 7f08b0a03740 1 Processor -- start 2024-09-05T12:02:28.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.362+0000 7f08aca00640 1 -- 172.21.15.59:0/804959300 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0890042030 con 0x7f08a81049c0 2024-09-05T12:02:28.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 -- start start 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca846050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca850910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca853240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5631ca7ac3e0 con 0x5631ca845c70 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631ca753d80 con 0x5631ca846590 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08b0a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631ca7a83a0 con 0x5631ca850e50 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca846050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f089e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca853240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca850910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca850910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45846/0 (socket says 172.21.15.59:45846) 2024-09-05T12:02:28.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.363+0000 7f08a6a00640 1 -- 172.21.15.59:0/3246195281 learned_addr learned my addr 172.21.15.59:0/3246195281 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 -- 172.21.15.59:0/3246195281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 msgr2=0x5631ca850910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca850910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 -- 172.21.15.59:0/3246195281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 msgr2=0x5631ca846050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca846050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 -- 172.21.15.59:0/3246195281 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ca772ae0 con 0x5631ca850e50 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f08a7400640 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca846050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f08aca00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 0x7f088c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f089e000640 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca853240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08900ece50 tx=0x7f08900f5cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc9a6b09897d4db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f08a7400640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 0x7f088c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.364+0000 7f08aca00640 1 -- 172.21.15.59:0/804959300 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f08900ef820 con 0x7f08a81049c0 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f0897e00640 1 -- 172.21.15.59:0/3246195281 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08900fe020 con 0x5631ca850e50 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f0897e00640 1 -- 172.21.15.59:0/3246195281 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08900bc310 con 0x5631ca850e50 2024-09-05T12:02:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f0897e00640 1 -- 172.21.15.59:0/3246195281 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08900bb9e0 con 0x5631ca850e50 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f08b0a03740 1 -- 172.21.15.59:0/3246195281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 msgr2=0x5631ca853240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f08a7400640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 0x7f088c0a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005fd0 tx=0x7f08a0009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.365+0000 7f08b0a03740 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca853240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08900ece50 tx=0x7f08900f5cb0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 -- 172.21.15.59:0/3246195281 shutdown_connections 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca853240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca850910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 --2- 172.21.15.59:0/3246195281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca846050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 -- 172.21.15.59:0/3246195281 >> 172.21.15.59:0/3246195281 conn(0x5631ca842200 msgr2=0x5631ca8425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 -- 172.21.15.59:0/3246195281 shutdown_connections 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 -- 172.21.15.59:0/3246195281 wait complete. 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.366+0000 7f08b0a03740 1 -- start start 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 Processor -- start 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca84e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca84e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5631ca8537e0 con 0x5631ca850e50 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631ca7670f0 con 0x5631ca846590 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631ca853960 con 0x5631ca845c70 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08b0a03740 10 client.0 _ll_register_callbacks cb 0x5631ca74ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca84e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35346/0 (socket says 172.21.15.59:35346) 2024-09-05T12:02:28.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f08a6a00640 1 -- 172.21.15.59:0/1924881682 learned_addr learned my addr 172.21.15.59:0/1924881682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:28.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.367+0000 7f089e000640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca84e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 -- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 msgr2=0x5631ca84e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca84e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 -- 172.21.15.59:0/1924881682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 msgr2=0x5631ca84e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca84e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 -- 172.21.15.59:0/1924881682 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ca51d3c0 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f089e000640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca84e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0898012d20 tx=0x7f089800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a570bd6503842516 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f0894c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.368+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898012110 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08900bc310 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0898004050 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f08980122b0 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 10 client.26017 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5631ca85f810 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08900bc310 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0898010670 con 0x5631ca845c70 2024-09-05T12:02:28.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 10 client.26017 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.369+0000 7f08b0a03740 20 client.26017 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca860c20 0x5631ca863070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f08b0a03740 4 client.26017 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631ca7b9950 con 0x5631ca860c20 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f089e000640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca860c20 0x5631ca863070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f089e000640 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca860c20 0x5631ca863070 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4567e3503c3029 server_cookie=ce0a714ae701849 in_seq=0 out_seq=0 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.370+0000 7f0894c00640 10 client.26017 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.371+0000 7f0894c00640 1 -- 172.21.15.59:0/1924881682 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631ca7b9950 con 0x5631ca860c20 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.371+0000 7f0894c00640 10 client.26017 handle_command_reply: tid=0 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.371+0000 7f08b0a03740 1 client.26017 shutdown 2024-09-05T12:02:28.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.371+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 msgr2=0x5631ca84dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.371+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0898012d20 tx=0x7f089800f790 comp rx=0 tx=0).stop 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 shutdown_connections 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca860c20 0x5631ca863070 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca84e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f0894c00640 0 client.26017 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca84e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1924881682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca845c70 0x5631ca84dd00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 >> 172.21.15.59:0/1924881682 conn(0x5631ca842200 msgr2=0x5631ca847f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 shutdown_connections 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 1 -- 172.21.15.59:0/1924881682 wait complete. 2024-09-05T12:02:28.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.372+0000 7f08b0a03740 20 client.26017 trim_cache size 0 max 16384 2024-09-05T12:02:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.385+0000 7f08b0a03740 1 Processor -- start 2024-09-05T12:02:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 -- start start 2024-09-05T12:02:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca868e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca840b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81af120 con 0x5631ca845c70 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81b19a0 con 0x5631ca846590 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08b0a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81af790 con 0x5631ca850e50 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35642/0 (socket says 172.21.15.59:35642) 2024-09-05T12:02:28.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08a6a00640 1 -- 172.21.15.59:0/4254799821 learned_addr learned my addr 172.21.15.59:0/4254799821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f089e000640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca868e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.386+0000 7f08a7400640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca840b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 -- 172.21.15.59:0/4254799821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 msgr2=0x5631ca840b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca840b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 -- 172.21.15.59:0/4254799821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 msgr2=0x5631ca868e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca868e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 -- 172.21.15.59:0/4254799821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ca842030 con 0x5631ca845c70 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a7400640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca840b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f08a6a00640 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08980059f0 tx=0x7f0898005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520ee983d9cf66ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f0897400640 1 -- 172.21.15.59:0/4254799821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898012110 con 0x5631ca845c70 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f0897400640 1 -- 172.21.15.59:0/4254799821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f089802e070 con 0x5631ca845c70 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.387+0000 7f0897400640 1 -- 172.21.15.59:0/4254799821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089801c510 con 0x5631ca845c70 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- 172.21.15.59:0/4254799821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 msgr2=0x5631ca868940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08980059f0 tx=0x7f0898005db0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- 172.21.15.59:0/4254799821 shutdown_connections 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca850e50 0x5631ca840b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca846590 0x5631ca868e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 --2- 172.21.15.59:0/4254799821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca845c70 0x5631ca868940 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- 172.21.15.59:0/4254799821 >> 172.21.15.59:0/4254799821 conn(0x5631ca842200 msgr2=0x5631ca84cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- 172.21.15.59:0/4254799821 shutdown_connections 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- 172.21.15.59:0/4254799821 wait complete. 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.388+0000 7f08b0a03740 1 -- start start 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 Processor -- start 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 0x5631ca8c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca8540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81afe60 con 0x5631ca850e50 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81b12d0 con 0x5631ca845c70 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81b0530 con 0x5631ca846590 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08b0a03740 10 client.0 _ll_register_callbacks cb 0x7f0898009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca8540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.389+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca8540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35648/0 (socket says 172.21.15.59:35648) 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a7400640 1 -- 172.21.15.59:0/1191447208 learned_addr learned my addr 172.21.15.59:0/1191447208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45886/0 (socket says 172.21.15.59:45886) 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f089e000640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 0x5631ca8c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 -- 172.21.15.59:0/1191447208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 msgr2=0x5631ca8c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 0x5631ca8c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 -- 172.21.15.59:0/1191447208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 msgr2=0x5631ca8540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca8540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 -- 172.21.15.59:0/1191447208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ca83d770 con 0x5631ca845c70 2024-09-05T12:02:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f089e000640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 0x5631ca8c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f08a6a00640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f089800a010 tx=0x7f0898010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e34b9ecb66d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f0894c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.390+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898036020 con 0x5631ca845c70 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089802e070 con 0x5631ca845c70 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f089801c6d0 con 0x5631ca845c70 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0898012110 con 0x5631ca845c70 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 10 client.26099 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:28.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5631ca8cb2a0 con 0x5631ca845c70 2024-09-05T12:02:28.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f089802e070 con 0x5631ca845c70 2024-09-05T12:02:28.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0898029890 con 0x5631ca845c70 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 10 client.26099 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 20 client.26099 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca8cb900 0x5631ca8cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 4 client.26099 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.391+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631ca7b3e60 con 0x5631ca8cb900 2024-09-05T12:02:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f089e000640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca8cb900 0x5631ca8cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f089e000640 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca8cb900 0x5631ca8cdcf0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f021f44ac9dcad1 server_cookie=346db4adcb2bfd60 in_seq=0 out_seq=0 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f0894c00640 10 client.26099 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f0894c00640 1 -- 172.21.15.59:0/1191447208 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631ca7b3e60 con 0x5631ca8cb900 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f0894c00640 10 client.26099 handle_command_reply: tid=0 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.392+0000 7f08b0a03740 1 client.26099 shutdown 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 msgr2=0x5631ca8c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f089800a010 tx=0x7f0898010660 comp rx=0 tx=0).stop 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 shutdown_connections 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631ca8cb900 0x5631ca8cdcf0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631ca850e50 0x5631ca8540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631ca846590 0x5631ca8c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 --2- 172.21.15.59:0/1191447208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631ca845c70 0x5631ca8c3c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f0894c00640 0 client.26099 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 >> 172.21.15.59:0/1191447208 conn(0x5631ca842200 msgr2=0x5631ca84dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 shutdown_connections 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 1 -- 172.21.15.59:0/1191447208 wait complete. 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.393+0000 7f08b0a03740 20 client.26099 trim_cache size 0 max 16384 2024-09-05T12:02:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 msgr2=0x7f088c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 0x7f088c0a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005fd0 tx=0x7f08a0009500 comp rx=0 tx=0).stop 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 msgr2=0x7f08a819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0890006010 tx=0x7f0890004040 comp rx=0 tx=0).stop 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 shutdown_connections 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f088c0a2c00 0x7f088c0a50c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8109140 0x7f08a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08a81049c0 0x7f08a819aba0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 --2- 172.21.15.59:0/804959300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08a8104010 0x7f08a819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 >> 172.21.15.59:0/804959300 conn(0x7f08a80ffa50 msgr2=0x7f08a8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 shutdown_connections 2024-09-05T12:02:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:28.394+0000 7f08ade00640 1 -- 172.21.15.59:0/804959300 wait complete. 2024-09-05T12:02:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:28 smithi189 ceph-mon[34772]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:02:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:28 smithi089 ceph-mon[34499]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:02:29.198 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:28 smithi059 ceph-mon[30025]: pgmap v538: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:02:30.238 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:02:30.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:02:30.630 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:31.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:30 smithi059 ceph-mon[30025]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:30 smithi189 ceph-mon[34772]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 -- 172.21.15.59:0/2543833952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390109120 msgr2=0x7fc390110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 --2- 172.21.15.59:0/2543833952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390109120 0x7fc390110ca0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ca0 tx=0x7fc38802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 -- 172.21.15.59:0/2543833952 shutdown_connections 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 --2- 172.21.15.59:0/2543833952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390109120 0x7fc390110ca0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 --2- 172.21.15.59:0/2543833952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3901049a0 0x7fc3901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 --2- 172.21.15.59:0/2543833952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc390103ff0 0x7fc3901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 -- 172.21.15.59:0/2543833952 >> 172.21.15.59:0/2543833952 conn(0x7fc3900ffa90 msgr2=0x7fc390101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 -- 172.21.15.59:0/2543833952 shutdown_connections 2024-09-05T12:02:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.177+0000 7fc397400640 1 -- 172.21.15.59:0/2543833952 wait complete. 2024-09-05T12:02:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.178+0000 7fc397400640 1 Processor -- start 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.178+0000 7fc397400640 1 -- start start 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 0x7fc39019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc390109120 0x7fc39019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901141a0 con 0x7fc390103ff0 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc390114020 con 0x7fc3901049a0 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc397400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc390114320 con 0x7fc390109120 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc394a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc394a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48078/0 (socket says 172.21.15.59:48078) 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc394a00640 1 -- 172.21.15.59:0/3387751795 learned_addr learned my addr 172.21.15.59:0/3387751795 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc387e00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 0x7fc39019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc395400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc390109120 0x7fc39019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc387e00640 1 -- 172.21.15.59:0/3387751795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc390109120 msgr2=0x7fc39019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc387e00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc390109120 0x7fc39019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc387e00640 1 -- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 msgr2=0x7fc39019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.179+0000 7fc387e00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc387e00640 1 -- 172.21.15.59:0/3387751795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39019f590 con 0x7fc3901049a0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc394a00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc387e00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 0x7fc39019ab20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc37800e9f0 tx=0x7fc37800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc38e000640 1 -- 172.21.15.59:0/3387751795 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37800ccc0 con 0x7fc3901049a0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39019f880 con 0x7fc3901049a0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc38e000640 1 -- 172.21.15.59:0/3387751795 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc378005020 con 0x7fc3901049a0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc38e000640 1 -- 172.21.15.59:0/3387751795 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3780049e0 con 0x7fc3901049a0 2024-09-05T12:02:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.180+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901a7160 con 0x7fc3901049a0 2024-09-05T12:02:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:30 smithi089 ceph-mon[34499]: pgmap v539: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.183+0000 7fc399fe8740 1 Processor -- start 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.183+0000 7fc399fe8740 1 -- start start 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.183+0000 7fc38e000640 1 -- 172.21.15.59:0/3387751795 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc378005560 con 0x7fc3901049a0 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832637760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555832637ca0 0x55583263a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555832593290 con 0x55583262cac0 2024-09-05T12:02:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55583253ab40 con 0x555832637ca0 2024-09-05T12:02:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc399fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55583258f250 con 0x55583262d3e0 2024-09-05T12:02:31.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc395400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc387e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555832637ca0 0x55583263a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc394a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832637760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc395400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48082/0 (socket says 172.21.15.59:48082) 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.184+0000 7fc395400640 1 -- 172.21.15.59:0/971918652 learned_addr learned my addr 172.21.15.59:0/971918652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 -- 172.21.15.59:0/971918652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 msgr2=0x555832637760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832637760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 -- 172.21.15.59:0/971918652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555832637ca0 msgr2=0x55583263a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555832637ca0 0x55583263a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 -- 172.21.15.59:0/971918652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558323046a0 con 0x55583262cac0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc395400640 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ca0 tx=0x7fc3880093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e486e47d15edbfae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc385c00640 1 -- 172.21.15.59:0/971918652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388031db0 con 0x55583262cac0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc385c00640 1 -- 172.21.15.59:0/971918652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388036030 con 0x55583262cac0 2024-09-05T12:02:31.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc385c00640 1 -- 172.21.15.59:0/971918652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388004330 con 0x55583262cac0 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.185+0000 7fc38e000640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 0x7fc3740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc38e000640 1 -- 172.21.15.59:0/3387751795 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc3780cf300 con 0x7fc3901049a0 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc394a00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 0x7fc3740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 -- 172.21.15.59:0/971918652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 msgr2=0x55583262cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc388009ca0 tx=0x7fc3880093e0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 -- 172.21.15.59:0/971918652 shutdown_connections 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555832637ca0 0x55583263a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832637760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 --2- 172.21.15.59:0/971918652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x55583262cea0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 -- 172.21.15.59:0/971918652 >> 172.21.15.59:0/971918652 conn(0x555832629050 msgr2=0x555832629430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 -- 172.21.15.59:0/971918652 shutdown_connections 2024-09-05T12:02:31.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.186+0000 7fc399fe8740 1 -- 172.21.15.59:0/971918652 wait complete. 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.187+0000 7fc394a00640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 0x7fc3740a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc380009620 tx=0x7fc380005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.187+0000 7fc399fe8740 1 -- start start 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.187+0000 7fc399fe8740 1 Processor -- start 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 0x5558326348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555832637ca0 0x555832634e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55583263a730 con 0x55583262cac0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55583254dfa0 con 0x55583262d3e0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55583263a8b0 con 0x555832637ca0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc399fe8740 10 client.0 _ll_register_callbacks cb 0x5558322a1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc394a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc395400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555832637ca0 0x555832634e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc394a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48084/0 (socket says 172.21.15.59:48084) 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc394a00640 1 -- 172.21.15.59:0/1461602565 learned_addr learned my addr 172.21.15.59:0/1461602565 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.188+0000 7fc387e00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 0x5558326348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 -- 172.21.15.59:0/1461602565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555832637ca0 msgr2=0x555832634e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555832637ca0 0x555832634e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 -- 172.21.15.59:0/1461602565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 msgr2=0x555832637610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 -- 172.21.15.59:0/1461602565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555832635650 con 0x55583262d3e0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc394a00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc387e00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 0x5558326348c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55583262a3d0 tx=0x7fc3780d2ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6cdda788e508f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc37e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37801c070 con 0x55583262d3e0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc3780a3ed0 con 0x55583262d3e0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc388036030 con 0x55583262d3e0 2024-09-05T12:02:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc37809b230 con 0x55583262d3e0 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc399fe8740 10 client.26111 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.189+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555832646700 con 0x55583262d3e0 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388036030 con 0x55583262d3e0 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc378004bc0 con 0x55583262d3e0 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 10 client.26111 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 20 client.26111 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555832647bc0 0x55583264a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 4 client.26111 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558325a0800 con 0x555832647bc0 2024-09-05T12:02:31.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.190+0000 7fc394a00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555832647bc0 0x55583264a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.191+0000 7fc394a00640 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555832647bc0 0x55583264a020 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74578478ea17f58f server_cookie=7aabf0e7f1fc2123 in_seq=0 out_seq=0 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.191+0000 7fc37e000640 10 client.26111 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc37e000640 1 -- 172.21.15.59:0/1461602565 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558325a0800 con 0x555832647bc0 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc37e000640 10 client.26111 handle_command_reply: tid=0 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 client.26111 shutdown 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 msgr2=0x5558326348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 0x5558326348c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55583262a3d0 tx=0x7fc3780d2ef0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 shutdown_connections 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555832637ca0 0x555832634e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262d3e0 0x5558326348c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555832647bc0 0x55583264a020 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 --2- 172.21.15.59:0/1461602565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55583262cac0 0x555832637610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 >> 172.21.15.59:0/1461602565 conn(0x555832629050 msgr2=0x5558326281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc37e000640 0 client.26111 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 shutdown_connections 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 1 -- 172.21.15.59:0/1461602565 wait complete. 2024-09-05T12:02:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.192+0000 7fc399fe8740 20 client.26111 trim_cache size 0 max 16384 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.210+0000 7fc399fe8740 1 Processor -- start 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 -- start start 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x55583262ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x55583264f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901af020 con 0x555832637ca0 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b18a0 con 0x55583262cac0 2024-09-05T12:02:31.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc399fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901af690 con 0x55583262d3e0 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc387e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc387e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41562/0 (socket says 172.21.15.59:41562) 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc395400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x55583264f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc387e00640 1 -- 172.21.15.59:0/324890193 learned_addr learned my addr 172.21.15.59:0/324890193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.211+0000 7fc394a00640 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x55583262ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 -- 172.21.15.59:0/324890193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 msgr2=0x55583262ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x55583262ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 -- 172.21.15.59:0/324890193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 msgr2=0x55583264f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x55583264f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 -- 172.21.15.59:0/324890193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558326439c0 con 0x55583262d3e0 2024-09-05T12:02:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc387e00640 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc3780980c0 tx=0x7fc378086050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65f625c06ea4ada server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc385200640 1 -- 172.21.15.59:0/324890193 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37801c040 con 0x55583262d3e0 2024-09-05T12:02:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc385200640 1 -- 172.21.15.59:0/324890193 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3780cce50 con 0x55583262d3e0 2024-09-05T12:02:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.212+0000 7fc385200640 1 -- 172.21.15.59:0/324890193 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3780cf100 con 0x55583262d3e0 2024-09-05T12:02:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 -- 172.21.15.59:0/324890193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 msgr2=0x555832633900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc3780980c0 tx=0x7fc378086050 comp rx=0 tx=0).stop 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 -- 172.21.15.59:0/324890193 shutdown_connections 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x55583264f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832633900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 --2- 172.21.15.59:0/324890193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x55583262ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 -- 172.21.15.59:0/324890193 >> 172.21.15.59:0/324890193 conn(0x555832629050 msgr2=0x5558326331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 -- 172.21.15.59:0/324890193 shutdown_connections 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.213+0000 7fc399fe8740 1 -- 172.21.15.59:0/324890193 wait complete. 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 -- start start 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 Processor -- start 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832631db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x5558326322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901afd60 con 0x555832637ca0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b11d0 con 0x55583262cac0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b0430 con 0x55583262d3e0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.214+0000 7fc399fe8740 10 client.0 _ll_register_callbacks cb 0x555832635820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45090/0 (socket says 172.21.15.59:45090) 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 -- 172.21.15.59:0/2985674555 learned_addr learned my addr 172.21.15.59:0/2985674555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc387e00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832631db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc395400640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x5558326322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 -- 172.21.15.59:0/2985674555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 msgr2=0x555832631db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832631db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 -- 172.21.15.59:0/2985674555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 msgr2=0x5558326322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x5558326322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 -- 172.21.15.59:0/2985674555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558326b18c0 con 0x55583262cac0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc394a00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55583264a890 tx=0x7fc3800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517caeeddb8fb4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.215+0000 7fc37e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004110 con 0x55583262cac0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3780cce50 con 0x55583262cac0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc3800042b0 con 0x55583262cac0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc38003c050 con 0x55583262cac0 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 10 client.26117 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5558326b2ba0 con 0x55583262cac0 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3780cce50 con 0x55583262cac0 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc38001f8d0 con 0x55583262cac0 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 10 client.26117 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 20 client.26117 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5558326b3320 0x5558326b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 4 client.26117 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:02:31.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.216+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55583259ad10 con 0x5558326b3320 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.217+0000 7fc387e00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5558326b3320 0x5558326b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.217+0000 7fc387e00640 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5558326b3320 0x5558326b5740 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a793cb4511c56e5 server_cookie=5ef40f02563dd1d in_seq=0 out_seq=0 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.217+0000 7fc37e000640 10 client.26117 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.218+0000 7fc37e000640 1 -- 172.21.15.59:0/2985674555 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55583259ad10 con 0x5558326b3320 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.218+0000 7fc37e000640 10 client.26117 handle_command_reply: tid=0 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.218+0000 7fc399fe8740 1 client.26117 shutdown 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 msgr2=0x555832631870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55583264a890 tx=0x7fc3800046c0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 shutdown_connections 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5558326b3320 0x5558326b5740 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555832637ca0 0x5558326322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55583262d3e0 0x555832631db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 --2- 172.21.15.59:0/2985674555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55583262cac0 0x555832631870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 >> 172.21.15.59:0/2985674555 conn(0x555832629050 msgr2=0x555832633a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc37e000640 0 client.26117 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 shutdown_connections 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 1 -- 172.21.15.59:0/2985674555 wait complete. 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.219+0000 7fc399fe8740 20 client.26117 trim_cache size 0 max 16384 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "2e34184e-bd04-4ecd-ba76-5aca83b1c859", 2024-09-05T12:02:31.224 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:02:31.225 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 msgr2=0x7fc3740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 0x7fc3740a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc380009620 tx=0x7fc380005f90 comp rx=0 tx=0).stop 2024-09-05T12:02:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 msgr2=0x7fc39019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 0x7fc39019ab20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc37800e9f0 tx=0x7fc37800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 shutdown_connections 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc3740a2c00 0x7fc3740a50c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc390109120 0x7fc39019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc3901049a0 0x7fc39019ab20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 --2- 172.21.15.59:0/3387751795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc390103ff0 0x7fc39019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 >> 172.21.15.59:0/3387751795 conn(0x7fc3900ffa90 msgr2=0x7fc3901005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 shutdown_connections 2024-09-05T12:02:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:31.220+0000 7fc397400640 1 -- 172.21.15.59:0/3387751795 wait complete. 2024-09-05T12:02:31.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:02:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 -- 172.21.15.59:0/3875437101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074590 msgr2=0x7f7108074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 --2- 172.21.15.59:0/3875437101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074590 0x7f7108074970 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009ca0 tx=0x7f70f802a950 comp rx=0 tx=0).stop 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 -- 172.21.15.59:0/3875437101 shutdown_connections 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 --2- 172.21.15.59:0/3875437101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108112b90 0x7f7108114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 --2- 172.21.15.59:0/3875437101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108074eb0 0x7f7108112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 --2- 172.21.15.59:0/3875437101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074590 0x7f7108074970 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 -- 172.21.15.59:0/3875437101 >> 172.21.15.59:0/3875437101 conn(0x7f71080fd8c0 msgr2=0x7f71080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 -- 172.21.15.59:0/3875437101 shutdown_connections 2024-09-05T12:02:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.979+0000 7f710ce00640 1 -- 172.21.15.59:0/3875437101 wait complete. 2024-09-05T12:02:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.980+0000 7f710ce00640 1 Processor -- start 2024-09-05T12:02:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.980+0000 7f710ce00640 1 -- start start 2024-09-05T12:02:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108074590 0x7f71081a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108112b90 0x7f710819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71081185c0 con 0x7f7108074eb0 2024-09-05T12:02:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7108118440 con 0x7f7108112b90 2024-09-05T12:02:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f710ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7108118740 con 0x7f7108074590 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f70fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f70fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48114/0 (socket says 172.21.15.59:48114) 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f70fd400640 1 -- 172.21.15.59:0/2224861754 learned_addr learned my addr 172.21.15.59:0/2224861754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f7105e00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108074590 0x7f71081a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.981+0000 7f7106800640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108112b90 0x7f710819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 -- 172.21.15.59:0/2224861754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108074590 msgr2=0x7f71081a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108074590 0x7f71081a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 -- 172.21.15.59:0/2224861754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108112b90 msgr2=0x7f710819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108112b90 0x7f710819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 -- 172.21.15.59:0/2224861754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710819efe0 con 0x7f7108074eb0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70fd400640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70ff400640 1 -- 172.21.15.59:0/2224861754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000cd80 con 0x7f7108074eb0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70ff400640 1 -- 172.21.15.59:0/2224861754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0010910 con 0x7f7108074eb0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710819f2d0 con 0x7f7108074eb0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.982+0000 7f70ff400640 1 -- 172.21.15.59:0/2224861754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0018990 con 0x7f7108074eb0 2024-09-05T12:02:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.983+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081ab5c0 con 0x7f7108074eb0 2024-09-05T12:02:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.985+0000 7f70ff400640 1 -- 172.21.15.59:0/2224861754 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70f0010ac0 con 0x7f7108074eb0 2024-09-05T12:02:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 Processor -- start 2024-09-05T12:02:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 -- start start 2024-09-05T12:02:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb5fd70 0x55adafb62160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55adafabb3e0 con 0x55adafb554b0 2024-09-05T12:02:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55adafa62d80 con 0x55adafb54b90 2024-09-05T12:02:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f710fa1a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55adafab73a0 con 0x55adafb5fd70 2024-09-05T12:02:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f7105e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f7106800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.986+0000 7f7106800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45118/0 (socket says 172.21.15.59:45118) 2024-09-05T12:02:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 -- 172.21.15.59:0/1330066371 learned_addr learned my addr 172.21.15.59:0/1330066371 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f70fd400640 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb5fd70 0x55adafb62160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 -- 172.21.15.59:0/1330066371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb5fd70 msgr2=0x55adafb62160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb5fd70 0x55adafb62160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 -- 172.21.15.59:0/1330066371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 msgr2=0x55adafb5f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb5f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 -- 172.21.15.59:0/1330066371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adafa81ae0 con 0x55adafb54b90 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f7106800640 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7100009ba0 tx=0x7f7100011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da39365a5b5db7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.987+0000 7f70ff400640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 0x7f70ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f70ff400640 1 -- 172.21.15.59:0/2224861754 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f70f00cf7c0 con 0x7f7108074eb0 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f70f7400640 1 -- 172.21.15.59:0/1330066371 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100004230 con 0x55adafb54b90 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f70f7400640 1 -- 172.21.15.59:0/1330066371 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100009e10 con 0x55adafb54b90 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f70f7400640 1 -- 172.21.15.59:0/1330066371 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100004d70 con 0x55adafb54b90 2024-09-05T12:02:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f7105e00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 0x7f70ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f710fa1a740 1 -- 172.21.15.59:0/1330066371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 msgr2=0x55adafb54f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f7105e00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 0x7f70ec0a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f710819f9f0 tx=0x7f70f8005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.988+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7100009ba0 tx=0x7f7100011810 comp rx=0 tx=0).stop 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 -- 172.21.15.59:0/1330066371 shutdown_connections 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb5fd70 0x55adafb62160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb5f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1330066371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb54f70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 -- 172.21.15.59:0/1330066371 >> 172.21.15.59:0/1330066371 conn(0x55adafb51120 msgr2=0x55adafb51500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 -- 172.21.15.59:0/1330066371 shutdown_connections 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 -- 172.21.15.59:0/1330066371 wait complete. 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 -- start start 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.989+0000 7f710fa1a740 1 Processor -- start 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb5d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 0x55adafb5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55adafb629e0 con 0x55adafb5fd70 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55adafa760f0 con 0x55adafb54b90 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55adafb62b60 con 0x55adafb554b0 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f710fa1a740 10 client.0 _ll_register_callbacks cb 0x55adafa679e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f7105e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f70fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb5d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f7106800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 0x55adafb5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f7105e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45132/0 (socket says 172.21.15.59:45132) 2024-09-05T12:02:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f70fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb5d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41602/0 (socket says 172.21.15.59:41602) 2024-09-05T12:02:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.990+0000 7f7105e00640 1 -- 172.21.15.59:0/2623707665 learned_addr learned my addr 172.21.15.59:0/2623707665 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 -- 172.21.15.59:0/2623707665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 msgr2=0x55adafb5d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb5d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 -- 172.21.15.59:0/2623707665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 msgr2=0x55adafb5d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 0x55adafb5d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 -- 172.21.15.59:0/2623707665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adaf82c3c0 con 0x55adafb54b90 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7106800640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 0x55adafb5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f7105e00640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70f8035000 tx=0x7f70f80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef96b4d0601abdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.991+0000 7f70ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8055060 con 0x55adafb54b90 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100009e10 con 0x55adafb54b90 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f70f8003fd0 con 0x55adafb54b90 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f70f8034040 con 0x55adafb54b90 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f710fa1a740 10 client.26129 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55adafb6e740 con 0x55adafb54b90 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.992+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100009e10 con 0x55adafb54b90 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f70f80308f0 con 0x55adafb54b90 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 10 client.26129 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 20 client.26129 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafb6fc00 0x55adafb72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 4 client.26129 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adafac8950 con 0x55adafb6fc00 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f70fd400640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafb6fc00 0x55adafb72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f70fd400640 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafb6fc00 0x55adafb72010 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31b7d32d7060ea7 server_cookie=a7d7169e14c21d06 in_seq=0 out_seq=0 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.993+0000 7f70ebe00640 10 client.26129 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.994+0000 7f70ebe00640 1 -- 172.21.15.59:0/2623707665 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adafac8950 con 0x55adafb6fc00 2024-09-05T12:02:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.994+0000 7f70ebe00640 10 client.26129 handle_command_reply: tid=0 2024-09-05T12:02:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.994+0000 7f710fa1a740 1 client.26129 shutdown 2024-09-05T12:02:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.994+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 msgr2=0x55adafb5cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.994+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70f8035000 tx=0x7f70f80095c0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 shutdown_connections 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafb6fc00 0x55adafb72010 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb5fd70 0x55adafb5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb5d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 --2- 172.21.15.59:0/2623707665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb54b90 0x55adafb5cc80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 >> 172.21.15.59:0/2623707665 conn(0x55adafb51120 msgr2=0x55adafb56e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f70ebe00640 0 client.26129 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 shutdown_connections 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 1 -- 172.21.15.59:0/2623707665 wait complete. 2024-09-05T12:02:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:32.995+0000 7f710fa1a740 20 client.26129 trim_cache size 0 max 16384 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 Processor -- start 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 -- start start 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb54b90 0x55adafb57640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b34a0 con 0x55adafb554b0 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b5d20 con 0x55adafb5fd70 2024-09-05T12:02:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.013+0000 7f710fa1a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b3b10 con 0x55adafb54b90 2024-09-05T12:02:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f70fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7105e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb54b90 0x55adafb57640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f70fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48136/0 (socket says 172.21.15.59:48136) 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f70fd400640 1 -- 172.21.15.59:0/625105310 learned_addr learned my addr 172.21.15.59:0/625105310 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 -- 172.21.15.59:0/625105310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb54b90 msgr2=0x55adafb57640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb54b90 0x55adafb57640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 -- 172.21.15.59:0/625105310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 msgr2=0x55adafb500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f7106800640 1 -- 172.21.15.59:0/625105310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adafb5f5f0 con 0x55adafb5fd70 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.014+0000 7f70fd400640 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f7106800640 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb778e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71000057c0 tx=0x7f7100009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea31b5b72266e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f70f6a00640 1 -- 172.21.15.59:0/625105310 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710001e040 con 0x55adafb5fd70 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f70f6a00640 1 -- 172.21.15.59:0/625105310 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100029070 con 0x55adafb5fd70 2024-09-05T12:02:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f70f6a00640 1 -- 172.21.15.59:0/625105310 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100003ea0 con 0x55adafb5fd70 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 -- 172.21.15.59:0/625105310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 msgr2=0x55adafb778e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb778e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71000057c0 tx=0x7f7100009780 comp rx=0 tx=0).stop 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 -- 172.21.15.59:0/625105310 shutdown_connections 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb778e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb554b0 0x55adafb500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.015+0000 7f710fa1a740 1 --2- 172.21.15.59:0/625105310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb54b90 0x55adafb57640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.016+0000 7f710fa1a740 1 -- 172.21.15.59:0/625105310 >> 172.21.15.59:0/625105310 conn(0x55adafb51120 msgr2=0x55adafb4f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.016+0000 7f710fa1a740 1 -- 172.21.15.59:0/625105310 shutdown_connections 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.016+0000 7f710fa1a740 1 -- 172.21.15.59:0/625105310 wait complete. 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.016+0000 7f710fa1a740 1 -- start start 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.016+0000 7f710fa1a740 1 Processor -- start 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb54b90 0x55adafb63d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b41e0 con 0x55adafb54b90 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b5650 con 0x55adafb5fd70 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b48b0 con 0x55adafb554b0 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f710fa1a740 10 client.0 _ll_register_callbacks cb 0x7f70f8003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f7105e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb54b90 0x55adafb63d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41624/0 (socket says 172.21.15.59:41624) 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 -- 172.21.15.59:0/1791637660 learned_addr learned my addr 172.21.15.59:0/1791637660 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f7106800640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 -- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 msgr2=0x55adafb647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 -- 172.21.15.59:0/1791637660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb54b90 msgr2=0x55adafb63d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb54b90 0x55adafb63d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.017+0000 7f70fd400640 1 -- 172.21.15.59:0/1791637660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adafbd9b00 con 0x55adafb554b0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70fd400640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70f00980c0 tx=0x7f70f0018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0e9ed30d82af0e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00ccf60 con 0x55adafb554b0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f70f00cf3f0 con 0x55adafb554b0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100029070 con 0x55adafb554b0 2024-09-05T12:02:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f70f00ba350 con 0x55adafb554b0 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f710fa1a740 10 client.26059 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55adafbdadd0 con 0x55adafb554b0 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.018+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100029070 con 0x55adafb554b0 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f70f802f5f0 con 0x55adafb554b0 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 10 client.26059 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 20 client.26059 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafbdb550 0x55adafbdd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 4 client.26059 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:02:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adafac2e60 con 0x55adafbdb550 2024-09-05T12:02:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.019+0000 7f7105e00640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafbdb550 0x55adafbdd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f7105e00640 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafbdb550 0x55adafbdd970 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d871012864572f8 server_cookie=b527306eeb1f9418 in_seq=0 out_seq=0 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f70ebe00640 10 client.26059 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f70ebe00640 1 -- 172.21.15.59:0/1791637660 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55adafac2e60 con 0x55adafbdb550 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f70ebe00640 10 client.26059 handle_command_reply: tid=0 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 client.26059 shutdown 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 msgr2=0x55adafb642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70f00980c0 tx=0x7f70f0018420 comp rx=0 tx=0).stop 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 shutdown_connections 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55adafb5fd70 0x55adafb647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adafb554b0 0x55adafb642b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55adafbdb550 0x55adafbdd970 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 --2- 172.21.15.59:0/1791637660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55adafb54b90 0x55adafb63d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 >> 172.21.15.59:0/1791637660 conn(0x55adafb51120 msgr2=0x55adafb501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f70ebe00640 0 client.26059 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 shutdown_connections 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.020+0000 7f710fa1a740 1 -- 172.21.15.59:0/1791637660 wait complete. 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710fa1a740 20 client.26059 trim_cache size 0 max 16384 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "2e34184e-bd04-4ecd-ba76-5aca83b1c859": { 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "2e34184e-bd04-4ecd-ba76-5aca83b1c859", 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:02:33.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 msgr2=0x7f70ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 0x7f70ec0a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f710819f9f0 tx=0x7f70f8005d80 comp rx=0 tx=0).stop 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 msgr2=0x7f71081a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 shutdown_connections 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f70ec0a2b80 0x7f70ec0a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7108112b90 0x7f710819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7108074eb0 0x7f71081a4c30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 --2- 172.21.15.59:0/2224861754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7108074590 0x7f71081a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 >> 172.21.15.59:0/2224861754 conn(0x7f71080fd8c0 msgr2=0x7f71080fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 shutdown_connections 2024-09-05T12:02:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:33.021+0000 7f710ce00640 1 -- 172.21.15.59:0/2224861754 wait complete. 2024-09-05T12:02:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:32 smithi189 ceph-mon[34772]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:32 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:02:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:32 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:32 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:02:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:32 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:32 smithi089 ceph-mon[34499]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:32 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:02:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:32 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:32 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:02:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:32 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:32 smithi059 ceph-mon[30025]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:32 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:02:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:32 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:32 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:02:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:32 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:02:33.820 INFO:tasks.cephfs.filesystem:scrub status for tag:2e34184e-bd04-4ecd-ba76-5aca83b1c859 - {'path': '/', 'tag': '2e34184e-bd04-4ecd-ba76-5aca83b1c859', 'options': 'recursive,force'} 2024-09-05T12:02:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:34 smithi089 ceph-mon[34499]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:34 smithi059 ceph-mon[30025]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:34 smithi189 ceph-mon[34772]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:35 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:02:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:35 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:02:36.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:35 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:02:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:36 smithi089 ceph-mon[34499]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:37.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:36 smithi059 ceph-mon[30025]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:36 smithi189 ceph-mon[34772]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:02:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:38 smithi089 ceph-mon[34499]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T12:02:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:38 smithi059 ceph-mon[30025]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T12:02:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:38 smithi189 ceph-mon[34772]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 853 B/s wr, 3 op/s 2024-09-05T12:02:39.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:02:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 -- 172.21.15.59:0/1479852646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41019c0 msgr2=0x7f6df4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 --2- 172.21.15.59:0/1479852646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41019c0 0x7f6df4101e20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009930 tx=0x7f6ddc02a440 comp rx=0 tx=0).stop 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 -- 172.21.15.59:0/1479852646 shutdown_connections 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 --2- 172.21.15.59:0/1479852646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 0x7f6df410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 --2- 172.21.15.59:0/1479852646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41019c0 0x7f6df4101e20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 --2- 172.21.15.59:0/1479852646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41079c0 0x7f6df4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 -- 172.21.15.59:0/1479852646 >> 172.21.15.59:0/1479852646 conn(0x7f6df40fd740 msgr2=0x7f6df40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 -- 172.21.15.59:0/1479852646 shutdown_connections 2024-09-05T12:02:40.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.873+0000 7f6df8a00640 1 -- 172.21.15.59:0/1479852646 wait complete. 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.874+0000 7f6df8a00640 1 Processor -- start 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.874+0000 7f6df8a00640 1 -- start start 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 0x7f6df41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 0x7f6df4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111e10 con 0x7f6df41079c0 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111c90 con 0x7f6df4102360 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111f90 con 0x7f6df41019c0 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 0x7f6df41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49744/0 (socket says 172.21.15.59:49744) 2024-09-05T12:02:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6df3200640 1 -- 172.21.15.59:0/2298546039 learned_addr learned my addr 172.21.15.59:0/2298546039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.875+0000 7f6debe00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 0x7f6df4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 -- 172.21.15.59:0/2298546039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 msgr2=0x7f6df4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 0x7f6df4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 -- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 msgr2=0x7f6df419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 -- 172.21.15.59:0/2298546039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df419d280 con 0x7f6df41019c0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df3200640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df2800640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 0x7f6df41982d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d6d0 tx=0x7f6de400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4017070 con 0x7f6df41019c0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df419d570 con 0x7f6df41019c0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4009820 con 0x7f6df41019c0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.876+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004cd0 con 0x7f6df41019c0 2024-09-05T12:02:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.877+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a4e50 con 0x7f6df41019c0 2024-09-05T12:02:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.878+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df4101e20 con 0x7f6df41019c0 2024-09-05T12:02:40.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.880+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6de40099d0 con 0x7f6df41019c0 2024-09-05T12:02:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.881+0000 7f6deb400640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 0x7f6dd80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.882+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6de40cee50 con 0x7f6df41019c0 2024-09-05T12:02:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.882+0000 7f6debe00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 0x7f6dd80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:40.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.883+0000 7f6debe00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 0x7f6dd80a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:40.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:40.883+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de409b370 con 0x7f6df41019c0 2024-09-05T12:02:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.106+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6df4109f60 con 0x7f6df41019c0 2024-09-05T12:02:41.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:40 smithi059 ceph-mon[30025]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.107+0000 7f6deb400640 1 -- 172.21.15.59:0/2298546039 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f6de409f670 con 0x7f6df41019c0 2024-09-05T12:02:41.109 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:02:41.109 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:02:41.110 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:02:41.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 msgr2=0x7f6dd80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:41.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 0x7f6dd80a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc000bc0 comp rx=0 tx=0).stop 2024-09-05T12:02:41.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 msgr2=0x7f6df41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 0x7f6df41982d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d6d0 tx=0x7f6de400db90 comp rx=0 tx=0).stop 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 shutdown_connections 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6dd80a2c00 0x7f6dd80a50c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6df41079c0 0x7f6df419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6df4102360 0x7f6df4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 --2- 172.21.15.59:0/2298546039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41019c0 0x7f6df41982d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.110+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 >> 172.21.15.59:0/2298546039 conn(0x7f6df40fd740 msgr2=0x7f6df40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.111+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 shutdown_connections 2024-09-05T12:02:41.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:41.111+0000 7f6df8a00640 1 -- 172.21.15.59:0/2298546039 wait complete. 2024-09-05T12:02:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:40 smithi089 ceph-mon[34499]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:41.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:40 smithi189 ceph-mon[34772]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:41.934 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 4cea4795 --release --await' 2024-09-05T12:02:41.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 4cea4795 --release --await 2024-09-05T12:02:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2298546039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:02:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:02:42.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:41 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:02:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2298546039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:02:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:02:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:41 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:02:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2298546039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:02:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:02:42.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:41 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:02:42.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 -- 172.21.15.59:0/1175972136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c112a50 msgr2=0x7fb23c114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 --2- 172.21.15.59:0/1175972136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c112a50 0x7fb23c114e40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb234009ca0 tx=0x7fb23402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 -- 172.21.15.59:0/1175972136 shutdown_connections 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 --2- 172.21.15.59:0/1175972136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c112a50 0x7fb23c114e40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 --2- 172.21.15.59:0/1175972136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 0x7fb23c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 --2- 172.21.15.59:0/1175972136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c076720 0x7fb23c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 -- 172.21.15.59:0/1175972136 >> 172.21.15.59:0/1175972136 conn(0x7fb23c0ffa50 msgr2=0x7fb23c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 -- 172.21.15.59:0/1175972136 shutdown_connections 2024-09-05T12:02:42.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.899+0000 7fb244600640 1 -- 172.21.15.59:0/1175972136 wait complete. 2024-09-05T12:02:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 Processor -- start 2024-09-05T12:02:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 -- start start 2024-09-05T12:02:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 0x7fb23c1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 0x7fb23c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1185e0 con 0x7fb23c076720 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c118460 con 0x7fb23c077040 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.900+0000 7fb244600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c118760 con 0x7fb23c112a50 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49762/0 (socket says 172.21.15.59:49762) 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb242600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 0x7fb23c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 -- 172.21.15.59:0/4238712520 learned_addr learned my addr 172.21.15.59:0/4238712520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:42.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241200640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 0x7fb23c1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 -- 172.21.15.59:0/4238712520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 msgr2=0x7fb23c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 0x7fb23c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 -- 172.21.15.59:0/4238712520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 msgr2=0x7fb23c1a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 0x7fb23c1a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb241c00640 1 -- 172.21.15.59:0/4238712520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c19f040 con 0x7fb23c076720 2024-09-05T12:02:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.901+0000 7fb242600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 0x7fb23c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb241c00640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb23000e9f0 tx=0x7fb23000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb23a000640 1 -- 172.21.15.59:0/4238712520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23000cd80 con 0x7fb23c076720 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb23a000640 1 -- 172.21.15.59:0/4238712520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230010910 con 0x7fb23c076720 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb23a000640 1 -- 172.21.15.59:0/4238712520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230018990 con 0x7fb23c076720 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c19f330 con 0x7fb23c076720 2024-09-05T12:02:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.902+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1ab620 con 0x7fb23c076720 2024-09-05T12:02:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb23a000640 1 -- 172.21.15.59:0/4238712520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb230010ac0 con 0x7fb23c076720 2024-09-05T12:02:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 Processor -- start 2024-09-05T12:02:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 -- start start 2024-09-05T12:02:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100080b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x561000815fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x5610008188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561000771650 con 0x56100080b310 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561000718f10 con 0x56100080bc30 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.905+0000 7fb247063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56100076d610 con 0x5610008164f0 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb242600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100080b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x561000815fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x5610008188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x561000815fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49018/0 (socket says 172.21.15.59:49018) 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241c00640 1 -- 172.21.15.59:0/1342414403 learned_addr learned my addr 172.21.15.59:0/1342414403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb242600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100080b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49770/0 (socket says 172.21.15.59:49770) 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 -- 172.21.15.59:0/1342414403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 msgr2=0x561000815fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x561000815fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 -- 172.21.15.59:0/1342414403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 msgr2=0x56100080b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100080b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.906+0000 7fb241200640 1 -- 172.21.15.59:0/1342414403 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561000480580 con 0x5610008164f0 2024-09-05T12:02:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb241200640 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x5610008188e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb23c1a0100 tx=0x7fb228011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7d72c59a014bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb22ea00640 1 -- 172.21.15.59:0/1342414403 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228004050 con 0x5610008164f0 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb22ea00640 1 -- 172.21.15.59:0/1342414403 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb228005500 con 0x5610008164f0 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb23a000640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 0x7fb2240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb22ea00640 1 -- 172.21.15.59:0/1342414403 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228004d40 con 0x5610008164f0 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb23a000640 1 -- 172.21.15.59:0/4238712520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb2300cf720 con 0x7fb23c076720 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.907+0000 7fb242600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 0x7fb2240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 -- 172.21.15.59:0/1342414403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 msgr2=0x5610008188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x5610008188e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb23c1a0100 tx=0x7fb228011810 comp rx=0 tx=0).stop 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 -- 172.21.15.59:0/1342414403 shutdown_connections 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x5610008188e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x561000815fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 --2- 172.21.15.59:0/1342414403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100080b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 -- 172.21.15.59:0/1342414403 >> 172.21.15.59:0/1342414403 conn(0x5610008078a0 msgr2=0x561000807c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 -- 172.21.15.59:0/1342414403 shutdown_connections 2024-09-05T12:02:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb242600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 0x7fb2240a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb234005e30 tx=0x7fb234005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.908+0000 7fb247063740 1 -- 172.21.15.59:0/1342414403 wait complete. 2024-09-05T12:02:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 -- start start 2024-09-05T12:02:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 Processor -- start 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 0x5610008133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000813e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5610008190c0 con 0x56100080bc30 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56100072c350 con 0x5610008164f0 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561000819240 con 0x56100080b310 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.909+0000 7fb247063740 10 client.0 _ll_register_callbacks cb 0x56100074f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb242600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000813e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 0x5610008133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49774/0 (socket says 172.21.15.59:49774) 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 -- 172.21.15.59:0/4266744073 learned_addr learned my addr 172.21.15.59:0/4266744073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 -- 172.21.15.59:0/4266744073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 msgr2=0x5610008133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 0x5610008133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 -- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 msgr2=0x561000813e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000813e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 -- 172.21.15.59:0/4266744073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100054a960 con 0x56100080bc30 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241c00640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 0x5610008133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb241200640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb228009730 tx=0x7fb2280056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7531f89872d437b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.910+0000 7fb223400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:02:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228003a00 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb228005500 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb228003ba0 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb228033050 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb247063740 10 client.16140 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561000824ca0 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.911+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb228005500 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb228023430 con 0x56100080bc30 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 10 client.16140 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 10 client.16140 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 20 client.16140 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000825ff0 0x5610008283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 4 client.16140 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56100077ebc0 con 0x561000825ff0 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.912+0000 7fb241c00640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000825ff0 0x5610008283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.913+0000 7fb241c00640 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000825ff0 0x5610008283e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760212692c07104a server_cookie=eddbac722ac1f512 in_seq=0 out_seq=0 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.913+0000 7fb223400640 10 client.16140 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.913+0000 7fb223400640 1 -- 172.21.15.59:0/4266744073 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56100077ebc0 con 0x561000825ff0 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.913+0000 7fb223400640 10 client.16140 handle_command_reply: tid=0 2024-09-05T12:02:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 client.16140 shutdown 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 msgr2=0x5610008138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb228009730 tx=0x7fb2280056b0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 shutdown_connections 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000813e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000825ff0 0x5610008283e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080bc30 0x5610008138f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 --2- 172.21.15.59:0/4266744073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080b310 0x5610008133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 >> 172.21.15.59:0/4266744073 conn(0x5610008078a0 msgr2=0x56100080d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb223400640 0 client.16140 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 shutdown_connections 2024-09-05T12:02:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 1 -- 172.21.15.59:0/4266744073 wait complete. 2024-09-05T12:02:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.914+0000 7fb247063740 20 client.16140 trim_cache size 0 max 16384 2024-09-05T12:02:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 Processor -- start 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 -- start start 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 0x56100082e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000810610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b35a0 con 0x56100080b310 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b5e20 con 0x5610008164f0 2024-09-05T12:02:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.932+0000 7fb247063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b3c10 con 0x56100080bc30 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49788/0 (socket says 172.21.15.59:49788) 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 -- 172.21.15.59:0/4204925023 learned_addr learned my addr 172.21.15.59:0/4204925023 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb242600640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000810610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241200640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 0x56100082e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 -- 172.21.15.59:0/4204925023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 msgr2=0x56100082e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 0x56100082e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 -- 172.21.15.59:0/4204925023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 msgr2=0x561000810610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000810610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 -- 172.21.15.59:0/4204925023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610008076d0 con 0x56100080b310 2024-09-05T12:02:42.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241200640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 0x56100082e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb241c00640 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb230098540 tx=0x7fb2300ccc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dad22300c8ccebe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.933+0000 7fb22e000640 1 -- 172.21.15.59:0/4204925023 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23009bbc0 con 0x56100080b310 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb22e000640 1 -- 172.21.15.59:0/4204925023 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23009fec0 con 0x56100080b310 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb22e000640 1 -- 172.21.15.59:0/4204925023 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300d1be0 con 0x56100080b310 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 -- 172.21.15.59:0/4204925023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 msgr2=0x56100082db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb230098540 tx=0x7fb2300ccc80 comp rx=0 tx=0).stop 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 -- 172.21.15.59:0/4204925023 shutdown_connections 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610008164f0 0x561000810610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56100080bc30 0x56100082e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 --2- 172.21.15.59:0/4204925023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100082db70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 -- 172.21.15.59:0/4204925023 >> 172.21.15.59:0/4204925023 conn(0x5610008078a0 msgr2=0x56100080d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 -- 172.21.15.59:0/4204925023 shutdown_connections 2024-09-05T12:02:42.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.934+0000 7fb247063740 1 -- 172.21.15.59:0/4204925023 wait complete. 2024-09-05T12:02:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.935+0000 7fb247063740 1 -- start start 2024-09-05T12:02:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.935+0000 7fb247063740 1 Processor -- start 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.935+0000 7fb247063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.935+0000 7fb247063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x56100081ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.935+0000 7fb247063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x56100088fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb247063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b42e0 con 0x56100080b310 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb247063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b5750 con 0x56100080bc30 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb247063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c1b49b0 con 0x5610008164f0 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb247063740 10 client.0 _ll_register_callbacks cb 0x7fb230005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49792/0 (socket says 172.21.15.59:49792) 2024-09-05T12:02:42.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 -- 172.21.15.59:0/3742223566 learned_addr learned my addr 172.21.15.59:0/3742223566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241200640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x56100081ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb242600640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x56100088fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 -- 172.21.15.59:0/3742223566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 msgr2=0x56100088fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x56100088fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 -- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 msgr2=0x56100081ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x56100081ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.936+0000 7fb241c00640 1 -- 172.21.15.59:0/3742223566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561000828940 con 0x56100080b310 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb241c00640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b1040 tx=0x7fb2300ccec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8697df9c9ff927 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb223400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300b3020 con 0x56100080b310 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb2300d1be0 con 0x56100080b310 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23009fec0 con 0x56100080b310 2024-09-05T12:02:42.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb2300a4780 con 0x56100080b310 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb247063740 10 client.16146 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5610008911c0 con 0x56100080b310 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.937+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23009fec0 con 0x56100080b310 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb23009b010 con 0x56100080b310 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 10 client.16146 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 10 client.16146 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 20 client.16146 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000891880 0x561000893ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 4 client.16146 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4cea4795", "release": true, "await": true}] 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4cea4795", "release": true, "await": true}) -- 0x5610007790d0 con 0x561000891880 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb241200640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000891880 0x561000893ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:42.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb241200640 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000891880 0x561000893ca0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9caa1eae5719fb server_cookie=116f97e8a4fe363e in_seq=0 out_seq=0 2024-09-05T12:02:42.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.938+0000 7fb223400640 10 client.16146 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb223400640 1 -- 172.21.15.59:0/3742223566 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5610007790d0 con 0x561000891880 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb223400640 10 client.16146 handle_command_reply: tid=0 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 client.16146 shutdown 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 msgr2=0x56100081a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b1040 tx=0x7fb2300ccec0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 shutdown_connections 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610008164f0 0x56100088fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56100080bc30 0x56100081ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561000891880 0x561000893ca0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 --2- 172.21.15.59:0/3742223566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56100080b310 0x56100081a900 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 >> 172.21.15.59:0/3742223566 conn(0x5610008078a0 msgr2=0x561000814a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb223400640 0 client.16146 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 shutdown_connections 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.940+0000 7fb247063740 1 -- 172.21.15.59:0/3742223566 wait complete. 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb247063740 20 client.16146 trim_cache size 0 max 16384 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 20, 2024-09-05T12:02:42.946 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "4cea4795": { 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 20, 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:02:42.947 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:42.948 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 msgr2=0x7fb2240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 0x7fb2240a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb234005e30 tx=0x7fb234005d80 comp rx=0 tx=0).stop 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 msgr2=0x7fb23c1a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb23000e9f0 tx=0x7fb23000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 shutdown_connections 2024-09-05T12:02:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2240a2c00 0x7fb2240a50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23c112a50 0x7fb23c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb23c077040 0x7fb23c1a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 --2- 172.21.15.59:0/4238712520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb23c076720 0x7fb23c1a4750 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.941+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 >> 172.21.15.59:0/4238712520 conn(0x7fb23c0ffa50 msgr2=0x7fb23c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.942+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 shutdown_connections 2024-09-05T12:02:42.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:42.942+0000 7fb244600640 1 -- 172.21.15.59:0/4238712520 wait complete. 2024-09-05T12:02:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:42 smithi089 ceph-mon[34499]: pgmap v545: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:43.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:42 smithi059 ceph-mon[30025]: pgmap v545: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:42 smithi189 ceph-mon[34772]: pgmap v545: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:02:43.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4cea4795', total seconds elapsed: 33.9 2024-09-05T12:02:43.807 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:02:43.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:02:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:44.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 -- 172.21.15.59:0/2807925718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481049a0 msgr2=0x7f58481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 --2- 172.21.15.59:0/2807925718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481049a0 0x7f58481089f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f583402a410 comp rx=0 tx=0).stop 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 -- 172.21.15.59:0/2807925718 shutdown_connections 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 --2- 172.21.15.59:0/2807925718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 0x7f5848110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 --2- 172.21.15.59:0/2807925718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58481049a0 0x7f58481089f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 --2- 172.21.15.59:0/2807925718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5848103ff0 0x7f58481043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.727+0000 7f584ea00640 1 -- 172.21.15.59:0/2807925718 >> 172.21.15.59:0/2807925718 conn(0x7f58480ffa90 msgr2=0x7f5848101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.728+0000 7f584ea00640 1 -- 172.21.15.59:0/2807925718 shutdown_connections 2024-09-05T12:02:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.728+0000 7f584ea00640 1 -- 172.21.15.59:0/2807925718 wait complete. 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.729+0000 7f584ea00640 1 Processor -- start 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.729+0000 7f584ea00640 1 -- start start 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 0x7f584819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 0x7f584819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5848114170 con 0x7f5848103ff0 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5848113ff0 con 0x7f5848109120 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58481142f0 con 0x7f58481049a0 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f5847400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f5847e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 0x7f584819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f5847400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36738/0 (socket says 172.21.15.59:36738) 2024-09-05T12:02:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f5847400640 1 -- 172.21.15.59:0/1775300514 learned_addr learned my addr 172.21.15.59:0/1775300514 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.730+0000 7f584ca00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 0x7f584819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 -- 172.21.15.59:0/1775300514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 msgr2=0x7f584819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 0x7f584819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 -- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 msgr2=0x7f584819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 0x7f584819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 -- 172.21.15.59:0/1775300514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584819f600 con 0x7f58481049a0 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847e00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 0x7f584819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5847400640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5834004770 tx=0x7f5834003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583403f070 con 0x7f58481049a0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.731+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584819f890 con 0x7f58481049a0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.732+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834031ca0 con 0x7f58481049a0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.732+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834035030 con 0x7f58481049a0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.732+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481a71d0 con 0x7f58481049a0 2024-09-05T12:02:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.733+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58481043d0 con 0x7f58481049a0 2024-09-05T12:02:44.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.735+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5834042070 con 0x7f58481049a0 2024-09-05T12:02:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.736+0000 7f5844c00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 0x7f582c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.736+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58340efcc0 con 0x7f58481049a0 2024-09-05T12:02:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.737+0000 7f5847e00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 0x7f582c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.737+0000 7f5847e00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 0x7f582c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f583800bfd0 tx=0x7f583800b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.738+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58340bc160 con 0x7f58481049a0 2024-09-05T12:02:44.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.966+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f584819b9b0 con 0x7f58481049a0 2024-09-05T12:02:44.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.967+0000 7f5844c00640 1 -- 172.21.15.59:0/1775300514 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f58340c0460 con 0x7f58481049a0 2024-09-05T12:02:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:02:44.969 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:02:44.969 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 msgr2=0x7f582c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 0x7f582c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f583800bfd0 tx=0x7f583800b390 comp rx=0 tx=0).stop 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 msgr2=0x7f584819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5834004770 tx=0x7f5834003c10 comp rx=0 tx=0).stop 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 shutdown_connections 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f582c0a2c00 0x7f582c0a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5848109120 0x7f584819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58481049a0 0x7f584819ab90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.969+0000 7f584ea00640 1 --2- 172.21.15.59:0/1775300514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5848103ff0 0x7f584819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:44.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.970+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 >> 172.21.15.59:0/1775300514 conn(0x7f58480ffa90 msgr2=0x7f5848100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:44.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.970+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 shutdown_connections 2024-09-05T12:02:44.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:44.970+0000 7f584ea00640 1 -- 172.21.15.59:0/1775300514 wait complete. 2024-09-05T12:02:44.985 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:44 smithi059 ceph-mon[30025]: pgmap v546: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:44 smithi089 ceph-mon[34499]: pgmap v546: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:45.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:44 smithi189 ceph-mon[34772]: pgmap v546: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:45.805 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:02:45.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:02:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1775300514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:02:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1775300514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1775300514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:02:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 -- 172.21.15.59:0/1217442388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22901049a0 msgr2=0x7f22901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 --2- 172.21.15.59:0/1217442388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22901049a0 0x7f22901089f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f227c009900 tx=0x7f227c02a410 comp rx=0 tx=0).stop 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 -- 172.21.15.59:0/1217442388 shutdown_connections 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 --2- 172.21.15.59:0/1217442388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2290109120 0x7f2290110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 --2- 172.21.15.59:0/1217442388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22901049a0 0x7f22901089f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 --2- 172.21.15.59:0/1217442388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f22901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 -- 172.21.15.59:0/1217442388 >> 172.21.15.59:0/1217442388 conn(0x7f22900ffa90 msgr2=0x7f2290101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 -- 172.21.15.59:0/1217442388 shutdown_connections 2024-09-05T12:02:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.757+0000 7f2298200640 1 -- 172.21.15.59:0/1217442388 wait complete. 2024-09-05T12:02:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 Processor -- start 2024-09-05T12:02:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 -- start start 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 0x7f229019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 0x7f229019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2290114140 con 0x7f2290109120 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2290113fc0 con 0x7f22901049a0 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.758+0000 7f2298200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22901142c0 con 0x7f2290103ff0 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2294e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 0x7f229019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2296200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 0x7f229019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36756/0 (socket says 172.21.15.59:36756) 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2294e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 0x7f229019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49092/0 (socket says 172.21.15.59:49092) 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 -- 172.21.15.59:0/2754314064 learned_addr learned my addr 172.21.15.59:0/2754314064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 -- 172.21.15.59:0/2754314064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 msgr2=0x7f229019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 0x7f229019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 -- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 msgr2=0x7f229019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 0x7f229019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 -- 172.21.15.59:0/2754314064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229019f5d0 con 0x7f2290103ff0 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2296200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 0x7f229019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:02:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.759+0000 7f2295800640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.760+0000 7f228e000640 1 -- 172.21.15.59:0/2754314064 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228400cd80 con 0x7f2290103ff0 2024-09-05T12:02:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.760+0000 7f228e000640 1 -- 172.21.15.59:0/2754314064 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2284005500 con 0x7f2290103ff0 2024-09-05T12:02:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.760+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229019f8c0 con 0x7f2290103ff0 2024-09-05T12:02:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.760+0000 7f228e000640 1 -- 172.21.15.59:0/2754314064 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284010690 con 0x7f2290103ff0 2024-09-05T12:02:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.760+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22901a71a0 con 0x7f2290103ff0 2024-09-05T12:02:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 Processor -- start 2024-09-05T12:02:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 -- start start 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728853050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f228e000640 1 -- 172.21.15.59:0/2754314064 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22840046c0 con 0x7f2290103ff0 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728853590 0x55c728855980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7287ae530 con 0x55c728853590 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c728755860 con 0x55c728848cd0 2024-09-05T12:02:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.763+0000 7f229ae3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7287aa4f0 con 0x55c7288483b0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36758/0 (socket says 172.21.15.59:36758) 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 -- 172.21.15.59:0/1994349570 learned_addr learned my addr 172.21.15.59:0/1994349570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2295800640 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728853050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2294e00640 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728853590 0x55c728855980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 -- 172.21.15.59:0/1994349570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 msgr2=0x55c728853050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728853050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 -- 172.21.15.59:0/1994349570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728853590 msgr2=0x55c728855980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728853590 0x55c728855980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 -- 172.21.15.59:0/1994349570 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72851fb90 con 0x55c7288483b0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.764+0000 7f2296200640 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22880107a0 tx=0x7f2288010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5eb1eb726d5d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f2282a00640 1 -- 172.21.15.59:0/1994349570 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288003b70 con 0x55c7288483b0 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f2282a00640 1 -- 172.21.15.59:0/1994349570 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288010e50 con 0x55c7288483b0 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f2282a00640 1 -- 172.21.15.59:0/1994349570 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288004730 con 0x55c7288483b0 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f228e000640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 0x7f22780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f228e000640 1 -- 172.21.15.59:0/2754314064 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f22840cf890 con 0x7f2290103ff0 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f2295800640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 0x7f22780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f229ae3e740 1 -- 172.21.15.59:0/1994349570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 msgr2=0x55c728848790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.765+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22880107a0 tx=0x7f2288010c60 comp rx=0 tx=0).stop 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 -- 172.21.15.59:0/1994349570 shutdown_connections 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728853590 0x55c728855980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728853050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1994349570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c728848790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 -- 172.21.15.59:0/1994349570 >> 172.21.15.59:0/1994349570 conn(0x55c728844940 msgr2=0x55c728844d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 -- 172.21.15.59:0/1994349570 shutdown_connections 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 -- 172.21.15.59:0/1994349570 wait complete. 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f2295800640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 0x7f22780a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22840a43c0 tx=0x7f22840d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.766+0000 7f229ae3e740 1 -- start start 2024-09-05T12:02:46.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 Processor -- start 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7288483b0 0x55c7288507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c728851260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c728855fb0 con 0x55c7288483b0 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c728768e60 con 0x55c728848cd0 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c728856130 con 0x55c728853590 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.767+0000 7f229ae3e740 10 client.0 _ll_register_callbacks cb 0x55c72876dcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49110/0 (socket says 172.21.15.59:49110) 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 -- 172.21.15.59:0/244119367 learned_addr learned my addr 172.21.15.59:0/244119367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2296200640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7288483b0 0x55c7288507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2296200640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c728851260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 -- 172.21.15.59:0/244119367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 msgr2=0x55c728851260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c728851260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 -- 172.21.15.59:0/244119367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7288483b0 msgr2=0x55c7288507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7288483b0 0x55c7288507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2294e00640 1 -- 172.21.15.59:0/244119367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c728774990 con 0x55c728848cd0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.768+0000 7f2296200640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c728851260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2294e00640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f227c02a930 tx=0x7f227c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4d4f4a73ec68b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2277400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c035040 con 0x55c728848cd0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2288010e50 con 0x55c728848cd0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f227c004240 con 0x55c728848cd0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f227c004440 con 0x55c728848cd0 2024-09-05T12:02:46.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f229ae3e740 10 client.26177 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:46.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c728861f10 con 0x55c728848cd0 2024-09-05T12:02:46.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.769+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288010e50 con 0x55c728848cd0 2024-09-05T12:02:46.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f227c03d680 con 0x55c728848cd0 2024-09-05T12:02:46.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 10 client.26177 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:46.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 10 client.26177 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:46.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 20 client.26177 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c728863370 0x55c7288657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 4 client.26177 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c728751310 con 0x55c728863370 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.770+0000 7f2296200640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c728863370 0x55c7288657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.771+0000 7f2296200640 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c728863370 0x55c7288657b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d6295bfb15767a server_cookie=664078608c406f1 in_seq=0 out_seq=0 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.771+0000 7f2277400640 10 client.26177 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.771+0000 7f2277400640 1 -- 172.21.15.59:0/244119367 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c728751310 con 0x55c728863370 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.771+0000 7f2277400640 10 client.26177 handle_command_reply: tid=0 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.771+0000 7f229ae3e740 1 client.26177 shutdown 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 msgr2=0x55c728850ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f227c02a930 tx=0x7f227c031850 comp rx=0 tx=0).stop 2024-09-05T12:02:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 shutdown_connections 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c728851260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728848cd0 0x55c728850ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c728863370 0x55c7288657b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 --2- 172.21.15.59:0/244119367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7288483b0 0x55c7288507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 >> 172.21.15.59:0/244119367 conn(0x55c728844940 msgr2=0x55c728843c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f2277400640 0 client.26177 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 shutdown_connections 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 1 -- 172.21.15.59:0/244119367 wait complete. 2024-09-05T12:02:46.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.772+0000 7f229ae3e740 20 client.26177 trim_cache size 0 max 16384 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 Processor -- start 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 -- start start 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7288483b0 0x55c72884f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c72886b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22901af0b0 con 0x55c728848cd0 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b1930 con 0x55c7288483b0 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f229ae3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22901af720 con 0x55c728853590 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f2294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f2294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49878/0 (socket says 172.21.15.59:49878) 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.790+0000 7f2294e00640 1 -- 172.21.15.59:0/3054689476 learned_addr learned my addr 172.21.15.59:0/3054689476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2296200640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7288483b0 0x55c72884f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2296200640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c72886b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 -- 172.21.15.59:0/3054689476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 msgr2=0x55c72886b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c72886b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 -- 172.21.15.59:0/3054689476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7288483b0 msgr2=0x55c72884f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7288483b0 0x55c72884f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 -- 172.21.15.59:0/3054689476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c728855cf0 con 0x55c728848cd0 2024-09-05T12:02:46.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2296200640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c72886b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2294e00640 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f227c006010 tx=0x7f227c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc344893285f371 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2282000640 1 -- 172.21.15.59:0/3054689476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c035070 con 0x55c728848cd0 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2282000640 1 -- 172.21.15.59:0/3054689476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c005500 con 0x55c728848cd0 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.791+0000 7f2282000640 1 -- 172.21.15.59:0/3054689476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c004a40 con 0x55c728848cd0 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 -- 172.21.15.59:0/3054689476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 msgr2=0x55c728851070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f227c006010 tx=0x7f227c009830 comp rx=0 tx=0).stop 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 -- 172.21.15.59:0/3054689476 shutdown_connections 2024-09-05T12:02:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c728853590 0x55c72886b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c728851070 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 --2- 172.21.15.59:0/3054689476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7288483b0 0x55c72884f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 -- 172.21.15.59:0/3054689476 >> 172.21.15.59:0/3054689476 conn(0x55c728844940 msgr2=0x55c728852710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 -- 172.21.15.59:0/3054689476 shutdown_connections 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.792+0000 7f229ae3e740 1 -- 172.21.15.59:0/3054689476 wait complete. 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 -- start start 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 Processor -- start 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c72885b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 0x55c7288583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22901afdf0 con 0x55c728848cd0 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b1260 con 0x55c728853590 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22901b04c0 con 0x55c7288483b0 2024-09-05T12:02:46.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.793+0000 7f229ae3e740 10 client.0 _ll_register_callbacks cb 0x7f227c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49886/0 (socket says 172.21.15.59:49886) 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 -- 172.21.15.59:0/1399071326 learned_addr learned my addr 172.21.15.59:0/1399071326 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2296200640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c72885b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2296200640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 0x55c7288583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 -- 172.21.15.59:0/1399071326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 msgr2=0x55c72885b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c72885b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 -- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 msgr2=0x55c7288583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 0x55c7288583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 -- 172.21.15.59:0/1399071326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7288588e0 con 0x55c728848cd0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2296200640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 0x55c7288583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2294e00640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f227c02a930 tx=0x7f227c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966795f3fbaf9a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:02:46.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.794+0000 7f2277400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c035070 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227c005500 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f227c03d660 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f227c066050 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f229ae3e740 10 client.16170 fetch_fsmap learned FSMap version 21 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c7288ce7e0 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c005500 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f227c0464b0 con 0x55c728848cd0 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.795+0000 7f229ae3e740 10 client.16170 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f229ae3e740 10 client.16170 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f229ae3e740 20 client.16170 populate_metadata read hostname 'smithi059' 2024-09-05T12:02:46.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c7288ced50 0x55c7288d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:02:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f229ae3e740 4 client.16170 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:02:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c7287bbc60 con 0x55c7288ced50 2024-09-05T12:02:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f2296200640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c7288ced50 0x55c7288d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:02:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f2296200640 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c7288ced50 0x55c7288d1170 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d63f872a73c957 server_cookie=b5d27045be4b0dee in_seq=0 out_seq=0 2024-09-05T12:02:46.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.796+0000 7f2277400640 10 client.16170 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f2277400640 1 -- 172.21.15.59:0/1399071326 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7287bbc60 con 0x55c7288ced50 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f2277400640 10 client.16170 handle_command_reply: tid=0 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f229ae3e740 1 client.16170 shutdown 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 msgr2=0x55c72885b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f227c02a930 tx=0x7f227c0315b0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 shutdown_connections 2024-09-05T12:02:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.804+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c728853590 0x55c7288583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c7288ced50 0x55c7288d1170 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c728848cd0 0x55c72885b910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 --2- 172.21.15.59:0/1399071326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7288483b0 0x55c72885b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 >> 172.21.15.59:0/1399071326 conn(0x55c728844940 msgr2=0x55c72885e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f2277400640 0 client.16170 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 shutdown_connections 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 1 -- 172.21.15.59:0/1399071326 wait complete. 2024-09-05T12:02:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f229ae3e740 20 client.16170 trim_cache size 0 max 16384 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 22, 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "a3622184": { 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 22, 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:46.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:02:46.809 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 msgr2=0x7f22780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 0x7f22780a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22840a43c0 tx=0x7f22840d5530 comp rx=0 tx=0).stop 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 msgr2=0x7f229019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.805+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400edf0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 shutdown_connections 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f22780a2b80 0x7f22780a5040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2290109120 0x7f229019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22901049a0 0x7f229019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 --2- 172.21.15.59:0/2754314064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2290103ff0 0x7f229019a620 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:02:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 >> 172.21.15.59:0/2754314064 conn(0x7f22900ffa90 msgr2=0x7f2290100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:02:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 shutdown_connections 2024-09-05T12:02:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:02:46.806+0000 7f2298200640 1 -- 172.21.15.59:0/2754314064 wait complete. 2024-09-05T12:02:46.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:46 smithi059 ceph-mon[30025]: pgmap v547: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:46 smithi089 ceph-mon[34499]: pgmap v547: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:47.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:46 smithi189 ceph-mon[34772]: pgmap v547: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-05T12:02:47.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a3622184', quiesce took 3.9 seconds. Will release after: 26.1 2024-09-05T12:02:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:47 smithi089 ceph-mon[34499]: pgmap v548: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-09-05T12:02:48.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:47 smithi059 ceph-mon[30025]: pgmap v548: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-09-05T12:02:48.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:47 smithi189 ceph-mon[34772]: pgmap v548: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-09-05T12:02:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:50 smithi189 ceph-mon[34772]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.3 KiB/s wr, 2 op/s 2024-09-05T12:02:50.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:50 smithi089 ceph-mon[34499]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.3 KiB/s wr, 2 op/s 2024-09-05T12:02:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:50 smithi059 ceph-mon[30025]: pgmap v549: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.3 KiB/s wr, 2 op/s 2024-09-05T12:02:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:52 smithi189 ceph-mon[34772]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:52.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:52 smithi089 ceph-mon[34499]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:52.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:52 smithi059 ceph-mon[30025]: pgmap v550: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:54.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:54 smithi189 ceph-mon[34772]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:54.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:54 smithi089 ceph-mon[34499]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:54.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:54 smithi059 ceph-mon[30025]: pgmap v551: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:56 smithi059 ceph-mon[30025]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:56.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:56 smithi189 ceph-mon[34772]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:56 smithi089 ceph-mon[34499]: pgmap v552: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:02:58.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:02:58 smithi189 ceph-mon[34772]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:58.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:02:58 smithi089 ceph-mon[34499]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:02:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:02:58 smithi059 ceph-mon[30025]: pgmap v553: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:03:00.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:00 smithi189 ceph-mon[34772]: pgmap v554: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.9 KiB/s wr, 2 op/s 2024-09-05T12:03:00.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:00 smithi089 ceph-mon[34499]: pgmap v554: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.9 KiB/s wr, 2 op/s 2024-09-05T12:03:00.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:00 smithi059 ceph-mon[30025]: pgmap v554: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.9 KiB/s wr, 2 op/s 2024-09-05T12:03:02.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:02 smithi189 ceph-mon[34772]: pgmap v555: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:03:02.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:02 smithi089 ceph-mon[34499]: pgmap v555: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:03:02.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:02 smithi059 ceph-mon[30025]: pgmap v555: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:03:03.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:03:04.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:04 smithi059 ceph-mon[30025]: pgmap v556: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:03:04.210 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:04.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:04 smithi189 ceph-mon[34772]: pgmap v556: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:03:04.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:04 smithi089 ceph-mon[34499]: pgmap v556: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 -- 172.21.15.59:0/1587048387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324103ff0 msgr2=0x7f93241043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 --2- 172.21.15.59:0/1587048387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324103ff0 0x7f93241043d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9318009900 tx=0x7f931802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 -- 172.21.15.59:0/1587048387 shutdown_connections 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 --2- 172.21.15.59:0/1587048387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9324109120 0x7f9324110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 --2- 172.21.15.59:0/1587048387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93241049a0 0x7f93241089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 --2- 172.21.15.59:0/1587048387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324103ff0 0x7f93241043d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 -- 172.21.15.59:0/1587048387 >> 172.21.15.59:0/1587048387 conn(0x7f93240ffa90 msgr2=0x7f9324101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 -- 172.21.15.59:0/1587048387 shutdown_connections 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.795+0000 7f932ba00640 1 -- 172.21.15.59:0/1587048387 wait complete. 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.796+0000 7f932ba00640 1 Processor -- start 2024-09-05T12:03:04.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.796+0000 7f932ba00640 1 -- start start 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324103ff0 0x7f932419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93241049a0 0x7f932419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9324114120 con 0x7f9324109120 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9324113fa0 con 0x7f9324103ff0 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f932ba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93241142a0 con 0x7f93241049a0 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f9329a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f9329a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60178/0 (socket says 172.21.15.59:60178) 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f9329a00640 1 -- 172.21.15.59:0/616396837 learned_addr learned my addr 172.21.15.59:0/616396837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f9329000640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324103ff0 0x7f932419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.797+0000 7f9323e00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93241049a0 0x7f932419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 -- 172.21.15.59:0/616396837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93241049a0 msgr2=0x7f932419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93241049a0 0x7f932419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 -- 172.21.15.59:0/616396837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324103ff0 msgr2=0x7f932419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324103ff0 0x7f932419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 -- 172.21.15.59:0/616396837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932419f5b0 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9329a00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f931c00eb10 tx=0x7f931c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9321600640 1 -- 172.21.15.59:0/616396837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c004b20 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9321600640 1 -- 172.21.15.59:0/616396837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c004cc0 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f9321600640 1 -- 172.21.15.59:0/616396837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c010760 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932419f8a0 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.798+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241a7180 con 0x7f9324109120 2024-09-05T12:03:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.802+0000 7f932e44b740 1 Processor -- start 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f9321600640 1 -- 172.21.15.59:0/616396837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f931c005020 con 0x7f9324109120 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 -- start start 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 0x56463bd2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 0x56463bd35830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd38160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56463bc913e0 con 0x56463bd2ab90 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56463bc38d80 con 0x56463bd35d70 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f932e44b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56463bc8d3a0 con 0x56463bd2b4b0 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 0x56463bd2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f9329000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 0x56463bd35830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 0x56463bd2af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60190/0 (socket says 172.21.15.59:60190) 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.803+0000 7f9323e00640 1 -- 172.21.15.59:0/1216119682 learned_addr learned my addr 172.21.15.59:0/1216119682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9323e00640 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd38160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 -- 172.21.15.59:0/1216119682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 msgr2=0x56463bd38160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd38160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 -- 172.21.15.59:0/1216119682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 msgr2=0x56463bd2af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 0x56463bd2af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 -- 172.21.15.59:0/1216119682 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463bc57ae0 con 0x56463bd2b4b0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9329000640 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 0x56463bd35830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9318005940 tx=0x7f9318005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33cb09f5df0783e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9321600640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 0x7f930c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.804+0000 7f9321600640 1 -- 172.21.15.59:0/616396837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f931c0cf9a0 con 0x7f9324109120 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f9323e00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 0x7f930c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f9316000640 1 -- 172.21.15.59:0/1216119682 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318040070 con 0x56463bd2b4b0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f9316000640 1 -- 172.21.15.59:0/1216119682 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9318031870 con 0x56463bd2b4b0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f9316000640 1 -- 172.21.15.59:0/1216119682 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318035040 con 0x56463bd2b4b0 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f932e44b740 1 -- 172.21.15.59:0/1216119682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 msgr2=0x56463bd35830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f932e44b740 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 0x56463bd35830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9318005940 tx=0x7f9318005be0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.805+0000 7f9323e00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 0x7f930c0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x56463bd284a0 tx=0x7f93100097a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 -- 172.21.15.59:0/1216119682 shutdown_connections 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd38160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2b4b0 0x56463bd35830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 --2- 172.21.15.59:0/1216119682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2ab90 0x56463bd2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 -- 172.21.15.59:0/1216119682 >> 172.21.15.59:0/1216119682 conn(0x56463bd27120 msgr2=0x56463bd27500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 -- 172.21.15.59:0/1216119682 shutdown_connections 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 -- 172.21.15.59:0/1216119682 wait complete. 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 -- start start 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.806+0000 7f932e44b740 1 Processor -- start 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd32c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56463bd38980 con 0x56463bd2b4b0 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56463bc4c0f0 con 0x56463bd35d70 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56463bd38b00 con 0x56463bd2ab90 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f932e44b740 10 client.0 _ll_register_callbacks cb 0x56463bc3d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60194/0 (socket says 172.21.15.59:60194) 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f9323e00640 1 -- 172.21.15.59:0/311604853 learned_addr learned my addr 172.21.15.59:0/311604853 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f9329000640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd32c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.807+0000 7f9329a00640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 -- 172.21.15.59:0/311604853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 msgr2=0x56463bd32c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd32c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 -- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 msgr2=0x56463bd336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 -- 172.21.15.59:0/311604853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463ba023c0 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f9323e00640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93100135c0 tx=0x7f9310005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae45872df4485d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f930aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931002f060 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f931001c030 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.808+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9318031870 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9310036050 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 10 client.16182 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56463bd44610 con 0x56463bd2b4b0 2024-09-05T12:03:04.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9318031870 con 0x56463bd2b4b0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f931001f500 con 0x56463bd2b4b0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 10 client.16182 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 20 client.16182 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bd45ad0 0x56463bd47f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 4 client.16182 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.809+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56463bc9e950 con 0x56463bd45ad0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.810+0000 7f9329000640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bd45ad0 0x56463bd47f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.810+0000 7f9329000640 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bd45ad0 0x56463bd47f00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af6d02317dab213 server_cookie=d11ba34f96924a76 in_seq=0 out_seq=0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.810+0000 7f930aa00640 10 client.16182 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f930aa00640 1 -- 172.21.15.59:0/311604853 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56463bc9e950 con 0x56463bd45ad0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f930aa00640 10 client.16182 handle_command_reply: tid=0 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 client.16182 shutdown 2024-09-05T12:03:04.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 msgr2=0x56463bd33180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93100135c0 tx=0x7f9310005f90 comp rx=0 tx=0).stop 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 shutdown_connections 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bd336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bd45ad0 0x56463bd47f00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd33180 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 --2- 172.21.15.59:0/311604853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f930aa00640 0 client.16182 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.811+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 >> 172.21.15.59:0/311604853 conn(0x56463bd27120 msgr2=0x56463bd2ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.812+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 shutdown_connections 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.812+0000 7f932e44b740 1 -- 172.21.15.59:0/311604853 wait complete. 2024-09-05T12:03:04.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.812+0000 7f932e44b740 20 client.16182 trim_cache size 0 max 16384 2024-09-05T12:03:04.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 Processor -- start 2024-09-05T12:03:04.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 -- start start 2024-09-05T12:03:04.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 0x56463bd4d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93241af080 con 0x56463bd35d70 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f93241b1900 con 0x56463bd2b4b0 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.830+0000 7f932e44b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93241af6f0 con 0x56463bd2ab90 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60206/0 (socket says 172.21.15.59:60206) 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 -- 172.21.15.59:0/1129000405 learned_addr learned my addr 172.21.15.59:0/1129000405 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:04.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9323e00640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 0x56463bd4d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329000640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 -- 172.21.15.59:0/1129000405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 msgr2=0x56463bd2d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd2d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 -- 172.21.15.59:0/1129000405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 msgr2=0x56463bd4d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 0x56463bd4d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 -- 172.21.15.59:0/1129000405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463bd26f50 con 0x56463bd35d70 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9323e00640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 0x56463bd4d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.831+0000 7f9329a00640 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f931c09ba90 tx=0x7f931c09bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb96308a1443b525 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f9315600640 1 -- 172.21.15.59:0/1129000405 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c09c490 con 0x56463bd35d70 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f9315600640 1 -- 172.21.15.59:0/1129000405 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c0b9070 con 0x56463bd35d70 2024-09-05T12:03:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f9315600640 1 -- 172.21.15.59:0/1129000405 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0a4570 con 0x56463bd35d70 2024-09-05T12:03:04.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 -- 172.21.15.59:0/1129000405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 msgr2=0x56463bd4dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f931c09ba90 tx=0x7f931c09bc20 comp rx=0 tx=0).stop 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 -- 172.21.15.59:0/1129000405 shutdown_connections 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd35d70 0x56463bd4dd10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd2b4b0 0x56463bd4d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 --2- 172.21.15.59:0/1129000405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.832+0000 7f932e44b740 1 -- 172.21.15.59:0/1129000405 >> 172.21.15.59:0/1129000405 conn(0x56463bd27120 msgr2=0x56463bd34cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.833+0000 7f932e44b740 1 -- 172.21.15.59:0/1129000405 shutdown_connections 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.833+0000 7f932e44b740 1 -- 172.21.15.59:0/1129000405 wait complete. 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.833+0000 7f932e44b740 1 -- start start 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.833+0000 7f932e44b740 1 Processor -- start 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bdaf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93241afdc0 con 0x56463bd2b4b0 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f93241b1230 con 0x56463bd35d70 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93241b0490 con 0x56463bd2ab90 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f932e44b740 10 client.0 _ll_register_callbacks cb 0x7f931000fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40352/0 (socket says 172.21.15.59:40352) 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60218/0 (socket says 172.21.15.59:60218) 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 -- 172.21.15.59:0/3181277625 learned_addr learned my addr 172.21.15.59:0/3181277625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329a00640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bdaf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 -- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 msgr2=0x56463bdaf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bdaf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 -- 172.21.15.59:0/3181277625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 msgr2=0x56463bd3a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9329000640 1 -- 172.21.15.59:0/3181277625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463bd2c110 con 0x56463bd2ab90 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.834+0000 7f9323e00640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f9329000640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f931802aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4c23c75a51cbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f930aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318040070 con 0x56463bd2ab90 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9318004030 con 0x56463bd2ab90 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c0b9070 con 0x56463bd2ab90 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9318062050 con 0x56463bd2ab90 2024-09-05T12:03:04.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f932e44b740 10 client.26125 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.835+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56463bdb0c30 con 0x56463bd2ab90 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c0b9070 con 0x56463bd2ab90 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f931803c4d0 con 0x56463bd2ab90 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 10 client.26125 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 20 client.26125 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bdb0f20 0x56463bdb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 4 client.26125 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:03:04.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56463bc98e60 con 0x56463bdb0f20 2024-09-05T12:03:04.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.836+0000 7f9323e00640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bdb0f20 0x56463bdb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:04.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f9323e00640 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bdb0f20 0x56463bdb3310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ccae20e0453d76a server_cookie=219283032d650d9a in_seq=0 out_seq=0 2024-09-05T12:03:04.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f930aa00640 10 client.26125 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:04.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f930aa00640 1 -- 172.21.15.59:0/3181277625 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56463bc98e60 con 0x56463bdb0f20 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f930aa00640 10 client.26125 handle_command_reply: tid=0 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 client.26125 shutdown 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 msgr2=0x56463bd39e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f931802aba0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 shutdown_connections 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56463bd35d70 0x56463bdaf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56463bdb0f20 0x56463bdb3310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56463bd2b4b0 0x56463bd3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 --2- 172.21.15.59:0/3181277625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56463bd2ab90 0x56463bd39e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f930aa00640 0 client.26125 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.837+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 >> 172.21.15.59:0/3181277625 conn(0x56463bd27120 msgr2=0x56463bd32b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 shutdown_connections 2024-09-05T12:03:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932e44b740 1 -- 172.21.15.59:0/3181277625 wait complete. 2024-09-05T12:03:04.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932e44b740 20 client.26125 trim_cache size 0 max 16384 2024-09-05T12:03:04.845 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:04.845 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:03:04.845 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:03:04.845 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 msgr2=0x7f930c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 0x7f930c0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x56463bd284a0 tx=0x7f93100097a0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 msgr2=0x7f932419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.838+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f931c00eb10 tx=0x7f931c00c2d0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 shutdown_connections 2024-09-05T12:03:04.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f930c0a2b80 0x7f930c0a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9324109120 0x7f932419eed0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93241049a0 0x7f932419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 --2- 172.21.15.59:0/616396837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324103ff0 0x7f932419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 >> 172.21.15.59:0/616396837 conn(0x7f93240ffa90 msgr2=0x7f9324100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 shutdown_connections 2024-09-05T12:03:04.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:04.839+0000 7f932ba00640 1 -- 172.21.15.59:0/616396837 wait complete. 2024-09-05T12:03:05.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:03:05.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:03:06.156 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:06.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:06 smithi189 ceph-mon[34772]: pgmap v557: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:03:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:06 smithi089 ceph-mon[34499]: pgmap v557: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:03:06.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:06 smithi059 ceph-mon[30025]: pgmap v557: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:03:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 -- 172.21.15.59:0/2659743939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 msgr2=0x7faa64110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 --2- 172.21.15.59:0/2659743939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa64110650 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009ca0 tx=0x7faa5c02a950 comp rx=0 tx=0).stop 2024-09-05T12:03:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 -- 172.21.15.59:0/2659743939 shutdown_connections 2024-09-05T12:03:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 --2- 172.21.15.59:0/2659743939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa64110650 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 --2- 172.21.15.59:0/2659743939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa64074ee0 0x7faa6410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 --2- 172.21.15.59:0/2659743939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa640745c0 0x7faa640749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 -- 172.21.15.59:0/2659743939 >> 172.21.15.59:0/2659743939 conn(0x7faa640fd740 msgr2=0x7faa640ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 -- 172.21.15.59:0/2659743939 shutdown_connections 2024-09-05T12:03:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.738+0000 7faa69600640 1 -- 172.21.15.59:0/2659743939 wait complete. 2024-09-05T12:03:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.739+0000 7faa69600640 1 Processor -- start 2024-09-05T12:03:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 -- start start 2024-09-05T12:03:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa64074ee0 0x7faa6419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa6419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa64114000 con 0x7faa6410e260 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa64113e80 con 0x7faa640745c0 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa69600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa64114180 con 0x7faa64074ee0 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa68c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa6419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa63e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa63e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49334/0 (socket says 172.21.15.59:49334) 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa68c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa6419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60242/0 (socket says 172.21.15.59:60242) 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa63400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa64074ee0 0x7faa6419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.740+0000 7faa63e00640 1 -- 172.21.15.59:0/1444662202 learned_addr learned my addr 172.21.15.59:0/1444662202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 -- 172.21.15.59:0/1444662202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa64074ee0 msgr2=0x7faa6419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa64074ee0 0x7faa6419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 -- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 msgr2=0x7faa6419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa6419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 -- 172.21.15.59:0/1444662202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6419f430 con 0x7faa640745c0 2024-09-05T12:03:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.741+0000 7faa63e00640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9f0 tx=0x7faa5400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.742+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400cd80 con 0x7faa640745c0 2024-09-05T12:03:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.742+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6419f720 con 0x7faa640745c0 2024-09-05T12:03:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.742+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54005500 con 0x7faa640745c0 2024-09-05T12:03:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.742+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54010690 con 0x7faa640745c0 2024-09-05T12:03:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.742+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a7000 con 0x7faa640745c0 2024-09-05T12:03:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.743+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa640749a0 con 0x7faa640745c0 2024-09-05T12:03:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.746+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa540046c0 con 0x7faa640745c0 2024-09-05T12:03:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.747+0000 7faa60c00640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.748+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7faa540cf890 con 0x7faa640745c0 2024-09-05T12:03:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.748+0000 7faa63400640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.749+0000 7faa63400640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 0x7faa480a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa500095f0 tx=0x7faa50005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:06.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.752+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa5409c380 con 0x7faa640745c0 2024-09-05T12:03:06.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.975+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6419b7e0 con 0x7faa640745c0 2024-09-05T12:03:06.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.976+0000 7faa60c00640 1 -- 172.21.15.59:0/1444662202 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7faa5409bc80 con 0x7faa640745c0 2024-09-05T12:03:06.979 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:03:06.979 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:03:06.980 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 msgr2=0x7faa480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 0x7faa480a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa500095f0 tx=0x7faa50005f90 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 msgr2=0x7faa6419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9f0 tx=0x7faa5400edf0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 shutdown_connections 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa6410e260 0x7faa6419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa64074ee0 0x7faa6419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 --2- 172.21.15.59:0/1444662202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa640745c0 0x7faa6419a480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 >> 172.21.15.59:0/1444662202 conn(0x7faa640fd740 msgr2=0x7faa640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 shutdown_connections 2024-09-05T12:03:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:06.980+0000 7faa69600640 1 -- 172.21.15.59:0/1444662202 wait complete. 2024-09-05T12:03:07.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:07 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1444662202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:07.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:07 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1444662202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:07.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:07 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1444662202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:07.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:03:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:08 smithi059 ceph-mon[30025]: pgmap v558: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:03:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:08 smithi089 ceph-mon[34499]: pgmap v558: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:03:08.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:08 smithi189 ceph-mon[34772]: pgmap v558: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.814+0000 7fac7f000640 1 -- 172.21.15.59:0/3081959544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 msgr2=0x7fac78112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.814+0000 7fac7f000640 1 --2- 172.21.15.59:0/3081959544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac78112330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fac64009900 tx=0x7fac6402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 -- 172.21.15.59:0/3081959544 shutdown_connections 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 --2- 172.21.15.59:0/3081959544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac78114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 --2- 172.21.15.59:0/3081959544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac78112330 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 --2- 172.21.15.59:0/3081959544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 0x7fac78076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 -- 172.21.15.59:0/3081959544 >> 172.21.15.59:0/3081959544 conn(0x7fac780ff8d0 msgr2=0x7fac78101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 -- 172.21.15.59:0/3081959544 shutdown_connections 2024-09-05T12:03:08.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.815+0000 7fac7f000640 1 -- 172.21.15.59:0/3081959544 wait complete. 2024-09-05T12:03:08.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 Processor -- start 2024-09-05T12:03:08.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 -- start start 2024-09-05T12:03:08.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 0x7fac781a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac781a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac781183e0 con 0x7fac78077040 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac78118260 con 0x7fac78112870 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac78118560 con 0x7fac78076720 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.816+0000 7fac7d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac781a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 0x7fac781a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac781a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60264/0 (socket says 172.21.15.59:60264) 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7d200640 1 -- 172.21.15.59:0/1319000506 learned_addr learned my addr 172.21.15.59:0/1319000506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49356/0 (socket says 172.21.15.59:49356) 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 -- 172.21.15.59:0/1319000506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 msgr2=0x7fac781a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 0x7fac781a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 -- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 msgr2=0x7fac781a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac781a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 -- 172.21.15.59:0/1319000506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7819ef80 con 0x7fac78112870 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.817+0000 7fac7e600640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac7807aa10 tx=0x7fac7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.818+0000 7fac76000640 1 -- 172.21.15.59:0/1319000506 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000cdf0 con 0x7fac78112870 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.818+0000 7fac76000640 1 -- 172.21.15.59:0/1319000506 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac70005020 con 0x7fac78112870 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.818+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7819f270 con 0x7fac78112870 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.818+0000 7fac76000640 1 -- 172.21.15.59:0/1319000506 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70010670 con 0x7fac78112870 2024-09-05T12:03:08.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.818+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac781ab3f0 con 0x7fac78112870 2024-09-05T12:03:08.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 Processor -- start 2024-09-05T12:03:08.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac76000640 1 -- 172.21.15.59:0/1319000506 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac700054d0 con 0x7fac78112870 2024-09-05T12:03:08.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 -- start start 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 0x5574ff2dd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574ff2363e0 con 0x5574ff2cfe40 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574ff1ddd80 con 0x5574ff2db020 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.821+0000 7fac82fc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5574ff2323a0 con 0x5574ff2d0760 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40402/0 (socket says 172.21.15.59:40402) 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 -- 172.21.15.59:0/472638676 learned_addr learned my addr 172.21.15.59:0/472638676 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:08.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7d200640 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 0x5574ff2dd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 -- 172.21.15.59:0/472638676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 msgr2=0x5574ff2dd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 0x5574ff2dd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 -- 172.21.15.59:0/472638676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 msgr2=0x5574ff2d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7dc00640 1 -- 172.21.15.59:0/472638676 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ff1fcae0 con 0x5574ff2d0760 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.822+0000 7fac7d200640 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 0x5574ff2dd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:03:08.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac7dc00640 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00bd10 tx=0x7fac6c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c44e6edee3eeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac76000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 0x7fac600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac6aa00640 1 -- 172.21.15.59:0/472638676 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c004030 con 0x5574ff2d0760 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac6aa00640 1 -- 172.21.15.59:0/472638676 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c0041d0 con 0x5574ff2d0760 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac6aa00640 1 -- 172.21.15.59:0/472638676 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0117e0 con 0x5574ff2d0760 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac76000640 1 -- 172.21.15.59:0/1319000506 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fac700cfb10 con 0x7fac78112870 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.823+0000 7fac7d200640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 0x7fac600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- 172.21.15.59:0/472638676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 msgr2=0x5574ff2daae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00bd10 tx=0x7fac6c0107a0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- 172.21.15.59:0/472638676 shutdown_connections 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2db020 0x5574ff2dd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2daae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 --2- 172.21.15.59:0/472638676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- 172.21.15.59:0/472638676 >> 172.21.15.59:0/472638676 conn(0x5574ff2cc3d0 msgr2=0x5574ff2cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac7d200640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 0x7fac600a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac64031550 tx=0x7fac640095a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- 172.21.15.59:0/472638676 shutdown_connections 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- 172.21.15.59:0/472638676 wait complete. 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.824+0000 7fac82fc4740 1 -- start start 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 Processor -- start 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 0x5574ff2d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5574ff2ddbf0 con 0x5574ff2cfe40 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574ff1f10f0 con 0x5574ff2d0760 2024-09-05T12:03:08.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5574ff2ddd70 con 0x5574ff2db020 2024-09-05T12:03:08.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.825+0000 7fac82fc4740 10 client.0 _ll_register_callbacks cb 0x5574ff1d8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:08.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49388/0 (socket says 172.21.15.59:49388) 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 -- 172.21.15.59:0/2305319918 learned_addr learned my addr 172.21.15.59:0/2305319918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7e600640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 0x5574ff2d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 -- 172.21.15.59:0/2305319918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 msgr2=0x5574ff2d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 0x5574ff2d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 -- 172.21.15.59:0/2305319918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 msgr2=0x5574ff2d7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 -- 172.21.15.59:0/2305319918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fefa73c0 con 0x5574ff2d0760 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7e600640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 0x5574ff2d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7dc00640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac7d200640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fac64009a20 tx=0x7fac6402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b1380b8aeb7012 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.826+0000 7fac5f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64035040 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6c0041d0 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fac64005240 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fac64005440 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac82fc4740 10 client.26216 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5574ff2e9c20 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.827+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c0041d0 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fac64003a30 con 0x5574ff2d0760 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 10 client.26216 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 20 client.26216 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff2eb0e0 0x5574ff2ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 4 client.26216 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ff243950 con 0x5574ff2eb0e0 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.828+0000 7fac7dc00640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff2eb0e0 0x5574ff2ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.829+0000 7fac7dc00640 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff2eb0e0 0x5574ff2ed550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d259c24cbc6c34e server_cookie=6671cd715907e6fd in_seq=0 out_seq=0 2024-09-05T12:03:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.829+0000 7fac5f400640 10 client.26216 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac5f400640 1 -- 172.21.15.59:0/2305319918 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574ff243950 con 0x5574ff2eb0e0 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac5f400640 10 client.26216 handle_command_reply: tid=0 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 client.26216 shutdown 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 msgr2=0x5574ff2d84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fac64009a20 tx=0x7fac6402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 shutdown_connections 2024-09-05T12:03:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2db020 0x5574ff2d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2d0760 0x5574ff2d84d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff2eb0e0 0x5574ff2ed550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 --2- 172.21.15.59:0/2305319918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2cfe40 0x5574ff2d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac5f400640 0 client.26216 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 >> 172.21.15.59:0/2305319918 conn(0x5574ff2cc3d0 msgr2=0x5574ff2d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 shutdown_connections 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.830+0000 7fac82fc4740 1 -- 172.21.15.59:0/2305319918 wait complete. 2024-09-05T12:03:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.831+0000 7fac82fc4740 20 client.26216 trim_cache size 0 max 16384 2024-09-05T12:03:08.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.844+0000 7fac82fc4740 1 Processor -- start 2024-09-05T12:03:08.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 -- start start 2024-09-05T12:03:08.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b3350 con 0x5574ff2db020 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b5bd0 con 0x5574ff2cfe40 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac82fc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b39c0 con 0x5574ff2d0760 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49402/0 (socket says 172.21.15.59:49402) 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 -- 172.21.15.59:0/1098766300 learned_addr learned my addr 172.21.15.59:0/1098766300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7e600640 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7d200640 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 -- 172.21.15.59:0/1098766300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 msgr2=0x5574ff2f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 -- 172.21.15.59:0/1098766300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 msgr2=0x5574ff2f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.845+0000 7fac7dc00640 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.846+0000 7fac7dc00640 1 -- 172.21.15.59:0/1098766300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ff2cc200 con 0x5574ff2cfe40 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.846+0000 7fac7dc00640 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee725165ecb68035 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.846+0000 7fac6a000640 1 -- 172.21.15.59:0/1098766300 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c005240 con 0x5574ff2cfe40 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.846+0000 7fac6a000640 1 -- 172.21.15.59:0/1098766300 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c0053e0 con 0x5574ff2cfe40 2024-09-05T12:03:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.846+0000 7fac6a000640 1 -- 172.21.15.59:0/1098766300 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c011830 con 0x5574ff2cfe40 2024-09-05T12:03:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- 172.21.15.59:0/1098766300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 msgr2=0x5574ff2cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c00b790 comp rx=0 tx=0).stop 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- 172.21.15.59:0/1098766300 shutdown_connections 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 --2- 172.21.15.59:0/1098766300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2cb380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- 172.21.15.59:0/1098766300 >> 172.21.15.59:0/1098766300 conn(0x5574ff2cc3d0 msgr2=0x5574ff2cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- 172.21.15.59:0/1098766300 shutdown_connections 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- 172.21.15.59:0/1098766300 wait complete. 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 -- start start 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.847+0000 7fac82fc4740 1 Processor -- start 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b4090 con 0x5574ff2db020 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b5500 con 0x5574ff2cfe40 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac781b4760 con 0x5574ff2d0760 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac82fc4740 10 client.0 _ll_register_callbacks cb 0x7fac64031630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:49406/0 (socket says 172.21.15.59:49406) 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 -- 172.21.15.59:0/3473198098 learned_addr learned my addr 172.21.15.59:0/3473198098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7e600640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7d200640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 -- 172.21.15.59:0/3473198098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 msgr2=0x5574ff2de870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2de870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.848+0000 7fac7dc00640 1 -- 172.21.15.59:0/3473198098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 msgr2=0x5574ff2dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac7dc00640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2dedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac7dc00640 1 -- 172.21.15.59:0/3473198098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ff2df3f0 con 0x5574ff2cfe40 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac7dc00640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b620 tx=0x7fac6c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15fedb07c5f5d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac5f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0037f0 con 0x5574ff2cfe40 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6c0053e0 con 0x5574ff2cfe40 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fac6c003990 con 0x5574ff2cfe40 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fac6c003b90 con 0x5574ff2cfe40 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac82fc4740 10 client.26222 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:08.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.849+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5574ff355c60 con 0x5574ff2cfe40 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6c0053e0 con 0x5574ff2cfe40 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fac6c039470 con 0x5574ff2cfe40 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 10 client.26222 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 20 client.26222 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff356130 0x5574ff358520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 4 client.26222 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574ff23de60 con 0x5574ff356130 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.850+0000 7fac7d200640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff356130 0x5574ff358520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac7d200640 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff356130 0x5574ff358520 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=644e45008bf0e2c7 server_cookie=1757a4bb98a1ea4b in_seq=0 out_seq=0 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac5f400640 10 client.26222 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:08.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac5f400640 1 -- 172.21.15.59:0/3473198098 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574ff23de60 con 0x5574ff356130 2024-09-05T12:03:08.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac5f400640 10 client.26222 handle_command_reply: tid=0 2024-09-05T12:03:08.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 client.26222 shutdown 2024-09-05T12:03:08.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 msgr2=0x5574ff2e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b620 tx=0x7fac6c005c80 comp rx=0 tx=0).stop 2024-09-05T12:03:08.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 shutdown_connections 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5574ff356130 0x5574ff358520 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5574ff2db020 0x5574ff2dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574ff2d0760 0x5574ff2de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac5f400640 0 client.26222 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 --2- 172.21.15.59:0/3473198098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574ff2cfe40 0x5574ff2e3540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.851+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 >> 172.21.15.59:0/3473198098 conn(0x5574ff2cc3d0 msgr2=0x5574ff2da010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 shutdown_connections 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac82fc4740 1 -- 172.21.15.59:0/3473198098 wait complete. 2024-09-05T12:03:08.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac82fc4740 20 client.26222 trim_cache size 0 max 16384 2024-09-05T12:03:08.860 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:03:08.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 msgr2=0x7fac600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 0x7fac600a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac64031550 tx=0x7fac640095a0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 msgr2=0x7fac7819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac7807aa10 tx=0x7fac7000ef10 comp rx=0 tx=0).stop 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 shutdown_connections 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac600a2b80 0x7fac600a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac78112870 0x7fac7819e710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac78077040 0x7fac781a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 --2- 172.21.15.59:0/1319000506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac78076720 0x7fac781a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.852+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 >> 172.21.15.59:0/1319000506 conn(0x7fac780ff8d0 msgr2=0x7fac78101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.853+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 shutdown_connections 2024-09-05T12:03:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:08.853+0000 7fac7f000640 1 -- 172.21.15.59:0/1319000506 wait complete. 2024-09-05T12:03:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:10 smithi089 ceph-mon[34499]: pgmap v559: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:10 smithi059 ceph-mon[30025]: pgmap v559: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:10 smithi189 ceph-mon[34772]: pgmap v559: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:10.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:03:10.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:03:11.116 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:11.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.700+0000 7f8284e00640 1 -- 172.21.15.59:0/3771642622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280112a50 msgr2=0x7f8280114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.700+0000 7f8284e00640 1 --2- 172.21.15.59:0/3771642622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280112a50 0x7f8280114e40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8278009ca0 tx=0x7f827802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 -- 172.21.15.59:0/3771642622 shutdown_connections 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 --2- 172.21.15.59:0/3771642622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280112a50 0x7f8280114e40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 --2- 172.21.15.59:0/3771642622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280077040 0x7f8280112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 --2- 172.21.15.59:0/3771642622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280076720 0x7f8280076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 -- 172.21.15.59:0/3771642622 >> 172.21.15.59:0/3771642622 conn(0x7f82800ffa50 msgr2=0x7f8280101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 -- 172.21.15.59:0/3771642622 shutdown_connections 2024-09-05T12:03:11.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.701+0000 7f8284e00640 1 -- 172.21.15.59:0/3771642622 wait complete. 2024-09-05T12:03:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 Processor -- start 2024-09-05T12:03:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 -- start start 2024-09-05T12:03:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 0x7f82801a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280112a50 0x7f828019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82801185e0 con 0x7f8280076720 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8280118460 con 0x7f8280077040 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.702+0000 7f8284e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8280118760 con 0x7f8280112a50 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39214/0 (socket says 172.21.15.59:39214) 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f827de00640 1 -- 172.21.15.59:0/2816623799 learned_addr learned my addr 172.21.15.59:0/2816623799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 0x7f82801a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f827e800640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280112a50 0x7f828019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 -- 172.21.15.59:0/2816623799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280112a50 msgr2=0x7f828019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280112a50 0x7f828019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 -- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 msgr2=0x7f82801a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f8277e00640 1 -- 172.21.15.59:0/2816623799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828019efa0 con 0x7f8280077040 2024-09-05T12:03:11.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.703+0000 7f827de00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8277e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 0x7f82801a4bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9f0 tx=0x7f826800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8276a00640 1 -- 172.21.15.59:0/2816623799 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800ccc0 con 0x7f8280077040 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8276a00640 1 -- 172.21.15.59:0/2816623799 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8268010910 con 0x7f8280077040 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8276a00640 1 -- 172.21.15.59:0/2816623799 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268018960 con 0x7f8280077040 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828019f290 con 0x7f8280077040 2024-09-05T12:03:11.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.704+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82801ab580 con 0x7f8280077040 2024-09-05T12:03:11.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 Processor -- start 2024-09-05T12:03:11.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 -- start start 2024-09-05T12:03:11.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae980090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563eae8d9290 con 0x563eae9733e0 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563eae880b40 con 0x563eae97dca0 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.708+0000 7f828785d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563eae8d5250 con 0x563eae972ac0 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f8276a00640 1 -- 172.21.15.59:0/2816623799 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8268010ac0 con 0x7f8280077040 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39224/0 (socket says 172.21.15.59:39224) 2024-09-05T12:03:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827de00640 1 -- 172.21.15.59:0/580051477 learned_addr learned my addr 172.21.15.59:0/580051477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43816/0 (socket says 172.21.15.59:43816) 2024-09-05T12:03:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 -- 172.21.15.59:0/580051477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 msgr2=0x563eae980090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f8277e00640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae980090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae980090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 -- 172.21.15.59:0/580051477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 msgr2=0x563eae97d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827e800640 1 -- 172.21.15.59:0/580051477 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eae64a6a0 con 0x563eae972ac0 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f8277e00640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae980090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.709+0000 7f827de00640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.710+0000 7f827e800640 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f827802aeb0 tx=0x7f8278005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708bf3e8a6463665 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.710+0000 7f826f400640 1 -- 172.21.15.59:0/580051477 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278031db0 con 0x563eae972ac0 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.710+0000 7f826f400640 1 -- 172.21.15.59:0/580051477 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278036030 con 0x563eae972ac0 2024-09-05T12:03:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.710+0000 7f826f400640 1 -- 172.21.15.59:0/580051477 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780042d0 con 0x563eae972ac0 2024-09-05T12:03:11.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.710+0000 7f8276a00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 0x7f82640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f8276a00640 1 -- 172.21.15.59:0/2816623799 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f82680cf7d0 con 0x7f8280077040 2024-09-05T12:03:11.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 -- 172.21.15.59:0/580051477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 msgr2=0x563eae972ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f827802aeb0 tx=0x7f8278005ef0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f827de00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 0x7f82640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 -- 172.21.15.59:0/580051477 shutdown_connections 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae980090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae97d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 --2- 172.21.15.59:0/580051477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae972ac0 0x563eae972ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 -- 172.21.15.59:0/580051477 >> 172.21.15.59:0/580051477 conn(0x563eae96f050 msgr2=0x563eae96f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 -- 172.21.15.59:0/580051477 shutdown_connections 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.711+0000 7f828785d740 1 -- 172.21.15.59:0/580051477 wait complete. 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 -- start start 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f827de00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 0x7f82640a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82700127e0 tx=0x7f8270005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 Processor -- start 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae97d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae97ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563eae980730 con 0x563eae972ac0 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563eae893fa0 con 0x563eae97dca0 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.712+0000 7f828785d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563eae9808b0 con 0x563eae9733e0 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f828785d740 10 client.0 _ll_register_callbacks cb 0x563eae5e7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:11.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae97d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43832/0 (socket says 172.21.15.59:43832) 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 -- 172.21.15.59:0/2937418346 learned_addr learned my addr 172.21.15.59:0/2937418346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae97d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39226/0 (socket says 172.21.15.59:39226) 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f827e800640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae97ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 -- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 msgr2=0x563eae97ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae97ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 -- 172.21.15.59:0/2937418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 msgr2=0x563eae97d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae97d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.713+0000 7f8277e00640 1 -- 172.21.15.59:0/2937418346 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eae97b650 con 0x563eae9733e0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f8277e00640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f826809b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e8598b70fc5210a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f8263e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a4830 con 0x563eae9733e0 2024-09-05T12:03:11.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f82680cfbc0 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8278036030 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f82680cfdc0 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f828785d740 10 client.26161 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.714+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563eae98c5e0 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278036030 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f82680979c0 con 0x563eae9733e0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 10 client.26161 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 20 client.26161 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae98daf0 0x563eae98ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 4 client.26161 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eae8e6800 con 0x563eae98daf0 2024-09-05T12:03:11.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.715+0000 7f827de00640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae98daf0 0x563eae98ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.716+0000 7f827de00640 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae98daf0 0x563eae98ff00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9f664e0b7350b server_cookie=40002db997746c5e in_seq=0 out_seq=0 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.716+0000 7f8263e00640 10 client.26161 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f8263e00640 1 -- 172.21.15.59:0/2937418346 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563eae8e6800 con 0x563eae98daf0 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f8263e00640 10 client.26161 handle_command_reply: tid=0 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 client.26161 shutdown 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 msgr2=0x563eae97a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f826809b650 comp rx=0 tx=0).stop 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 shutdown_connections 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae97dca0 0x563eae97ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae9733e0 0x563eae97a8c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae98daf0 0x563eae98ff00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 --2- 172.21.15.59:0/2937418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae97d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 >> 172.21.15.59:0/2937418346 conn(0x563eae96f050 msgr2=0x563eae974d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f8263e00640 0 client.26161 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:11.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 shutdown_connections 2024-09-05T12:03:11.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.717+0000 7f828785d740 1 -- 172.21.15.59:0/2937418346 wait complete. 2024-09-05T12:03:11.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.718+0000 7f828785d740 20 client.26161 trim_cache size 0 max 16384 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.735+0000 7f828785d740 1 Processor -- start 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 -- start start 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae972ac0 0x563eae9957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae977b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b3500 con 0x563eae9733e0 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b5d80 con 0x563eae972ac0 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f828785d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b3b70 con 0x563eae97dca0 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f8277e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f827e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae977b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f8277e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39240/0 (socket says 172.21.15.59:39240) 2024-09-05T12:03:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f8277e00640 1 -- 172.21.15.59:0/991481046 learned_addr learned my addr 172.21.15.59:0/991481046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.736+0000 7f827de00640 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae972ac0 0x563eae9957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 -- 172.21.15.59:0/991481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 msgr2=0x563eae977b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae977b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 -- 172.21.15.59:0/991481046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae972ac0 msgr2=0x563eae9957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae972ac0 0x563eae9957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 -- 172.21.15.59:0/991481046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eae9804c0 con 0x563eae9733e0 2024-09-05T12:03:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f8277e00640 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f826809bc00 tx=0x7f82680a00f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b74c487f08901d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f826ea00640 1 -- 172.21.15.59:0/991481046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a4830 con 0x563eae9733e0 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f826ea00640 1 -- 172.21.15.59:0/991481046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82680cf420 con 0x563eae9733e0 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.737+0000 7f826ea00640 1 -- 172.21.15.59:0/991481046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826809b320 con 0x563eae9733e0 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- 172.21.15.59:0/991481046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 msgr2=0x563eae995d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f826809bc00 tx=0x7f82680a00f0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- 172.21.15.59:0/991481046 shutdown_connections 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae977b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae9733e0 0x563eae995d10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 --2- 172.21.15.59:0/991481046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae972ac0 0x563eae9957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- 172.21.15.59:0/991481046 >> 172.21.15.59:0/991481046 conn(0x563eae96f050 msgr2=0x563eae9799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- 172.21.15.59:0/991481046 shutdown_connections 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- 172.21.15.59:0/991481046 wait complete. 2024-09-05T12:03:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.738+0000 7f828785d740 1 -- start start 2024-09-05T12:03:11.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 Processor -- start 2024-09-05T12:03:11.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 0x563eae981710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae981e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b4240 con 0x563eae972ac0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b56b0 con 0x563eae9733e0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82801b4910 con 0x563eae97dca0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f828785d740 10 client.0 _ll_register_callbacks cb 0x563eae97b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f8277e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 0x563eae981710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f827de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39242/0 (socket says 172.21.15.59:39242) 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.739+0000 7f827de00640 1 -- 172.21.15.59:0/3033644208 learned_addr learned my addr 172.21.15.59:0/3033644208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f827e800640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae981e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 -- 172.21.15.59:0/3033644208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 msgr2=0x563eae981e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae981e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 -- 172.21.15.59:0/3033644208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 msgr2=0x563eae9863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 -- 172.21.15.59:0/3033644208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eae9f77a0 con 0x563eae9733e0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f827de00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8277e00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 0x563eae981710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f8268097e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05a0aac8d81240a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8263e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826809b2c0 con 0x563eae9733e0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8268010430 con 0x563eae9733e0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.740+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82680cf420 con 0x563eae9733e0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f82680a4830 con 0x563eae9733e0 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 10 client.26237 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:11.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563eae9f8980 con 0x563eae9733e0 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82680cf420 con 0x563eae9733e0 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f82680a11e0 con 0x563eae9733e0 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 10 client.26237 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 20 client.26237 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae9f8e00 0x563eae9fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 4 client.26237 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.741+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563eae8e0d10 con 0x563eae9f8e00 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.742+0000 7f827de00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae9f8e00 0x563eae9fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.742+0000 7f827de00640 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae9f8e00 0x563eae9fb220 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523a8ccb3afa3045 server_cookie=104536ea1f1beffb in_seq=0 out_seq=0 2024-09-05T12:03:11.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.742+0000 7f8263e00640 10 client.26237 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f8263e00640 1 -- 172.21.15.59:0/3033644208 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563eae8e0d10 con 0x563eae9f8e00 2024-09-05T12:03:11.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f8263e00640 10 client.26237 handle_command_reply: tid=0 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 client.26237 shutdown 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 msgr2=0x563eae981710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 0x563eae981710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f8268097e50 comp rx=0 tx=0).stop 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 shutdown_connections 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eae97dca0 0x563eae981e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563eae9733e0 0x563eae981710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.743+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563eae9f8e00 0x563eae9fb220 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f828785d740 1 --2- 172.21.15.59:0/3033644208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563eae972ac0 0x563eae9863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 >> 172.21.15.59:0/3033644208 conn(0x563eae96f050 msgr2=0x563eae9756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f8263e00640 0 client.26237 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 shutdown_connections 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f828785d740 1 -- 172.21.15.59:0/3033644208 wait complete. 2024-09-05T12:03:11.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f828785d740 20 client.26237 trim_cache size 0 max 16384 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "3c90532b-6352-49d9-aa35-a34b4e832cfd", 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:11.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 msgr2=0x7f82640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 0x7f82640a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82700127e0 tx=0x7f8270005f90 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 msgr2=0x7f82801a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.744+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 0x7f82801a4bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9f0 tx=0x7f826800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 shutdown_connections 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f82640a2b80 0x7f82640a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8280112a50 0x7f828019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8280077040 0x7f82801a4bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 --2- 172.21.15.59:0/2816623799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8280076720 0x7f82801a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 >> 172.21.15.59:0/2816623799 conn(0x7f82800ffa50 msgr2=0x7f8280101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 shutdown_connections 2024-09-05T12:03:11.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:11.745+0000 7f8284e00640 1 -- 172.21.15.59:0/2816623799 wait complete. 2024-09-05T12:03:12.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:12 smithi089 ceph-mon[34499]: pgmap v560: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:03:12.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:12 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:03:12.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:12 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:12 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:03:12.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:12 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:12 smithi059 ceph-mon[30025]: pgmap v560: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:03:12.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:12 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:03:12.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:12 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:12 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:03:12.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:12 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:12 smithi189 ceph-mon[34772]: pgmap v560: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:03:12.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:12 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:03:12.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:12 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:12 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:03:12.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:12 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:12.625 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:03:13.002 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 -- 172.21.15.59:0/3463344463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 msgr2=0x7f074810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 --2- 172.21.15.59:0/3463344463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074810de80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0730009900 tx=0x7f073002a480 comp rx=0 tx=0).stop 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 -- 172.21.15.59:0/3463344463 shutdown_connections 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 --2- 172.21.15.59:0/3463344463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 0x7f0748110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 --2- 172.21.15.59:0/3463344463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074810de80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 --2- 172.21.15.59:0/3463344463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f0748074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 -- 172.21.15.59:0/3463344463 >> 172.21.15.59:0/3463344463 conn(0x7f07480fd8c0 msgr2=0x7f07480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 -- 172.21.15.59:0/3463344463 shutdown_connections 2024-09-05T12:03:13.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.569+0000 7f074cc00640 1 -- 172.21.15.59:0/3463344463 wait complete. 2024-09-05T12:03:13.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.570+0000 7f074cc00640 1 Processor -- start 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.570+0000 7f074cc00640 1 -- start start 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.570+0000 7f074cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f074cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f074cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 0x7f074819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f074cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07481141b0 con 0x7f0748074f40 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f074cc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0748114030 con 0x7f0748074620 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f074cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0748114330 con 0x7f074810e3c0 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f073fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51228/0 (socket says 172.21.15.59:51228) 2024-09-05T12:03:13.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 -- 172.21.15.59:0/700699625 learned_addr learned my addr 172.21.15.59:0/700699625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0746800640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 0x7f074819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 -- 172.21.15.59:0/700699625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 msgr2=0x7f074819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 0x7f074819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 -- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 msgr2=0x7f074819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.571+0000 7f0745e00640 1 -- 172.21.15.59:0/700699625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074819f620 con 0x7f0748074620 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f0745e00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f073800e9f0 tx=0x7f073800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f073ea00640 1 -- 172.21.15.59:0/700699625 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073800cd80 con 0x7f0748074620 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f073ea00640 1 -- 172.21.15.59:0/700699625 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0738010910 con 0x7f0748074620 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074819f910 con 0x7f0748074620 2024-09-05T12:03:13.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f073ea00640 1 -- 172.21.15.59:0/700699625 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380188f0 con 0x7f0748074620 2024-09-05T12:03:13.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.572+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a71f0 con 0x7f0748074620 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.575+0000 7f074f6a9740 1 Processor -- start 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.575+0000 7f073ea00640 1 -- 172.21.15.59:0/700699625 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0738010ac0 con 0x7f0748074620 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.575+0000 7f074f6a9740 1 -- start start 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562df92283e0 con 0x562df92c1b90 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562df91cfd80 con 0x562df92ccd70 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f074f6a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562df92243a0 con 0x562df92c24b0 2024-09-05T12:03:13.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f0746800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f0745e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f073fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f0746800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39270/0 (socket says 172.21.15.59:39270) 2024-09-05T12:03:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f0746800640 1 -- 172.21.15.59:0/2959340925 learned_addr learned my addr 172.21.15.59:0/2959340925 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.576+0000 7f073fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51230/0 (socket says 172.21.15.59:51230) 2024-09-05T12:03:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 -- 172.21.15.59:0/2959340925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 msgr2=0x562df92cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 -- 172.21.15.59:0/2959340925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 msgr2=0x562df92c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 -- 172.21.15.59:0/2959340925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df91eeae0 con 0x562df92ccd70 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073fe00640 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f073002aa70 tx=0x7f0730005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1901477ae226814 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073ea00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 0x7f072c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f073ea00640 1 -- 172.21.15.59:0/700699625 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f07380cf8a0 con 0x7f0748074620 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f0737000640 1 -- 172.21.15.59:0/2959340925 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730035040 con 0x562df92ccd70 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f0737000640 1 -- 172.21.15.59:0/2959340925 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0730004400 con 0x562df92ccd70 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f0737000640 1 -- 172.21.15.59:0/2959340925 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300035f0 con 0x562df92ccd70 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.577+0000 7f0746800640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 0x7f072c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 -- 172.21.15.59:0/2959340925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 msgr2=0x562df92cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f073002aa70 tx=0x7f0730005de0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f0746800640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 0x7f072c0a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0740009fd0 tx=0x7f0740009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 -- 172.21.15.59:0/2959340925 shutdown_connections 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92cf160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 --2- 172.21.15.59:0/2959340925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 -- 172.21.15.59:0/2959340925 >> 172.21.15.59:0/2959340925 conn(0x562df92be120 msgr2=0x562df92be500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 -- 172.21.15.59:0/2959340925 shutdown_connections 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.578+0000 7f074f6a9740 1 -- 172.21.15.59:0/2959340925 wait complete. 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 -- start start 2024-09-05T12:03:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 Processor -- start 2024-09-05T12:03:13.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562df92cf980 con 0x562df92c24b0 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562df91e30f0 con 0x562df92c1b90 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562df92cfb00 con 0x562df92ccd70 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.579+0000 7f074f6a9740 10 client.0 _ll_register_callbacks cb 0x562df91d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f073fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0746800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51232/0 (socket says 172.21.15.59:51232) 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f073fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39280/0 (socket says 172.21.15.59:39280) 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 -- 172.21.15.59:0/1306447497 learned_addr learned my addr 172.21.15.59:0/1306447497 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 -- 172.21.15.59:0/1306447497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 msgr2=0x562df92ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 -- 172.21.15.59:0/1306447497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 msgr2=0x562df92ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 -- 172.21.15.59:0/1306447497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df8f993c0 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f0745e00640 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f073809b990 tx=0x7f073809bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ea9a27ccb78b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.580+0000 7f072b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073809b2e0 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0730004400 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0738010430 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f073809bc90 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f074f6a9740 10 client.26249 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562df92db640 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0730004400 con 0x562df92c1b90 2024-09-05T12:03:13.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0738018ab0 con 0x562df92c1b90 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f074f6a9740 10 client.26249 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.581+0000 7f074f6a9740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f074f6a9740 20 client.26249 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df92dcb00 0x562df92def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f074f6a9740 4 client.26249 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df9235950 con 0x562df92dcb00 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f073fe00640 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df92dcb00 0x562df92def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f073fe00640 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df92dcb00 0x562df92def50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4ee850f9b45ff8 server_cookie=497ddce1266a4e05 in_seq=0 out_seq=0 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.582+0000 7f072b400640 10 client.26249 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f072b400640 1 -- 172.21.15.59:0/1306447497 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562df9235950 con 0x562df92dcb00 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f072b400640 10 client.26249 handle_command_reply: tid=0 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 client.26249 shutdown 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 msgr2=0x562df92c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f073809b990 tx=0x7f073809bb20 comp rx=0 tx=0).stop 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 shutdown_connections 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df92dcb00 0x562df92def50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 --2- 172.21.15.59:0/1306447497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c9c70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f072b400640 0 client.26249 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.583+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 >> 172.21.15.59:0/1306447497 conn(0x562df92be120 msgr2=0x562df92c3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.584+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 shutdown_connections 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.584+0000 7f074f6a9740 1 -- 172.21.15.59:0/1306447497 wait complete. 2024-09-05T12:03:13.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.584+0000 7f074f6a9740 20 client.26249 trim_cache size 0 max 16384 2024-09-05T12:03:13.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 Processor -- start 2024-09-05T12:03:13.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 -- start start 2024-09-05T12:03:13.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07481af0d0 con 0x562df92c24b0 2024-09-05T12:03:13.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b1950 con 0x562df92c1b90 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f074f6a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07481af740 con 0x562df92ccd70 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.603+0000 7f073fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f0745e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51238/0 (socket says 172.21.15.59:51238) 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f0745e00640 1 -- 172.21.15.59:0/3885280414 learned_addr learned my addr 172.21.15.59:0/3885280414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39282/0 (socket says 172.21.15.59:39282) 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f0746800640 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 -- 172.21.15.59:0/3885280414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 msgr2=0x562df92e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 -- 172.21.15.59:0/3885280414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 msgr2=0x562df92c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 -- 172.21.15.59:0/3885280414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df92bdbc0 con 0x562df92c24b0 2024-09-05T12:03:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f073fe00640 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0730031550 tx=0x7f073002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37c24b1e8ac76c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.604+0000 7f0736600640 1 -- 172.21.15.59:0/3885280414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730035040 con 0x562df92c24b0 2024-09-05T12:03:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f0736600640 1 -- 172.21.15.59:0/3885280414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0730004490 con 0x562df92c24b0 2024-09-05T12:03:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f0736600640 1 -- 172.21.15.59:0/3885280414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730003850 con 0x562df92c24b0 2024-09-05T12:03:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 -- 172.21.15.59:0/3885280414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 msgr2=0x562df92e4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0730031550 tx=0x7f073002a480 comp rx=0 tx=0).stop 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 -- 172.21.15.59:0/3885280414 shutdown_connections 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92ccd70 0x562df92e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c24b0 0x562df92e4840 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 --2- 172.21.15.59:0/3885280414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92c1b90 0x562df92c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 -- 172.21.15.59:0/3885280414 >> 172.21.15.59:0/3885280414 conn(0x562df92be120 msgr2=0x562df92ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 -- 172.21.15.59:0/3885280414 shutdown_connections 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.605+0000 7f074f6a9740 1 -- 172.21.15.59:0/3885280414 wait complete. 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 -- start start 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 Processor -- start 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07481afe10 con 0x562df92c1b90 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b1280 con 0x562df92ccd70 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b04e0 con 0x562df92c24b0 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.606+0000 7f074f6a9740 10 client.0 _ll_register_callbacks cb 0x7f0738097f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:13.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f073fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39294/0 (socket says 172.21.15.59:39294) 2024-09-05T12:03:13.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 -- 172.21.15.59:0/4233244801 learned_addr learned my addr 172.21.15.59:0/4233244801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:13.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0746800640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 -- 172.21.15.59:0/4233244801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 msgr2=0x562df92c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 -- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 msgr2=0x562df92c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 -- 172.21.15.59:0/4233244801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df92c78b0 con 0x562df92c1b90 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f073fe00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:13.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f0745e00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f073800c6a0 tx=0x7f073809b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32e51811f1cf1a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:13.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.607+0000 7f072b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:13.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380b8060 con 0x562df92c1b90 2024-09-05T12:03:13.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f07380ba650 con 0x562df92c1b90 2024-09-05T12:03:13.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0730004490 con 0x562df92c1b90 2024-09-05T12:03:13.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f073809b5f0 con 0x562df92c1b90 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 10 client.16248 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562df9347900 con 0x562df92c1b90 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0730004490 con 0x562df92c1b90 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f07380a4b50 con 0x562df92c1b90 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 10 client.16248 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:13.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:13.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 20 client.16248 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:13.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df9348070 0x562df934a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:13.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 4 client.16248 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:03:13.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.608+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562df922fe60 con 0x562df9348070 2024-09-05T12:03:13.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.609+0000 7f073fe00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df9348070 0x562df934a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.609+0000 7f073fe00640 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df9348070 0x562df934a490 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671d6558d95d4718 server_cookie=59e59b427ad34b88 in_seq=0 out_seq=0 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.609+0000 7f072b400640 10 client.16248 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f072b400640 1 -- 172.21.15.59:0/4233244801 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562df922fe60 con 0x562df9348070 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f072b400640 10 client.16248 handle_command_reply: tid=0 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 client.16248 shutdown 2024-09-05T12:03:13.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 msgr2=0x562df92d5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f073800c6a0 tx=0x7f073809b310 comp rx=0 tx=0).stop 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 shutdown_connections 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562df92ccd70 0x562df92c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562df92c24b0 0x562df92c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562df9348070 0x562df934a490 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 --2- 172.21.15.59:0/4233244801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562df92c1b90 0x562df92d5580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 >> 172.21.15.59:0/4233244801 conn(0x562df92be120 msgr2=0x562df92bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f072b400640 0 client.16248 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:13.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 shutdown_connections 2024-09-05T12:03:13.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 1 -- 172.21.15.59:0/4233244801 wait complete. 2024-09-05T12:03:13.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.610+0000 7f074f6a9740 20 client.16248 trim_cache size 0 max 16384 2024-09-05T12:03:13.628 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:03:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:03:13.628 INFO:teuthology.orchestra.run.smithi059.stdout: "3c90532b-6352-49d9-aa35-a34b4e832cfd": { 2024-09-05T12:03:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:03:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "3c90532b-6352-49d9-aa35-a34b4e832cfd", 2024-09-05T12:03:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:03:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:13.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:13.630 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:13.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 msgr2=0x7f072c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 0x7f072c0a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0740009fd0 tx=0x7f0740009500 comp rx=0 tx=0).stop 2024-09-05T12:03:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 msgr2=0x7f074819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f073800e9f0 tx=0x7f073800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 shutdown_connections 2024-09-05T12:03:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f072c0a2c00 0x7f072c0a50c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f074810e3c0 0x7f074819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0748074f40 0x7f074819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 --2- 172.21.15.59:0/700699625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748074620 0x7f074819a670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 >> 172.21.15.59:0/700699625 conn(0x7f07480fd8c0 msgr2=0x7f07480fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 shutdown_connections 2024-09-05T12:03:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:13.611+0000 7f074cc00640 1 -- 172.21.15.59:0/700699625 wait complete. 2024-09-05T12:03:13.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:03:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:14 smithi089 ceph-mon[34499]: pgmap v561: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:14 smithi059 ceph-mon[30025]: pgmap v561: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:14.563 INFO:tasks.cephfs.filesystem:scrub status for tag:3c90532b-6352-49d9-aa35-a34b4e832cfd - {'path': '/', 'tag': '3c90532b-6352-49d9-aa35-a34b4e832cfd', 'options': 'recursive,force'} 2024-09-05T12:03:14.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:14 smithi189 ceph-mon[34772]: pgmap v561: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:03:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 -- 172.21.15.59:0/1106877225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074590 msgr2=0x7f1d18074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1106877225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074590 0x7f1d18074970 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802a480 comp rx=0 tx=0).stop 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 -- 172.21.15.59:0/1106877225 shutdown_connections 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1106877225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d18114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1106877225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074eb0 0x7f1d18112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1106877225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074590 0x7f1d18074970 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 -- 172.21.15.59:0/1106877225 >> 172.21.15.59:0/1106877225 conn(0x7f1d180fd8c0 msgr2=0x7f1d180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 -- 172.21.15.59:0/1106877225 shutdown_connections 2024-09-05T12:03:15.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.233+0000 7f1d1de00640 1 -- 172.21.15.59:0/1106877225 wait complete. 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.234+0000 7f1d1de00640 1 Processor -- start 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 -- start start 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074590 0x7f1d181a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d1819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181185a0 con 0x7f1d18074eb0 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18118420 con 0x7f1d18112b90 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d1de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18118720 con 0x7f1d18074590 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d0e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d0e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39322/0 (socket says 172.21.15.59:39322) 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d0e000640 1 -- 172.21.15.59:0/1439707550 learned_addr learned my addr 172.21.15.59:0/1439707550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:15.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d17400640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d1819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.235+0000 7f1d16a00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074590 0x7f1d181a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 -- 172.21.15.59:0/1439707550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074590 msgr2=0x7f1d181a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074590 0x7f1d181a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 -- 172.21.15.59:0/1439707550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 msgr2=0x7f1d1819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d1819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 -- 172.21.15.59:0/1439707550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1819f080 con 0x7f1d18074eb0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d17400640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d1819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d0e000640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000d950 tx=0x7f1d0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00017030 con 0x7f1d18074eb0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d00004020 con 0x7f1d18074eb0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d000047a0 con 0x7f1d18074eb0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.236+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1819f370 con 0x7f1d18074eb0 2024-09-05T12:03:15.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.237+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181ab660 con 0x7f1d18074eb0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.238+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d18074970 con 0x7f1d18074eb0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.239+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d000041d0 con 0x7f1d18074eb0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.241+0000 7f1d1ca00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 0x7f1cfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.241+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1d000cedf0 con 0x7f1d18074eb0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.241+0000 7f1d16a00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 0x7f1cfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.242+0000 7f1d16a00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 0x7f1cfc0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d08000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:15.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.246+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d0009b8e0 con 0x7f1d18074eb0 2024-09-05T12:03:15.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:15 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:03:15.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:15 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:03:15.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.467+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d1802d7c0 con 0x7f1d18074eb0 2024-09-05T12:03:15.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.467+0000 7f1d1ca00640 1 -- 172.21.15.59:0/1439707550 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f1d000b9e20 con 0x7f1d18074eb0 2024-09-05T12:03:15.469 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:03:15.470 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:03:15.470 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:03:15.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 msgr2=0x7f1cfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 0x7f1cfc0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d08000bc0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 msgr2=0x7f1d181a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000d950 tx=0x7f1d0000de10 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 shutdown_connections 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1cfc0a2b80 0x7f1cfc0a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d18112b90 0x7f1d1819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18074eb0 0x7f1d181a4cd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 --2- 172.21.15.59:0/1439707550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d18074590 0x7f1d181a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 >> 172.21.15.59:0/1439707550 conn(0x7f1d180fd8c0 msgr2=0x7f1d180ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 shutdown_connections 2024-09-05T12:03:15.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:15.470+0000 7f1d1de00640 1 -- 172.21.15.59:0/1439707550 wait complete. 2024-09-05T12:03:15.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:15 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:03:16.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id a3622184 --release --await' 2024-09-05T12:03:16.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id a3622184 --release --await 2024-09-05T12:03:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:16 smithi089 ceph-mon[34499]: pgmap v562: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T12:03:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:16 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1439707550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:16.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:16 smithi059 ceph-mon[30025]: pgmap v562: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T12:03:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:16 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1439707550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:16 smithi189 ceph-mon[34772]: pgmap v562: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-05T12:03:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:16 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1439707550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 -- 172.21.15.59:0/3934262730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 msgr2=0x7f47cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 --2- 172.21.15.59:0/3934262730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc104210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009900 tx=0x7f47bc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 -- 172.21.15.59:0/3934262730 shutdown_connections 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 --2- 172.21.15.59:0/3934262730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc108f60 0x7f47cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 --2- 172.21.15.59:0/3934262730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc1047e0 0x7f47cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 --2- 172.21.15.59:0/3934262730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc104210 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.171+0000 7f47d0c00640 1 -- 172.21.15.59:0/3934262730 >> 172.21.15.59:0/3934262730 conn(0x7f47cc0ff8d0 msgr2=0x7f47cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.172+0000 7f47d0c00640 1 -- 172.21.15.59:0/3934262730 shutdown_connections 2024-09-05T12:03:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.172+0000 7f47d0c00640 1 -- 172.21.15.59:0/3934262730 wait complete. 2024-09-05T12:03:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.172+0000 7f47d0c00640 1 Processor -- start 2024-09-05T12:03:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 -- start start 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc1047e0 0x7f47cc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc108f60 0x7f47cc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113fc0 con 0x7f47cc103e30 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113e40 con 0x7f47cc108f60 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47d0c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc114140 con 0x7f47cc1047e0 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39338/0 (socket says 172.21.15.59:39338) 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47cb400640 1 -- 172.21.15.59:0/2508108622 learned_addr learned my addr 172.21.15.59:0/2508108622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.173+0000 7f47caa00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc1047e0 0x7f47cc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cbe00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc108f60 0x7f47cc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 -- 172.21.15.59:0/2508108622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc1047e0 msgr2=0x7f47cc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc1047e0 0x7f47cc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 -- 172.21.15.59:0/2508108622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc108f60 msgr2=0x7f47cc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc108f60 0x7f47cc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 -- 172.21.15.59:0/2508108622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19f450 con 0x7f47cc103e30 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.174+0000 7f47cb400640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006010 tx=0x7f47bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.175+0000 7f47c3e00640 1 -- 172.21.15.59:0/2508108622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc03f030 con 0x7f47cc103e30 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.175+0000 7f47c3e00640 1 -- 172.21.15.59:0/2508108622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc004440 con 0x7f47cc103e30 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.175+0000 7f47c3e00640 1 -- 172.21.15.59:0/2508108622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc035070 con 0x7f47cc103e30 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.175+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19f6e0 con 0x7f47cc103e30 2024-09-05T12:03:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.175+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1a7020 con 0x7f47cc103e30 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 Processor -- start 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47c3e00640 1 -- 172.21.15.59:0/2508108622 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f47bc042030 con 0x7f47cc103e30 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 -- start start 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c90af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5565d3be9650 con 0x5565d3c83520 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565d3b90f10 con 0x5565d3c8e700 2024-09-05T12:03:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.178+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565d3be5610 con 0x5565d3c83e40 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39346/0 (socket says 172.21.15.59:39346) 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 -- 172.21.15.59:0/1354073452 learned_addr learned my addr 172.21.15.59:0/1354073452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47caa00640 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c90af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 -- 172.21.15.59:0/1354073452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 msgr2=0x5565d3c8e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 -- 172.21.15.59:0/1354073452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 msgr2=0x5565d3c90af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c90af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47cbe00640 1 -- 172.21.15.59:0/1354073452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d38f8580 con 0x5565d3c83520 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.179+0000 7f47caa00640 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c90af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47cbe00640 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009f50 tx=0x7f47c40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ee3b81a954370 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47c3e00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 0x7f47b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47b7e00640 1 -- 172.21.15.59:0/1354073452 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4004d20 con 0x5565d3c83520 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47b7e00640 1 -- 172.21.15.59:0/1354073452 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4004050 con 0x5565d3c83520 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47b7e00640 1 -- 172.21.15.59:0/1354073452 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4005280 con 0x5565d3c83520 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47c3e00640 1 -- 172.21.15.59:0/2508108622 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47bc0ef780 con 0x7f47cc103e30 2024-09-05T12:03:17.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.180+0000 7f47cb400640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 0x7f47b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1354073452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 msgr2=0x5565d3c83900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47cb400640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 0x7f47b00a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f47bc032c00 tx=0x7f47bc0f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009f50 tx=0x7f47c40118d0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1354073452 shutdown_connections 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c90af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1354073452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c83900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1354073452 >> 172.21.15.59:0/1354073452 conn(0x5565d3c7fab0 msgr2=0x5565d3c7fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1354073452 shutdown_connections 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.181+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1354073452 wait complete. 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 -- start start 2024-09-05T12:03:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 Processor -- start 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c8b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5565d3c912d0 con 0x5565d3c83520 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565d3ba4350 con 0x5565d3c8e700 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.182+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565d3c91450 con 0x5565d3c83e40 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47d4e5d740 10 client.0 _ll_register_callbacks cb 0x5565d3bc7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c8b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43950/0 (socket says 172.21.15.59:43950) 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 -- 172.21.15.59:0/2997299519 learned_addr learned my addr 172.21.15.59:0/2997299519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47cbe00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 -- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 msgr2=0x5565d3c8c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c8c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 -- 172.21.15.59:0/2997299519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 msgr2=0x5565d3c8b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c8b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47caa00640 1 -- 172.21.15.59:0/2997299519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d39c2960 con 0x5565d3c83e40 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.183+0000 7f47cbe00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47caa00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47b8012730 tx=0x7f47b8012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b5e84f1275eabb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004030 con 0x5565d3c83e40 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47b80041d0 con 0x5565d3c83e40 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c4004050 con 0x5565d3c83e40 2024-09-05T12:03:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f47b8025050 con 0x5565d3c83e40 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47d4e5d740 10 client.26203 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.184+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565d3c9d060 con 0x5565d3c83e40 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4004050 con 0x5565d3c83e40 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f47b8003a90 con 0x5565d3c83e40 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 10 client.26203 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 10 client.26203 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 20 client.26203 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3c9e520 0x5565d3ca0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 4 client.26203 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565d3bf6bc0 con 0x5565d3c9e520 2024-09-05T12:03:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.185+0000 7f47cbe00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3c9e520 0x5565d3ca0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.186+0000 7f47cbe00640 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3c9e520 0x5565d3ca0950 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba489f20826e6d2a server_cookie=29d7572c49db64ef in_seq=0 out_seq=0 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.186+0000 7f47b4c00640 10 client.26203 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.186+0000 7f47b4c00640 1 -- 172.21.15.59:0/2997299519 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565d3bf6bc0 con 0x5565d3c9e520 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47b4c00640 10 client.26203 handle_command_reply: tid=0 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 client.26203 shutdown 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 msgr2=0x5565d3c8bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47b8012730 tx=0x7f47b8012bf0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 shutdown_connections 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3c8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83e40 0x5565d3c8bc50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3c9e520 0x5565d3ca0950 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/2997299519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83520 0x5565d3c8b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47b4c00640 0 client.26203 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 >> 172.21.15.59:0/2997299519 conn(0x5565d3c7fab0 msgr2=0x5565d3c7ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 shutdown_connections 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.187+0000 7f47d4e5d740 1 -- 172.21.15.59:0/2997299519 wait complete. 2024-09-05T12:03:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.188+0000 7f47d4e5d740 20 client.26203 trim_cache size 0 max 16384 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.205+0000 7f47d4e5d740 1 Processor -- start 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.205+0000 7f47d4e5d740 1 -- start start 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c8d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3ca67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1aee60 con 0x5565d3c83e40 2024-09-05T12:03:17.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b16e0 con 0x5565d3c8e700 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1af4d0 con 0x5565d3c83520 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39356/0 (socket says 172.21.15.59:39356) 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 -- 172.21.15.59:0/457327901 learned_addr learned my addr 172.21.15.59:0/457327901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47cbe00640 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c8d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47cbe00640 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3ca67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 -- 172.21.15.59:0/457327901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 msgr2=0x5565d3c8d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c8d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 -- 172.21.15.59:0/457327901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 msgr2=0x5565d3ca67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.206+0000 7f47caa00640 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3ca67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.207+0000 7f47caa00640 1 -- 172.21.15.59:0/457327901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d3c8dfe0 con 0x5565d3c83e40 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.207+0000 7f47caa00640 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f47b80098d0 tx=0x7f47b80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5717d29661b305 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.207+0000 7f47b7400640 1 -- 172.21.15.59:0/457327901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b802e070 con 0x5565d3c83e40 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.207+0000 7f47b7400640 1 -- 172.21.15.59:0/457327901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8010e00 con 0x5565d3c83e40 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.207+0000 7f47b7400640 1 -- 172.21.15.59:0/457327901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8022040 con 0x5565d3c83e40 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 -- 172.21.15.59:0/457327901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 msgr2=0x5565d3ca6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f47b80098d0 tx=0x7f47b80057d0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 -- 172.21.15.59:0/457327901 shutdown_connections 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c8e700 0x5565d3ca67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c83e40 0x5565d3ca6260 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/457327901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 -- 172.21.15.59:0/457327901 >> 172.21.15.59:0/457327901 conn(0x5565d3c7fab0 msgr2=0x5565d3c8ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 -- 172.21.15.59:0/457327901 shutdown_connections 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.208+0000 7f47d4e5d740 1 -- 172.21.15.59:0/457327901 wait complete. 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 -- start start 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 Processor -- start 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c8e700 0x5565d3c940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1afba0 con 0x5565d3c8e700 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b1010 con 0x5565d3c83e40 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1b0270 con 0x5565d3c83520 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.209+0000 7f47d4e5d740 10 client.0 _ll_register_callbacks cb 0x7f47b8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c8e700 0x5565d3c940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:51302/0 (socket says 172.21.15.59:51302) 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 -- 172.21.15.59:0/1443591592 learned_addr learned my addr 172.21.15.59:0/1443591592 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 -- 172.21.15.59:0/1443591592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 msgr2=0x5565d3c93670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c93670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 -- 172.21.15.59:0/1443591592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c8e700 msgr2=0x5565d3c940f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c8e700 0x5565d3c940f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 -- 172.21.15.59:0/1443591592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d3d09240 con 0x5565d3c83e40 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47caa00640 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000c00 tx=0x7f47b80052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131caa2a315d0365 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.210+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b802e070 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b8010e00 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47b80038c0 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f47b803c050 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 10 client.26276 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565d3d0a4a0 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8010e00 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f47b80106c0 con 0x5565d3c83e40 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 10 client.26276 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 10 client.26276 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.211+0000 7f47d4e5d740 20 client.26276 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:17.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3d0aa80 0x5565d3d0cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:17.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47d4e5d740 4 client.26276 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a3622184", "release": true, "await": true}] 2024-09-05T12:03:17.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a3622184", "release": true, "await": true}) -- 0x5565d3bf10d0 con 0x5565d3d0aa80 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47cbe00640 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3d0aa80 0x5565d3d0cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47cbe00640 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3d0aa80 0x5565d3d0cea0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e887279e6074f4d server_cookie=b6e3ace05f49a32f in_seq=0 out_seq=0 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.212+0000 7f47b4c00640 10 client.26276 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47b4c00640 1 -- 172.21.15.59:0/1443591592 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5565d3bf10d0 con 0x5565d3d0aa80 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47b4c00640 10 client.26276 handle_command_reply: tid=0 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 client.26276 shutdown 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 msgr2=0x5565d3c93bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000c00 tx=0x7f47b80052e0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 shutdown_connections 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5565d3d0aa80 0x5565d3d0cea0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5565d3c8e700 0x5565d3c940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47b4c00640 0 client.26276 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565d3c83e40 0x5565d3c93bb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 --2- 172.21.15.59:0/1443591592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d3c83520 0x5565d3c93670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 >> 172.21.15.59:0/1443591592 conn(0x5565d3c7fab0 msgr2=0x5565d3c85790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 shutdown_connections 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.214+0000 7f47d4e5d740 1 -- 172.21.15.59:0/1443591592 wait complete. 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d4e5d740 20 client.26276 trim_cache size 0 max 16384 2024-09-05T12:03:17.219 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 24, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "a3622184": { 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 24, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:03:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:17.221 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 msgr2=0x7f47b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 0x7f47b00a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f47bc032c00 tx=0x7f47bc0f5530 comp rx=0 tx=0).stop 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 msgr2=0x7f47cc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006010 tx=0x7f47bc004040 comp rx=0 tx=0).stop 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 shutdown_connections 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47b00a2c00 0x7f47b00a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47cc108f60 0x7f47cc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47cc1047e0 0x7f47cc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 --2- 172.21.15.59:0/2508108622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47cc103e30 0x7f47cc19a4a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:17.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 >> 172.21.15.59:0/2508108622 conn(0x7f47cc0ff8d0 msgr2=0x7f47cc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:17.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.215+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 shutdown_connections 2024-09-05T12:03:17.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:17.216+0000 7f47d0c00640 1 -- 172.21.15.59:0/2508108622 wait complete. 2024-09-05T12:03:17.975 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a3622184', total seconds elapsed: 34.2 2024-09-05T12:03:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:18 smithi059 ceph-mon[30025]: pgmap v563: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:03:18.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:18 smithi089 ceph-mon[34499]: pgmap v563: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:03:18.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:18 smithi189 ceph-mon[34772]: pgmap v563: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:03:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:20 smithi089 ceph-mon[34499]: pgmap v564: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:03:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:20 smithi059 ceph-mon[30025]: pgmap v564: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:03:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:20 smithi189 ceph-mon[34772]: pgmap v564: 129 pgs: 129 active+clean; 2.6 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:03:22.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:22 smithi089 ceph-mon[34499]: pgmap v565: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:22.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:22 smithi059 ceph-mon[30025]: pgmap v565: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:22.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:22 smithi189 ceph-mon[34772]: pgmap v565: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:24.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:24 smithi089 ceph-mon[34499]: pgmap v566: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:24.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:24 smithi059 ceph-mon[30025]: pgmap v566: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:24 smithi189 ceph-mon[34772]: pgmap v566: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:26.391 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:26 smithi189 ceph-mon[34772]: pgmap v567: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:26.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:26 smithi089 ceph-mon[34499]: pgmap v567: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:26.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:26 smithi059 ceph-mon[30025]: pgmap v567: 129 pgs: 129 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:03:28.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:28 smithi089 ceph-mon[34499]: pgmap v568: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2024-09-05T12:03:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:28 smithi059 ceph-mon[30025]: pgmap v568: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2024-09-05T12:03:28.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:28 smithi189 ceph-mon[34772]: pgmap v568: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2024-09-05T12:03:30.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:30 smithi089 ceph-mon[34499]: pgmap v569: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 5 op/s 2024-09-05T12:03:30.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:30 smithi059 ceph-mon[30025]: pgmap v569: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 5 op/s 2024-09-05T12:03:30.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:30 smithi189 ceph-mon[34772]: pgmap v569: 129 pgs: 129 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 5 op/s 2024-09-05T12:03:32.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:32 smithi089 ceph-mon[34499]: pgmap v570: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:03:32.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:32 smithi059 ceph-mon[30025]: pgmap v570: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:03:32.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:32 smithi189 ceph-mon[34772]: pgmap v570: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:03:34.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:34 smithi089 ceph-mon[34499]: pgmap v571: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:03:34.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:34 smithi059 ceph-mon[30025]: pgmap v571: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:03:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:34 smithi189 ceph-mon[34772]: pgmap v571: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:03:36.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:36 smithi089 ceph-mon[34499]: pgmap v572: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:36.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:36 smithi059 ceph-mon[30025]: pgmap v572: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:36.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:36 smithi189 ceph-mon[34772]: pgmap v572: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:38 smithi089 ceph-mon[34499]: pgmap v573: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:38.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:38 smithi059 ceph-mon[30025]: pgmap v573: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:38 smithi189 ceph-mon[34772]: pgmap v573: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:40.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:40 smithi089 ceph-mon[34499]: pgmap v574: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:03:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:40 smithi059 ceph-mon[30025]: pgmap v574: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:03:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:40 smithi189 ceph-mon[34772]: pgmap v574: 129 pgs: 129 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:127/600: open fd = 3 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.410 INFO:tasks.workunit.client.0.smithi059.stdout:128/600: open fd = 3 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:129/600: open fd = 3 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:130/600: open fd = 3 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:131/600: open fd = 3 2024-09-05T12:03:42.411 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:132/600: open fd = 3 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:133/600: open fd = 3 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:134/600: open fd = 3 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.412 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:135/600: open fd = 3 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:136/600: open fd = 3 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:137/600: open fd = 3 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.413 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:138/600: open fd = 3 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:139/600: open fd = 3 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:140/600: open fd = 3 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.414 INFO:tasks.workunit.client.0.smithi059.stdout:141/600: open fd = 3 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:142/600: open fd = 3 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:143/600: open fd = 3 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:144/600: open fd = 3 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.415 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:145/600: open fd = 3 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:146/600: open fd = 3 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:147/600: open fd = 3 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.416 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:148/600: open fd = 3 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:149/600: open fd = 3 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:150/600: open fd = 3 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:151/600: open fd = 3 2024-09-05T12:03:42.417 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:152/600: open fd = 3 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:153/600: open fd = 3 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:154/600: open fd = 3 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.418 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:155/600: open fd = 3 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:156/600: open fd = 3 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:157/600: open fd = 3 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.419 INFO:tasks.workunit.client.0.smithi059.stdout:158/600: open fd = 3 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:159/600: open fd = 3 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:160/600: open fd = 3 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:161/600: open fd = 3 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.420 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:162/600: open fd = 3 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:163/600: open fd = 3 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:164/600: open fd = 3 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.421 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:165/600: open fd = 3 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:166/600: open fd = 3 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:167/600: open fd = 3 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.422 INFO:tasks.workunit.client.0.smithi059.stdout:168/600: open fd = 3 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:169/600: open fd = 3 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:170/600: open fd = 3 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:171/600: open fd = 3 2024-09-05T12:03:42.423 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:172/600: open fd = 3 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:173/600: open fd = 3 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:174/600: open fd = 3 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.424 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:175/600: open fd = 3 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:176/600: open fd = 3 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:177/600: open fd = 3 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.425 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:178/600: open fd = 3 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:179/600: open fd = 3 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:180/600: open fd = 3 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.426 INFO:tasks.workunit.client.0.smithi059.stdout:181/600: open fd = 3 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:182/600: open fd = 3 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:183/600: open fd = 3 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:184/600: open fd = 3 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.427 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:185/600: open fd = 3 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:186/600: open fd = 3 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:187/600: open fd = 3 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.428 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.429 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:03:42.429 INFO:tasks.workunit.client.0.smithi059.stdout:188/600: open fd = 3 2024-09-05T12:03:42.429 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.429 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:03:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:42 smithi089 ceph-mon[34499]: pgmap v575: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:03:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:42 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:03:42.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:42 smithi059 ceph-mon[30025]: pgmap v575: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:03:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:42 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:03:42.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:42 smithi189 ceph-mon[34772]: pgmap v575: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:03:42.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:42 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:03:43.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:03:43.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:03:43.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:03:43.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:03:43.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:03:43.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:03:43.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:03:43.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:03:43.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:03:43.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:03:43.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:03:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:03:44.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:03:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:44 smithi089 ceph-mon[34499]: pgmap v576: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:03:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:44 smithi059 ceph-mon[30025]: pgmap v576: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:03:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:44 smithi189 ceph-mon[34772]: pgmap v576: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:03:45.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.531+0000 7ff697400640 1 -- 172.21.15.59:0/4122613355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690104010 msgr2=0x7ff6901043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.531+0000 7ff697400640 1 --2- 172.21.15.59:0/4122613355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690104010 0x7ff6901043f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff684009900 tx=0x7ff68402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 -- 172.21.15.59:0/4122613355 shutdown_connections 2024-09-05T12:03:45.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 --2- 172.21.15.59:0/4122613355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 0x7ff690110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 --2- 172.21.15.59:0/4122613355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6901049c0 0x7ff690108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 --2- 172.21.15.59:0/4122613355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690104010 0x7ff6901043f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 -- 172.21.15.59:0/4122613355 >> 172.21.15.59:0/4122613355 conn(0x7ff6900ffa50 msgr2=0x7ff690101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 -- 172.21.15.59:0/4122613355 shutdown_connections 2024-09-05T12:03:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.532+0000 7ff697400640 1 -- 172.21.15.59:0/4122613355 wait complete. 2024-09-05T12:03:45.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.533+0000 7ff697400640 1 Processor -- start 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.533+0000 7ff697400640 1 -- start start 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 0x7ff69019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 0x7ff69019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 0x7ff69019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901141b0 con 0x7ff6901049c0 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff690114030 con 0x7ff690109140 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff697400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff690114330 con 0x7ff690104010 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff694a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 0x7ff69019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 0x7ff69019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff694a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 0x7ff69019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34896/0 (socket says 172.21.15.59:34896) 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff694a00640 1 -- 172.21.15.59:0/714025015 learned_addr learned my addr 172.21.15.59:0/714025015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:45.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.534+0000 7ff695400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 0x7ff69019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 -- 172.21.15.59:0/714025015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 msgr2=0x7ff69019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 0x7ff69019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 -- 172.21.15.59:0/714025015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 msgr2=0x7ff69019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 0x7ff69019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 -- 172.21.15.59:0/714025015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69019f670 con 0x7ff6901049c0 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68fe00640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 0x7ff69019ac00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9f0 tx=0x7ff67c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68d600640 1 -- 172.21.15.59:0/714025015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00ccc0 con 0x7ff6901049c0 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68d600640 1 -- 172.21.15.59:0/714025015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff67c005020 con 0x7ff6901049c0 2024-09-05T12:03:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69019f960 con 0x7ff6901049c0 2024-09-05T12:03:45.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff68d600640 1 -- 172.21.15.59:0/714025015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0049e0 con 0x7ff6901049c0 2024-09-05T12:03:45.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.535+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6901a7240 con 0x7ff6901049c0 2024-09-05T12:03:45.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff68d600640 1 -- 172.21.15.59:0/714025015 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff67c005560 con 0x7ff6901049c0 2024-09-05T12:03:45.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 Processor -- start 2024-09-05T12:03:45.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 -- start start 2024-09-05T12:03:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799accf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799ad7d70 0x561799ada160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561799a333e0 con 0x561799ad7d70 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5617999dad80 con 0x561799acd4b0 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.539+0000 7ff699dbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561799a2f3a0 con 0x561799accb90 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56172/0 (socket says 172.21.15.59:56172) 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 -- 172.21.15.59:0/4043407556 learned_addr learned my addr 172.21.15.59:0/4043407556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799ad7d70 0x561799ada160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff695400640 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799accf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 -- 172.21.15.59:0/4043407556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 msgr2=0x561799accf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799accf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 -- 172.21.15.59:0/4043407556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799ad7d70 msgr2=0x561799ada160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799ad7d70 0x561799ada160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.540+0000 7ff694a00640 1 -- 172.21.15.59:0/4043407556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617999f9ae0 con 0x561799acd4b0 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff694a00640 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff684005e80 tx=0x7ff684005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05783bc21b7fa01 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff695400640 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799accf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:45.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff68d600640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 0x7ff6780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff681c00640 1 -- 172.21.15.59:0/4043407556 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684035040 con 0x561799acd4b0 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff681c00640 1 -- 172.21.15.59:0/4043407556 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6840044b0 con 0x561799acd4b0 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff681c00640 1 -- 172.21.15.59:0/4043407556 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684003850 con 0x561799acd4b0 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff68d600640 1 -- 172.21.15.59:0/714025015 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff67c0cf430 con 0x7ff6901049c0 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.541+0000 7ff695400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 0x7ff6780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 -- 172.21.15.59:0/4043407556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 msgr2=0x561799ad7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff684005e80 tx=0x7ff684005c00 comp rx=0 tx=0).stop 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 -- 172.21.15.59:0/4043407556 shutdown_connections 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799ad7d70 0x561799ada160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799ad7830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 --2- 172.21.15.59:0/4043407556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799accf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 -- 172.21.15.59:0/4043407556 >> 172.21.15.59:0/4043407556 conn(0x561799ac9120 msgr2=0x561799ac9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 -- 172.21.15.59:0/4043407556 shutdown_connections 2024-09-05T12:03:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff695400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 0x7ff6780a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688009600 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.542+0000 7ff699dbc740 1 -- 172.21.15.59:0/4043407556 wait complete. 2024-09-05T12:03:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.543+0000 7ff699dbc740 1 -- start start 2024-09-05T12:03:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.543+0000 7ff699dbc740 1 Processor -- start 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ad50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799ad5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561799ada980 con 0x561799acd4b0 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5617999ee0f0 con 0x561799accb90 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561799adab00 con 0x561799ad7d70 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff699dbc740 10 client.0 _ll_register_callbacks cb 0x5617999df9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ad50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56182/0 (socket says 172.21.15.59:56182) 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ad50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59682/0 (socket says 172.21.15.59:59682) 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff694a00640 1 -- 172.21.15.59:0/2254746153 learned_addr learned my addr 172.21.15.59:0/2254746153 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.544+0000 7ff695400640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799ad5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 -- 172.21.15.59:0/2254746153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 msgr2=0x561799ad5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799ad5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 -- 172.21.15.59:0/2254746153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 msgr2=0x561799ad50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ad50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 -- 172.21.15.59:0/2254746153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617997a43c0 con 0x561799accb90 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff694a00640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff684031550 tx=0x7ff6840049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2102e14404599dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff676000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684053060 con 0x561799accb90 2024-09-05T12:03:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6840044b0 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.545+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff6840042b0 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff68405a050 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 10 client.26288 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561799ae65b0 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6840044b0 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff684003b80 con 0x561799accb90 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 10 client.26288 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 20 client.26288 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799ae7900 0x561799ae9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 4 client.26288 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.546+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561799a40950 con 0x561799ae7900 2024-09-05T12:03:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.547+0000 7ff68fe00640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799ae7900 0x561799ae9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.547+0000 7ff68fe00640 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799ae7900 0x561799ae9cf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537992ded74dedaf server_cookie=d0afc5cf450bc308 in_seq=0 out_seq=0 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.547+0000 7ff676000640 10 client.26288 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff676000640 1 -- 172.21.15.59:0/2254746153 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561799a40950 con 0x561799ae7900 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff676000640 10 client.26288 handle_command_reply: tid=0 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 client.26288 shutdown 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 msgr2=0x561799ad4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff684031550 tx=0x7ff6840049b0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 shutdown_connections 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799ad5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799ae7900 0x561799ae9cf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ad50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2254746153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799accb90 0x561799ad4ba0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 >> 172.21.15.59:0/2254746153 conn(0x561799ac9120 msgr2=0x561799acee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.548+0000 7ff676000640 0 client.26288 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.549+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 shutdown_connections 2024-09-05T12:03:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.549+0000 7ff699dbc740 1 -- 172.21.15.59:0/2254746153 wait complete. 2024-09-05T12:03:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.549+0000 7ff699dbc740 20 client.26288 trim_cache size 0 max 16384 2024-09-05T12:03:45.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 Processor -- start 2024-09-05T12:03:45.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 -- start start 2024-09-05T12:03:45.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799aef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799aef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901aefc0 con 0x561799accb90 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901b1840 con 0x561799acd4b0 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.567+0000 7ff699dbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901af630 con 0x561799ad7d70 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59696/0 (socket says 172.21.15.59:59696) 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 -- 172.21.15.59:0/3923672047 learned_addr learned my addr 172.21.15.59:0/3923672047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff68fe00640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799aef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff695400640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799aef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 -- 172.21.15.59:0/3923672047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 msgr2=0x561799aef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799aef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 -- 172.21.15.59:0/3923672047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 msgr2=0x561799aef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799aef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff694a00640 1 -- 172.21.15.59:0/3923672047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561799ac89b0 con 0x561799accb90 2024-09-05T12:03:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.568+0000 7ff68fe00640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799aef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.569+0000 7ff694a00640 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff684004980 tx=0x7ff684005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b42aa9dfe948949 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.569+0000 7ff681200640 1 -- 172.21.15.59:0/3923672047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684035050 con 0x561799accb90 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.569+0000 7ff681200640 1 -- 172.21.15.59:0/3923672047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff684031680 con 0x561799accb90 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.569+0000 7ff681200640 1 -- 172.21.15.59:0/3923672047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684005490 con 0x561799accb90 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- 172.21.15.59:0/3923672047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 msgr2=0x561799ad3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff684004980 tx=0x7ff684005c00 comp rx=0 tx=0).stop 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- 172.21.15.59:0/3923672047 shutdown_connections 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799ad7d70 0x561799aef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799acd4b0 0x561799aef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 --2- 172.21.15.59:0/3923672047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799accb90 0x561799ad3e20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- 172.21.15.59:0/3923672047 >> 172.21.15.59:0/3923672047 conn(0x561799ac9120 msgr2=0x561799acee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- 172.21.15.59:0/3923672047 shutdown_connections 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- 172.21.15.59:0/3923672047 wait complete. 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 -- start start 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.570+0000 7ff699dbc740 1 Processor -- start 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799ae0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 0x561799ad14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901afd00 con 0x561799acd4b0 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901b1170 con 0x561799ad7d70 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6901b03d0 con 0x561799accb90 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff699dbc740 10 client.0 _ll_register_callbacks cb 0x7ff684005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff694a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799ae0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59698/0 (socket says 172.21.15.59:59698) 2024-09-05T12:03:45.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff68fe00640 1 -- 172.21.15.59:0/2087017598 learned_addr learned my addr 172.21.15.59:0/2087017598 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.571+0000 7ff695400640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 0x561799ad14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 -- 172.21.15.59:0/2087017598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 msgr2=0x561799ae0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799ae0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 -- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 msgr2=0x561799ad14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 0x561799ad14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 -- 172.21.15.59:0/2087017598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561799ad1cd0 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff695400640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 0x561799ad14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff68fe00640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0ccac0 tx=0x7ff67c0d2e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25100630e9cd44d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff676000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c01c040 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff67c0cf0b0 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.572+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff684031680 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff67c0051d0 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 10 client.16290 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561799b51e70 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff684031680 con 0x561799acd4b0 2024-09-05T12:03:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff67c097e90 con 0x561799acd4b0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 10 client.16290 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 20 client.16290 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799b525a0 0x561799b549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 4 client.16290 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.573+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561799a3ae60 con 0x561799b525a0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.574+0000 7ff694a00640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799b525a0 0x561799b549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.574+0000 7ff694a00640 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799b525a0 0x561799b549c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0f4fdb5c4c2010 server_cookie=3680409a6222775d in_seq=0 out_seq=0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.574+0000 7ff676000640 10 client.16290 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.574+0000 7ff676000640 1 -- 172.21.15.59:0/2087017598 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561799a3ae60 con 0x561799b525a0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.574+0000 7ff676000640 10 client.16290 handle_command_reply: tid=0 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 client.16290 shutdown 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 msgr2=0x561799ae0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0ccac0 tx=0x7ff67c0d2e70 comp rx=0 tx=0).stop 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 shutdown_connections 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561799ad7d70 0x561799ad14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561799b525a0 0x561799b549c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561799acd4b0 0x561799ae0580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 --2- 172.21.15.59:0/2087017598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561799accb90 0x561799ae0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff676000640 0 client.16290 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 >> 172.21.15.59:0/2087017598 conn(0x561799ac9120 msgr2=0x561799ac79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 shutdown_connections 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 1 -- 172.21.15.59:0/2087017598 wait complete. 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.575+0000 7ff699dbc740 20 client.16290 trim_cache size 0 max 16384 2024-09-05T12:03:45.580 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:45.581 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:03:45.581 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:03:45.581 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:45.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 msgr2=0x7ff6780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 0x7ff6780a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688009600 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 msgr2=0x7ff69019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 0x7ff69019ac00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9f0 tx=0x7ff67c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 shutdown_connections 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff6780a2b80 0x7ff6780a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff690109140 0x7ff69019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6901049c0 0x7ff69019ac00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 --2- 172.21.15.59:0/714025015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff690104010 0x7ff69019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 >> 172.21.15.59:0/714025015 conn(0x7ff6900ffa50 msgr2=0x7ff690100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 shutdown_connections 2024-09-05T12:03:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:45.576+0000 7ff697400640 1 -- 172.21.15.59:0/714025015 wait complete. 2024-09-05T12:03:46.448 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:03:46.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:03:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:46 smithi089 ceph-mon[34499]: pgmap v577: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:46 smithi059 ceph-mon[30025]: pgmap v577: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:46 smithi189 ceph-mon[34772]: pgmap v577: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:47.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 -- 172.21.15.59:0/2501288647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8103e30 msgr2=0x7f19c8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 --2- 172.21.15.59:0/2501288647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8103e30 0x7f19c8104210 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009900 tx=0x7f19b802a410 comp rx=0 tx=0).stop 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 -- 172.21.15.59:0/2501288647 shutdown_connections 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 --2- 172.21.15.59:0/2501288647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c8108f60 0x7f19c8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 --2- 172.21.15.59:0/2501288647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c81047e0 0x7f19c8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 --2- 172.21.15.59:0/2501288647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8103e30 0x7f19c8104210 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 -- 172.21.15.59:0/2501288647 >> 172.21.15.59:0/2501288647 conn(0x7f19c80ff910 msgr2=0x7f19c8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 -- 172.21.15.59:0/2501288647 shutdown_connections 2024-09-05T12:03:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.405+0000 7f19c7e00640 1 -- 172.21.15.59:0/2501288647 wait complete. 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.406+0000 7f19c7e00640 1 Processor -- start 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.406+0000 7f19c7e00640 1 -- start start 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c8103e30 0x7f19c819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 0x7f19c819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8113fc0 con 0x7f19c8108f60 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8113e40 con 0x7f19c8103e30 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8114140 con 0x7f19c81047e0 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 0x7f19c819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34956/0 (socket says 172.21.15.59:34956) 2024-09-05T12:03:47.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 -- 172.21.15.59:0/3666946844 learned_addr learned my addr 172.21.15.59:0/3666946844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:47.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c6a00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c8103e30 0x7f19c819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 -- 172.21.15.59:0/3666946844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c8103e30 msgr2=0x7f19c819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c8103e30 0x7f19c819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 -- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 msgr2=0x7f19c819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 0x7f19c819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19bfe00640 1 -- 172.21.15.59:0/3666946844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c819f470 con 0x7f19c81047e0 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.407+0000 7f19c7400640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 0x7f19c819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19bfe00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19b000e9f0 tx=0x7f19b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b000cd80 con 0x7f19c81047e0 2024-09-05T12:03:47.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c819f760 con 0x7f19c81047e0 2024-09-05T12:03:47.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0010910 con 0x7f19c81047e0 2024-09-05T12:03:47.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00188f0 con 0x7f19c81047e0 2024-09-05T12:03:47.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.408+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a7040 con 0x7f19c81047e0 2024-09-05T12:03:47.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.409+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c8105310 con 0x7f19c81047e0 2024-09-05T12:03:47.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.412+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19b0010ac0 con 0x7f19c81047e0 2024-09-05T12:03:47.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.413+0000 7f19bf400640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 0x7f19ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:47.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.413+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f19b00cfba0 con 0x7f19c81047e0 2024-09-05T12:03:47.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.413+0000 7f19c6a00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 0x7f19ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:47.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.414+0000 7f19c6a00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 0x7f19ac0a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009230 tx=0x7f19b80058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:47.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.415+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19b009c040 con 0x7f19c81047e0 2024-09-05T12:03:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.640+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19c8104210 con 0x7f19c81047e0 2024-09-05T12:03:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.642+0000 7f19bf400640 1 -- 172.21.15.59:0/3666946844 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f19c8104210 con 0x7f19c81047e0 2024-09-05T12:03:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:03:47.644 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:03:47.644 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:03:47.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 msgr2=0x7f19ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:47.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 0x7f19ac0a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009230 tx=0x7f19b80058d0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 msgr2=0x7f19c819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19b000e9f0 tx=0x7f19b000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 shutdown_connections 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f19ac0a2c00 0x7f19ac0a50c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19c8108f60 0x7f19c819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81047e0 0x7f19c819aa00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 --2- 172.21.15.59:0/3666946844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19c8103e30 0x7f19c819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 >> 172.21.15.59:0/3666946844 conn(0x7f19c80ff910 msgr2=0x7f19c8100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 shutdown_connections 2024-09-05T12:03:47.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:47.645+0000 7f19c7e00640 1 -- 172.21.15.59:0/3666946844 wait complete. 2024-09-05T12:03:47.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:47 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3666946844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:47.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3666946844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:48.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:47 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3666946844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:03:48.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:03:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:48.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:48 smithi059 ceph-mon[30025]: pgmap v578: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:48 smithi089 ceph-mon[34499]: pgmap v578: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:48 smithi189 ceph-mon[34772]: pgmap v578: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 -- 172.21.15.59:0/234442978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 msgr2=0x7ff88810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 --2- 172.21.15.59:0/234442978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff88810e890 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff880009cd0 tx=0x7ff88002a8e0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 -- 172.21.15.59:0/234442978 shutdown_connections 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 --2- 172.21.15.59:0/234442978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff88810e890 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 --2- 172.21.15.59:0/234442978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881019c0 0x7ff888101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 --2- 172.21.15.59:0/234442978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881079c0 0x7ff888107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 -- 172.21.15.59:0/234442978 >> 172.21.15.59:0/234442978 conn(0x7ff8880fd740 msgr2=0x7ff8880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 -- 172.21.15.59:0/234442978 shutdown_connections 2024-09-05T12:03:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.352+0000 7ff88ee00640 1 -- 172.21.15.59:0/234442978 wait complete. 2024-09-05T12:03:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.353+0000 7ff88ee00640 1 Processor -- start 2024-09-05T12:03:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.353+0000 7ff88ee00640 1 -- start start 2024-09-05T12:03:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 0x7ff888198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881079c0 0x7ff88819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111dc0 con 0x7ff888102360 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111c40 con 0x7ff8881019c0 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111f40 con 0x7ff8881079c0 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 0x7ff888198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59708/0 (socket says 172.21.15.59:59708) 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88d000640 1 -- 172.21.15.59:0/2946347279 learned_addr learned my addr 172.21.15.59:0/2946347279 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.354+0000 7ff88e400640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881079c0 0x7ff88819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 -- 172.21.15.59:0/2946347279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881079c0 msgr2=0x7ff88819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881079c0 0x7ff88819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 -- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 msgr2=0x7ff8881987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 -- 172.21.15.59:0/2946347279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819d210 con 0x7ff8881019c0 2024-09-05T12:03:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88d000640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88da00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 0x7ff888198260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005fd0 tx=0x7ff87c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff886000640 1 -- 172.21.15.59:0/2946347279 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c009c30 con 0x7ff8881019c0 2024-09-05T12:03:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819d500 con 0x7ff8881019c0 2024-09-05T12:03:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff886000640 1 -- 172.21.15.59:0/2946347279 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c009dd0 con 0x7ff8881019c0 2024-09-05T12:03:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff886000640 1 -- 172.21.15.59:0/2946347279 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c010460 con 0x7ff8881019c0 2024-09-05T12:03:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.355+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a4de0 con 0x7ff8881019c0 2024-09-05T12:03:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.358+0000 7ff893053740 1 Processor -- start 2024-09-05T12:03:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.358+0000 7ff886000640 1 -- 172.21.15.59:0/2946347279 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff87c003f50 con 0x7ff8881019c0 2024-09-05T12:03:49.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 -- start start 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673527140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 0x559673535ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55967348d3e0 con 0x559673526d60 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559673434d80 con 0x559673536430 2024-09-05T12:03:49.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff893053740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5596734893a0 con 0x559673527680 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff88d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff88d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56234/0 (socket says 172.21.15.59:56234) 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff88d000640 1 -- 172.21.15.59:0/4161050097 learned_addr learned my addr 172.21.15.59:0/4161050097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff88e400640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673527140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.359+0000 7ff88da00640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 0x559673535ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 -- 172.21.15.59:0/4161050097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 msgr2=0x559673535ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 0x559673535ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 -- 172.21.15.59:0/4161050097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 msgr2=0x559673527140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673527140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 -- 172.21.15.59:0/4161050097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559673453ae0 con 0x559673536430 2024-09-05T12:03:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88da00640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 0x559673535ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff88d000640 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff8740115f0 tx=0x7ff874011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6357de752e3ddaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff886000640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 0x7ff8700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.360+0000 7ff87aa00640 1 -- 172.21.15.59:0/4161050097 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87401e070 con 0x559673536430 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff87aa00640 1 -- 172.21.15.59:0/4161050097 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87400fba0 con 0x559673536430 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff87aa00640 1 -- 172.21.15.59:0/4161050097 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874019cd0 con 0x559673536430 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff886000640 1 -- 172.21.15.59:0/2946347279 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff87c0cf2f0 con 0x7ff8881019c0 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff88e400640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 0x7ff8700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 -- 172.21.15.59:0/4161050097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 msgr2=0x559673538820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff8740115f0 tx=0x7ff874011ab0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 -- 172.21.15.59:0/4161050097 shutdown_connections 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673536430 0x559673538820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673527680 0x559673535ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 --2- 172.21.15.59:0/4161050097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673527140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 -- 172.21.15.59:0/4161050097 >> 172.21.15.59:0/4161050097 conn(0x5596735233d0 msgr2=0x5596735237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff893053740 1 -- 172.21.15.59:0/4161050097 shutdown_connections 2024-09-05T12:03:49.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.361+0000 7ff88e400640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 0x7ff8700a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff880004ab0 tx=0x7ff880004a40 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.362+0000 7ff893053740 1 -- 172.21.15.59:0/4161050097 wait complete. 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.362+0000 7ff893053740 1 -- start start 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.362+0000 7ff893053740 1 Processor -- start 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x5596735338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x559673533de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559673539060 con 0x559673526d60 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5596734480f0 con 0x559673527680 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5596735391e0 con 0x559673536430 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff893053740 10 client.0 _ll_register_callbacks cb 0x55967342fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x559673533de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59724/0 (socket says 172.21.15.59:59724) 2024-09-05T12:03:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 -- 172.21.15.59:0/313737485 learned_addr learned my addr 172.21.15.59:0/313737485 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:49.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88d000640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x5596735338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 -- 172.21.15.59:0/313737485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 msgr2=0x559673533de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x559673533de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 -- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 msgr2=0x5596735338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x5596735338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.363+0000 7ff88da00640 1 -- 172.21.15.59:0/313737485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596731fe3c0 con 0x559673526d60 2024-09-05T12:03:49.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff88da00640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff87c097bf0 tx=0x7ff87c09b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8def626287ff82 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff86f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c020070 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff87c0035f0 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87400fba0 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff87c09fa90 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff893053740 10 client.16308 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559673545130 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.364+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87400fba0 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff87c0d2c00 con 0x559673526d60 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 10 client.16308 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 20 client.16308 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:49.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735465f0 0x559673548a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 4 client.16308 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55967349a950 con 0x5596735465f0 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.365+0000 7ff88d000640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735465f0 0x559673548a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.366+0000 7ff88d000640 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735465f0 0x559673548a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea51af138836d56 server_cookie=86762335522ca1c9 in_seq=0 out_seq=0 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.366+0000 7ff86f400640 10 client.16308 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff86f400640 1 -- 172.21.15.59:0/313737485 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55967349a950 con 0x5596735465f0 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff86f400640 10 client.16308 handle_command_reply: tid=0 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 client.16308 shutdown 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 msgr2=0x559673533360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff87c097bf0 tx=0x7ff87c09b620 comp rx=0 tx=0).stop 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 shutdown_connections 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x559673533de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x5596735338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735465f0 0x559673548a00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 --2- 172.21.15.59:0/313737485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673526d60 0x559673533360 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 >> 172.21.15.59:0/313737485 conn(0x5596735233d0 msgr2=0x559673529090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff86f400640 0 client.16308 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 shutdown_connections 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 1 -- 172.21.15.59:0/313737485 wait complete. 2024-09-05T12:03:49.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.367+0000 7ff893053740 20 client.16308 trim_cache size 0 max 16384 2024-09-05T12:03:49.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 Processor -- start 2024-09-05T12:03:49.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 -- start start 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673527680 0x55967354e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x55967354e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881aab90 con 0x559673527680 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ad410 con 0x559673526d60 2024-09-05T12:03:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff893053740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ab200 con 0x559673536430 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff88da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff88da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56246/0 (socket says 172.21.15.59:56246) 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff88da00640 1 -- 172.21.15.59:0/3926501003 learned_addr learned my addr 172.21.15.59:0/3926501003 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff88d000640 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673527680 0x55967354e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.382+0000 7ff88e400640 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x55967354e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 -- 172.21.15.59:0/3926501003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 msgr2=0x55967354e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x55967354e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 -- 172.21.15.59:0/3926501003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673527680 msgr2=0x55967354e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673527680 0x55967354e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 -- 172.21.15.59:0/3926501003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559673527470 con 0x559673526d60 2024-09-05T12:03:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff88da00640 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0a3e20 tx=0x7ff87c005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f9883128d57dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff87a000640 1 -- 172.21.15.59:0/3926501003 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c020070 con 0x559673526d60 2024-09-05T12:03:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff87a000640 1 -- 172.21.15.59:0/3926501003 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c0a4350 con 0x559673526d60 2024-09-05T12:03:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.383+0000 7ff87a000640 1 -- 172.21.15.59:0/3926501003 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0a3ec0 con 0x559673526d60 2024-09-05T12:03:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- 172.21.15.59:0/3926501003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 msgr2=0x559673528f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0a3e20 tx=0x7ff87c005ab0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- 172.21.15.59:0/3926501003 shutdown_connections 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673536430 0x55967354e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673527680 0x55967354e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 --2- 172.21.15.59:0/3926501003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673526d60 0x559673528f40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- 172.21.15.59:0/3926501003 >> 172.21.15.59:0/3926501003 conn(0x5596735233d0 msgr2=0x559673528830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- 172.21.15.59:0/3926501003 shutdown_connections 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- 172.21.15.59:0/3926501003 wait complete. 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 -- start start 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.384+0000 7ff893053740 1 Processor -- start 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x559673530a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 0x559673530fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ab8d0 con 0x559673536430 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881acd40 con 0x559673527680 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881abfa0 con 0x559673526d60 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff893053740 10 client.0 _ll_register_callbacks cb 0x7ff87c097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff88da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff88da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35004/0 (socket says 172.21.15.59:35004) 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff88da00640 1 -- 172.21.15.59:0/293896657 learned_addr learned my addr 172.21.15.59:0/293896657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff88e400640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 0x559673530fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.385+0000 7ff88d000640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x559673530a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 -- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 msgr2=0x559673530a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x559673530a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 -- 172.21.15.59:0/293896657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 msgr2=0x559673530fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 0x559673530fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 -- 172.21.15.59:0/293896657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596735b04a0 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88e400640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 0x559673530fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff88da00640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004650 tx=0x7ff87c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcd2af1864b7042 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff86f400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c020050 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff87c09abe0 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87c0a4350 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff87c09ae20 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff893053740 10 client.26251 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.386+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5596735b04a0 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c0a4350 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff87c0a0c20 con 0x559673526d60 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 10 client.26251 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 10 client.26251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 20 client.26251 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735b1a20 0x5596735b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 4 client.26251 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:03:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559673494e60 con 0x5596735b1a20 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.387+0000 7ff88d000640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735b1a20 0x5596735b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff88d000640 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735b1a20 0x5596735b3e10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cc9a555e1eb84e server_cookie=9ccaddc3a221d308 in_seq=0 out_seq=0 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff86f400640 10 client.26251 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff86f400640 1 -- 172.21.15.59:0/293896657 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559673494e60 con 0x5596735b1a20 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff86f400640 10 client.26251 handle_command_reply: tid=0 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 client.26251 shutdown 2024-09-05T12:03:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 msgr2=0x559673530530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004650 tx=0x7ff87c0035f0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 shutdown_connections 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596735b1a20 0x5596735b3e10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559673536430 0x559673530fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559673527680 0x559673530a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 --2- 172.21.15.59:0/293896657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559673526d60 0x559673530530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff86f400640 0 client.26251 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 >> 172.21.15.59:0/293896657 conn(0x5596735233d0 msgr2=0x559673529050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 shutdown_connections 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.388+0000 7ff893053740 1 -- 172.21.15.59:0/293896657 wait complete. 2024-09-05T12:03:49.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff893053740 20 client.26251 trim_cache size 0 max 16384 2024-09-05T12:03:49.395 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:03:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 msgr2=0x7ff8700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 0x7ff8700a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff880004ab0 tx=0x7ff880004a40 comp rx=0 tx=0).stop 2024-09-05T12:03:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 msgr2=0x7ff888198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 0x7ff888198260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005fd0 tx=0x7ff87c00dde0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 shutdown_connections 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b80 0x7ff8700a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881079c0 0x7ff88819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff888102360 0x7ff8881987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 --2- 172.21.15.59:0/2946347279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8881019c0 0x7ff888198260 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 >> 172.21.15.59:0/2946347279 conn(0x7ff8880fd740 msgr2=0x7ff8880fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.389+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 shutdown_connections 2024-09-05T12:03:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:49.390+0000 7ff88ee00640 1 -- 172.21.15.59:0/2946347279 wait complete. 2024-09-05T12:03:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:50 smithi089 ceph-mon[34499]: pgmap v579: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:03:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:50 smithi059 ceph-mon[30025]: pgmap v579: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:03:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:50 smithi189 ceph-mon[34772]: pgmap v579: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:03:51.265 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:03:51.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:03:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 -- 172.21.15.59:0/3956259015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 msgr2=0x7fca50107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 --2- 172.21.15.59:0/3956259015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca50107da0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fca40009930 tx=0x7fca4002a420 comp rx=0 tx=0).stop 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 -- 172.21.15.59:0/3956259015 shutdown_connections 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 --2- 172.21.15.59:0/3956259015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50102360 0x7fca5010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 --2- 172.21.15.59:0/3956259015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca50101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 --2- 172.21.15.59:0/3956259015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca50107da0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 -- 172.21.15.59:0/3956259015 >> 172.21.15.59:0/3956259015 conn(0x7fca500fd740 msgr2=0x7fca500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 -- 172.21.15.59:0/3956259015 shutdown_connections 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.241+0000 7fca54a00640 1 -- 172.21.15.59:0/3956259015 wait complete. 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 Processor -- start 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 -- start start 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50102360 0x7fca5019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca5019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111dc0 con 0x7fca50102360 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111c40 con 0x7fca501019c0 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.242+0000 7fca54a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111f40 con 0x7fca501079c0 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36000/0 (socket says 172.21.15.59:36000) 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50102360 0x7fca5019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 -- 172.21.15.59:0/3588954603 learned_addr learned my addr 172.21.15.59:0/3588954603 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4f200640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca5019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 -- 172.21.15.59:0/3588954603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 msgr2=0x7fca5019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca5019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 -- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50102360 msgr2=0x7fca5019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50102360 0x7fca5019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4e800640 1 -- 172.21.15.59:0/3588954603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5019f430 con 0x7fca501019c0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.243+0000 7fca4f200640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca5019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca4e800640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca40009230 tx=0x7fca40004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca47e00640 1 -- 172.21.15.59:0/3588954603 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4003f030 con 0x7fca501019c0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca47e00640 1 -- 172.21.15.59:0/3588954603 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca40004440 con 0x7fca501019c0 2024-09-05T12:03:52.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5019f6c0 con 0x7fca501019c0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca47e00640 1 -- 172.21.15.59:0/3588954603 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035070 con 0x7fca501019c0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.244+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a7000 con 0x7fca501019c0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.248+0000 7fca57e29740 1 Processor -- start 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.248+0000 7fca57e29740 1 -- start start 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b634090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 0x55e68b63e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e68b59a290 con 0x55e68b63ee90 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e68b541b40 con 0x55e68b633cb0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca57e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e68b596250 con 0x55e68b6345d0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca47e00640 1 -- 172.21.15.59:0/3588954603 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca40042030 con 0x7fca501019c0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51664/0 (socket says 172.21.15.59:51664) 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca45e00640 1 -- 172.21.15.59:0/875399087 learned_addr learned my addr 172.21.15.59:0/875399087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca4f200640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b634090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.249+0000 7fca4e800640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 0x55e68b63e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 -- 172.21.15.59:0/875399087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 msgr2=0x55e68b63e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 0x55e68b63e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 -- 172.21.15.59:0/875399087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 msgr2=0x55e68b634090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b634090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 -- 172.21.15.59:0/875399087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68b30b6a0 con 0x55e68b63ee90 2024-09-05T12:03:52.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca4e800640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 0x55e68b63e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:52.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca45e00640 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fca38009c90 tx=0x7fca38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7b68b50243a16f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca3f400640 1 -- 172.21.15.59:0/875399087 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3801e070 con 0x55e68b63ee90 2024-09-05T12:03:52.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca3f400640 1 -- 172.21.15.59:0/875399087 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca38019b10 con 0x55e68b63ee90 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.250+0000 7fca3f400640 1 -- 172.21.15.59:0/875399087 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800ea70 con 0x55e68b63ee90 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca47e00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca47e00640 1 -- 172.21.15.59:0/3588954603 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca400ef690 con 0x7fca501019c0 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca4f200640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 -- 172.21.15.59:0/875399087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 msgr2=0x55e68b641280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fca38009c90 tx=0x7fca38011810 comp rx=0 tx=0).stop 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 -- 172.21.15.59:0/875399087 shutdown_connections 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b63ee90 0x55e68b641280 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b6345d0 0x55e68b63e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 --2- 172.21.15.59:0/875399087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b634090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 -- 172.21.15.59:0/875399087 >> 172.21.15.59:0/875399087 conn(0x55e68b630240 msgr2=0x55e68b630620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.251+0000 7fca57e29740 1 -- 172.21.15.59:0/875399087 shutdown_connections 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.252+0000 7fca57e29740 1 -- 172.21.15.59:0/875399087 wait complete. 2024-09-05T12:03:52.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.252+0000 7fca4f200640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.252+0000 7fca57e29740 1 -- start start 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.252+0000 7fca57e29740 1 Processor -- start 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b63e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b63ee90 0x55e68b63bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e68b641b00 con 0x55e68b6345d0 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e68b554fa0 con 0x55e68b633cb0 2024-09-05T12:03:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e68b641c80 con 0x55e68b63ee90 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca57e29740 10 client.0 _ll_register_callbacks cb 0x55e68b2a8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51672/0 (socket says 172.21.15.59:51672) 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca45e00640 1 -- 172.21.15.59:0/3662705272 learned_addr learned my addr 172.21.15.59:0/3662705272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca4e800640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b63e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.253+0000 7fca4f200640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b63ee90 0x55e68b63bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 -- 172.21.15.59:0/3662705272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b63ee90 msgr2=0x55e68b63bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b63ee90 0x55e68b63bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 -- 172.21.15.59:0/3662705272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 msgr2=0x55e68b63ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 -- 172.21.15.59:0/3662705272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68b63c790 con 0x55e68b633cb0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca45e00640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca4e800640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b63e7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca400ecdd0 tx=0x7fca400bb6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5a173595ec7c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca33e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40049070 con 0x55e68b633cb0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca38019b10 con 0x55e68b633cb0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca400ef340 con 0x55e68b633cb0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fca400bfe90 con 0x55e68b633cb0 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca57e29740 10 client.26330 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:52.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.254+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e68b64da10 con 0x55e68b633cb0 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca38019b10 con 0x55e68b633cb0 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fca400d4ce0 con 0x55e68b633cb0 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 10 client.26330 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 20 client.26330 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b64ee70 0x55e68b6512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 4 client.26330 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.255+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e68b5a7800 con 0x55e68b64ee70 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.256+0000 7fca45e00640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b64ee70 0x55e68b6512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.256+0000 7fca45e00640 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b64ee70 0x55e68b6512c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb19604151c2883 server_cookie=50dac34f5132d28 in_seq=0 out_seq=0 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.256+0000 7fca33e00640 10 client.26330 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca33e00640 1 -- 172.21.15.59:0/3662705272 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e68b5a7800 con 0x55e68b64ee70 2024-09-05T12:03:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca33e00640 10 client.26330 handle_command_reply: tid=0 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 client.26330 shutdown 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 msgr2=0x55e68b63e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b63e7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca400ecdd0 tx=0x7fca400bb6d0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 shutdown_connections 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b63ee90 0x55e68b63bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b64ee70 0x55e68b6512c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b6345d0 0x55e68b63ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 --2- 172.21.15.59:0/3662705272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b633cb0 0x55e68b63e7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 >> 172.21.15.59:0/3662705272 conn(0x55e68b630240 msgr2=0x55e68b635f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca33e00640 0 client.26330 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 shutdown_connections 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.257+0000 7fca57e29740 1 -- 172.21.15.59:0/3662705272 wait complete. 2024-09-05T12:03:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.258+0000 7fca57e29740 20 client.26330 trim_cache size 0 max 16384 2024-09-05T12:03:52.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.275+0000 7fca57e29740 1 Processor -- start 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 -- start start 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b63d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 0x55e68b62f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca501aee30 con 0x55e68b630240 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca501b16b0 con 0x55e68b651420 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca57e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca501af4a0 con 0x55e68b633cb0 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca4f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca4e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b63d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 0x55e68b62f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca4f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44240/0 (socket says 172.21.15.59:44240) 2024-09-05T12:03:52.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.276+0000 7fca4f200640 1 -- 172.21.15.59:0/140277202 learned_addr learned my addr 172.21.15.59:0/140277202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 -- 172.21.15.59:0/140277202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 msgr2=0x55e68b63d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b63d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 -- 172.21.15.59:0/140277202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 msgr2=0x55e68b62f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 0x55e68b62f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 -- 172.21.15.59:0/140277202 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68b639fa0 con 0x55e68b633cb0 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca45e00640 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 0x55e68b62f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:52.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca4f200640 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca48003d00 tx=0x7fca4801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3609e042accafa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca3ea00640 1 -- 172.21.15.59:0/140277202 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48003d30 con 0x55e68b633cb0 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca3ea00640 1 -- 172.21.15.59:0/140277202 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca480045b0 con 0x55e68b633cb0 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.277+0000 7fca3ea00640 1 -- 172.21.15.59:0/140277202 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48005740 con 0x55e68b633cb0 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- 172.21.15.59:0/140277202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 msgr2=0x55e68b646f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca48003d00 tx=0x7fca4801b000 comp rx=0 tx=0).stop 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- 172.21.15.59:0/140277202 shutdown_connections 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b633cb0 0x55e68b646f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b630240 0x55e68b62f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 --2- 172.21.15.59:0/140277202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b63d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- 172.21.15.59:0/140277202 >> 172.21.15.59:0/140277202 conn(0x55e68b63c0f0 msgr2=0x55e68b62ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- 172.21.15.59:0/140277202 shutdown_connections 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- 172.21.15.59:0/140277202 wait complete. 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.278+0000 7fca57e29740 1 -- start start 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 Processor -- start 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b633cb0 0x55e68b645d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b646ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca501afb70 con 0x55e68b633cb0 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca501b0fe0 con 0x55e68b651420 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca501b0240 con 0x55e68b630240 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.279+0000 7fca57e29740 10 client.0 _ll_register_callbacks cb 0x7fca40009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b633cb0 0x55e68b645d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44250/0 (socket says 172.21.15.59:44250) 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 -- 172.21.15.59:0/1317223229 learned_addr learned my addr 172.21.15.59:0/1317223229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4f200640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b646ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 -- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 msgr2=0x55e68b646ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b646ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 -- 172.21.15.59:0/1317223229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b633cb0 msgr2=0x55e68b645d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b633cb0 0x55e68b645d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 -- 172.21.15.59:0/1317223229 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68b647380 con 0x55e68b630240 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca4e800640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca400c42c0 tx=0x7fca400bb800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad58b48a8010bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:52.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.280+0000 7fca33e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40049070 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca400bafe0 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca480045b0 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fca400bb4f0 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 10 client.26269 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e68b6ba750 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca480045b0 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fca400d4ce0 con 0x55e68b630240 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 10 client.26269 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 20 client.26269 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b6345d0 0x55e68b6bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 4 client.26269 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:03:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.281+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e68b5a1d10 con 0x55e68b6345d0 2024-09-05T12:03:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.282+0000 7fca45e00640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b6345d0 0x55e68b6bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.282+0000 7fca45e00640 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b6345d0 0x55e68b6bccf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9b0c247a0ba5e7 server_cookie=e65aecd99d7f81cb in_seq=0 out_seq=0 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.282+0000 7fca33e00640 10 client.26269 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca33e00640 1 -- 172.21.15.59:0/1317223229 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e68b5a1d10 con 0x55e68b6345d0 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca33e00640 10 client.26269 handle_command_reply: tid=0 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 client.26269 shutdown 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 msgr2=0x55e68b6457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca400c42c0 tx=0x7fca400bb800 comp rx=0 tx=0).stop 2024-09-05T12:03:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 shutdown_connections 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e68b651420 0x55e68b646ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.283+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e68b6345d0 0x55e68b6bccf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e68b633cb0 0x55e68b645d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 1 --2- 172.21.15.59:0/1317223229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e68b630240 0x55e68b6457c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 >> 172.21.15.59:0/1317223229 conn(0x55e68b63c0f0 msgr2=0x55e68b62f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca33e00640 0 client.26269 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 shutdown_connections 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 1 -- 172.21.15.59:0/1317223229 wait complete. 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca57e29740 20 client.26269 trim_cache size 0 max 16384 2024-09-05T12:03:52.290 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "dc9113bb-ae34-479d-908c-a50aa4a83ccc", 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 msgr2=0x7fca340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48009500 comp rx=0 tx=0).stop 2024-09-05T12:03:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 msgr2=0x7fca5019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.284+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca40009230 tx=0x7fca40004040 comp rx=0 tx=0).stop 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 shutdown_connections 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca5019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca50102360 0x7fca5019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 --2- 172.21.15.59:0/3588954603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca501019c0 0x7fca5019a480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 >> 172.21.15.59:0/3588954603 conn(0x7fca500fd740 msgr2=0x7fca500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 shutdown_connections 2024-09-05T12:03:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:52.285+0000 7fca54a00640 1 -- 172.21.15.59:0/3588954603 wait complete. 2024-09-05T12:03:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:52 smithi089 ceph-mon[34499]: pgmap v580: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:52 smithi059 ceph-mon[30025]: pgmap v580: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:53.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:03:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:52 smithi189 ceph-mon[34772]: pgmap v580: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:03:53.413 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:03:53.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:53 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:03:53.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:53 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:53.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:53 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:03:53.843 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:53 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 -- 172.21.15.59:0/722983905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958103e30 msgr2=0x7fa958104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 --2- 172.21.15.59:0/722983905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958103e30 0x7fa958104210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa948009900 tx=0x7fa94802a410 comp rx=0 tx=0).stop 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 -- 172.21.15.59:0/722983905 shutdown_connections 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 --2- 172.21.15.59:0/722983905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa958108f60 0x7fa958110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 --2- 172.21.15.59:0/722983905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9581047e0 0x7fa958108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 --2- 172.21.15.59:0/722983905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958103e30 0x7fa958104210 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 -- 172.21.15.59:0/722983905 >> 172.21.15.59:0/722983905 conn(0x7fa9580ff910 msgr2=0x7fa958101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:53.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 -- 172.21.15.59:0/722983905 shutdown_connections 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.956+0000 7fa957200640 1 -- 172.21.15.59:0/722983905 wait complete. 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.957+0000 7fa957200640 1 Processor -- start 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.957+0000 7fa957200640 1 -- start start 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9581047e0 0x7fa95819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa958113fd0 con 0x7fa958108f60 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa958113e50 con 0x7fa9581047e0 2024-09-05T12:03:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa957200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa958114150 con 0x7fa958103e30 2024-09-05T12:03:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa956800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:44268/0 (socket says 172.21.15.59:44268) 2024-09-05T12:03:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa956800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51706/0 (socket says 172.21.15.59:51706) 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 -- 172.21.15.59:0/4082095372 learned_addr learned my addr 172.21.15.59:0/4082095372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa94fe00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9581047e0 0x7fa95819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 -- 172.21.15.59:0/4082095372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9581047e0 msgr2=0x7fa95819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9581047e0 0x7fa95819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 -- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 msgr2=0x7fa95819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.958+0000 7fa955e00640 1 -- 172.21.15.59:0/4082095372 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95819f420 con 0x7fa958103e30 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa956800640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa955e00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa948004770 tx=0x7fa948003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa94ea00640 1 -- 172.21.15.59:0/4082095372 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94803f070 con 0x7fa958103e30 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa94ea00640 1 -- 172.21.15.59:0/4082095372 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948031ca0 con 0x7fa958103e30 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95819f6b0 con 0x7fa958103e30 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa94ea00640 1 -- 172.21.15.59:0/4082095372 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948035030 con 0x7fa958103e30 2024-09-05T12:03:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.959+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581a6ff0 con 0x7fa958103e30 2024-09-05T12:03:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.962+0000 7fa95f612740 1 Processor -- start 2024-09-05T12:03:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.962+0000 7fa94ea00640 1 -- 172.21.15.59:0/4082095372 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa948042070 con 0x7fa958103e30 2024-09-05T12:03:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 -- start start 2024-09-05T12:03:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff19977130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff199819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19984320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff198dd3e0 con 0x55ff19977670 2024-09-05T12:03:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff19884d80 con 0x55ff19981f30 2024-09-05T12:03:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa95f612740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff198d93a0 con 0x55ff19976d50 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa955e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff199819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa955e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff199819f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51712/0 (socket says 172.21.15.59:51712) 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa955e00640 1 -- 172.21.15.59:0/723235764 learned_addr learned my addr 172.21.15.59:0/723235764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa94fe00640 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19984320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.963+0000 7fa956800640 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff19977130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 -- 172.21.15.59:0/723235764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 msgr2=0x55ff19984320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19984320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 -- 172.21.15.59:0/723235764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 msgr2=0x55ff199819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff199819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 -- 172.21.15.59:0/723235764 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff198a3ae0 con 0x55ff19976d50 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa956800640 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff19977130 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa950005fd0 tx=0x7fa950011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea7e41b6644c2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa94ea00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 0x7fa93c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa94ea00640 1 -- 172.21.15.59:0/4082095372 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa9480ef760 con 0x7fa958103e30 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.964+0000 7fa947000640 1 -- 172.21.15.59:0/723235764 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950004d20 con 0x55ff19976d50 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa947000640 1 -- 172.21.15.59:0/723235764 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95000f9a0 con 0x55ff19976d50 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa947000640 1 -- 172.21.15.59:0/723235764 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950004430 con 0x55ff19976d50 2024-09-05T12:03:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa955e00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 0x7fa93c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 -- 172.21.15.59:0/723235764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 msgr2=0x55ff19977130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff19977130 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa950005fd0 tx=0x7fa950011bf0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa955e00640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 0x7fa93c0a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa9480f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 -- 172.21.15.59:0/723235764 shutdown_connections 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19984320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff199819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 --2- 172.21.15.59:0/723235764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff19977130 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.965+0000 7fa95f612740 1 -- 172.21.15.59:0/723235764 >> 172.21.15.59:0/723235764 conn(0x55ff199732e0 msgr2=0x55ff199736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.966+0000 7fa95f612740 1 -- 172.21.15.59:0/723235764 shutdown_connections 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.966+0000 7fa95f612740 1 -- 172.21.15.59:0/723235764 wait complete. 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.966+0000 7fa95f612740 1 -- start start 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.966+0000 7fa95f612740 1 Processor -- start 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.966+0000 7fa95f612740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19977670 0x55ff1997f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff1997f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff19984ba0 con 0x55ff19976d50 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff198980f0 con 0x55ff19981f30 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff19984d20 con 0x55ff19977670 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa95f612740 10 client.0 _ll_register_callbacks cb 0x55ff198899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51716/0 (socket says 172.21.15.59:51716) 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 -- 172.21.15.59:0/1984570804 learned_addr learned my addr 172.21.15.59:0/1984570804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19977670 0x55ff1997f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 -- 172.21.15.59:0/1984570804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19977670 msgr2=0x55ff1997f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19977670 0x55ff1997f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 -- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 msgr2=0x55ff1997f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff1997f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.967+0000 7fa956800640 1 -- 172.21.15.59:0/1984570804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1964e3c0 con 0x55ff19976d50 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa956800640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa950003cb0 tx=0x7fa950003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fdfe85c0ac8815 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa93b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950005000 con 0x55ff19976d50 2024-09-05T12:03:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa9500051a0 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95000f9a0 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa950032050 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa95f612740 10 client.16344 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.968+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff19990850 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95000f9a0 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa950018070 con 0x55ff19976d50 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 10 client.16344 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 20 client.16344 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff19991c50 0x55ff19994040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 4 client.16344 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff198ea950 con 0x55ff19991c50 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.969+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff19991c50 0x55ff19994040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.970+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff19991c50 0x55ff19994040 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23715e49c2058566 server_cookie=3bb05bbd7fed3dcd in_seq=0 out_seq=0 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.970+0000 7fa93b400640 10 client.16344 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa93b400640 1 -- 172.21.15.59:0/1984570804 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff198ea950 con 0x55ff19991c50 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa93b400640 10 client.16344 handle_command_reply: tid=0 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 client.16344 shutdown 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 msgr2=0x55ff1997ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa950003cb0 tx=0x7fa950003ce0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 shutdown_connections 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff1997f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19977670 0x55ff1997f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff19991c50 0x55ff19994040 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 --2- 172.21.15.59:0/1984570804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19976d50 0x55ff1997ee20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa93b400640 0 client.16344 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 >> 172.21.15.59:0/1984570804 conn(0x55ff199732e0 msgr2=0x55ff19979010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 shutdown_connections 2024-09-05T12:03:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.971+0000 7fa95f612740 1 -- 172.21.15.59:0/1984570804 wait complete. 2024-09-05T12:03:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.972+0000 7fa95f612740 20 client.16344 trim_cache size 0 max 16384 2024-09-05T12:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 Processor -- start 2024-09-05T12:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 -- start start 2024-09-05T12:03:53.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff199997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19971b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581aef70 con 0x55ff19977670 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581b17f0 con 0x55ff19981f30 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa95f612740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581af5e0 con 0x55ff19976d50 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.990+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51730/0 (socket says 172.21.15.59:51730) 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 -- 172.21.15.59:0/2938305622 learned_addr learned my addr 172.21.15.59:0/2938305622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa956800640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19971b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa956800640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff199997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 -- 172.21.15.59:0/2938305622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 msgr2=0x55ff199997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff199997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 -- 172.21.15.59:0/2938305622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 msgr2=0x55ff19971b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19971b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 -- 172.21.15.59:0/2938305622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1997d370 con 0x55ff19977670 2024-09-05T12:03:53.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa956800640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff199997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:53.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa94fe00640 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ff1997e6b0 tx=0x7fa94000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5ecdb24aad41af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa946600640 1 -- 172.21.15.59:0/2938305622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940003d60 con 0x55ff19977670 2024-09-05T12:03:53.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.991+0000 7fa946600640 1 -- 172.21.15.59:0/2938305622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa940011920 con 0x55ff19977670 2024-09-05T12:03:53.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa946600640 1 -- 172.21.15.59:0/2938305622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400269d0 con 0x55ff19977670 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 -- 172.21.15.59:0/2938305622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 msgr2=0x55ff19999d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ff1997e6b0 tx=0x7fa94000b390 comp rx=0 tx=0).stop 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 -- 172.21.15.59:0/2938305622 shutdown_connections 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19981f30 0x55ff19971b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19999d10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 --2- 172.21.15.59:0/2938305622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19976d50 0x55ff199997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 -- 172.21.15.59:0/2938305622 >> 172.21.15.59:0/2938305622 conn(0x55ff199732e0 msgr2=0x55ff1997df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 -- 172.21.15.59:0/2938305622 shutdown_connections 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.992+0000 7fa95f612740 1 -- 172.21.15.59:0/2938305622 wait complete. 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.993+0000 7fa95f612740 1 -- start start 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.993+0000 7fa95f612740 1 Processor -- start 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.993+0000 7fa95f612740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19976d50 0x55ff1998a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.993+0000 7fa95f612740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa95f612740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 0x55ff19985f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa95f612740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581afcb0 con 0x55ff19977670 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa95f612740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581b1120 con 0x55ff19976d50 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa95f612740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581b0380 con 0x55ff19981f30 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa95f612740 10 client.0 _ll_register_callbacks cb 0x7fa950011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51746/0 (socket says 172.21.15.59:51746) 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 -- 172.21.15.59:0/1381793315 learned_addr learned my addr 172.21.15.59:0/1381793315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa956800640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19976d50 0x55ff1998a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa956800640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 0x55ff19985f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 -- 172.21.15.59:0/1381793315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 msgr2=0x55ff19985f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 0x55ff19985f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 -- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19976d50 msgr2=0x55ff1998a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19976d50 0x55ff1998a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa94fe00640 1 -- 172.21.15.59:0/1381793315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff19986750 con 0x55ff19977670 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.994+0000 7fa956800640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 0x55ff19985f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b8c0 tx=0x7fa9400264a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c9fe27b576b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa93b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940011640 con 0x55ff19977670 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa940011ad0 con 0x55ff19977670 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa940011920 con 0x55ff19977670 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa940039050 con 0x55ff19977670 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa95f612740 10 client.16350 fetch_fsmap learned FSMap version 21 2024-09-05T12:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff199fc780 con 0x55ff19977670 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.995+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa940011920 con 0x55ff19977670 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa940030450 con 0x55ff19977670 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 10 client.16350 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 20 client.16350 populate_metadata read hostname 'smithi059' 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff199fce10 0x55ff199ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:03:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 4 client.16350 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff198e4e60 con 0x55ff199fce10 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.996+0000 7fa956800640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff199fce10 0x55ff199ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa956800640 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff199fce10 0x55ff199ff230 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f412e56c38e3b73 server_cookie=8692e06c0f4eba8 in_seq=0 out_seq=0 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa93b400640 10 client.16350 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa93b400640 1 -- 172.21.15.59:0/1381793315 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ff198e4e60 con 0x55ff199fce10 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa93b400640 10 client.16350 handle_command_reply: tid=0 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 client.16350 shutdown 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 msgr2=0x55ff19985a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b8c0 tx=0x7fa9400264a0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 shutdown_connections 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff19981f30 0x55ff19985f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff199fce10 0x55ff199ff230 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff19977670 0x55ff19985a10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 --2- 172.21.15.59:0/1381793315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff19976d50 0x55ff1998a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 >> 172.21.15.59:0/1381793315 conn(0x55ff199732e0 msgr2=0x55ff19971b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa93b400640 0 client.16350 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 shutdown_connections 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.997+0000 7fa95f612740 1 -- 172.21.15.59:0/1381793315 wait complete. 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa95f612740 20 client.16350 trim_cache size 0 max 16384 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout: "dc9113bb-ae34-479d-908c-a50aa4a83ccc": { 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:03:54.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "dc9113bb-ae34-479d-908c-a50aa4a83ccc", 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 msgr2=0x7fa93c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 0x7fa93c0a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa9480f5530 comp rx=0 tx=0).stop 2024-09-05T12:03:54.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 msgr2=0x7fa95819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa948004770 tx=0x7fa948003c10 comp rx=0 tx=0).stop 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 shutdown_connections 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.998+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa93c0a2c00 0x7fa93c0a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa958108f60 0x7fa95819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9581047e0 0x7fa95819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 --2- 172.21.15.59:0/4082095372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa958103e30 0x7fa95819a470 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 >> 172.21.15.59:0/4082095372 conn(0x7fa9580ff910 msgr2=0x7fa958100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 shutdown_connections 2024-09-05T12:03:54.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:03:53.999+0000 7fa957200640 1 -- 172.21.15.59:0/4082095372 wait complete. 2024-09-05T12:03:54.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:53 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:03:54.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:53 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:53 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:03:54.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:53 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:53 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:03:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:53 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:53 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:03:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:53 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:03:54.894 INFO:tasks.cephfs.filesystem:scrub status for tag:dc9113bb-ae34-479d-908c-a50aa4a83ccc - {'path': '/', 'tag': 'dc9113bb-ae34-479d-908c-a50aa4a83ccc', 'options': 'recursive,force'} 2024-09-05T12:03:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:54 smithi059 ceph-mon[30025]: pgmap v581: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:54 smithi189 ceph-mon[34772]: pgmap v581: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:54 smithi089 ceph-mon[34499]: pgmap v581: 129 pgs: 129 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:03:55.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:55 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:03:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:55 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:03:56.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:55 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:03:56.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:56 smithi059 ceph-mon[30025]: pgmap v582: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:56.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:56 smithi189 ceph-mon[34772]: pgmap v582: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:56 smithi089 ceph-mon[34499]: pgmap v582: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:03:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:03:58 smithi059 ceph-mon[30025]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:03:59.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:03:58 smithi189 ceph-mon[34772]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:03:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:03:58 smithi089 ceph-mon[34499]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:00 smithi059 ceph-mon[30025]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:00 smithi189 ceph-mon[34772]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:00 smithi089 ceph-mon[34499]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:02 smithi059 ceph-mon[30025]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:02.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:02 smithi089 ceph-mon[34499]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:02 smithi189 ceph-mon[34772]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:04 smithi189 ceph-mon[34772]: pgmap v586: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:04 smithi089 ceph-mon[34499]: pgmap v586: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:04 smithi059 ceph-mon[30025]: pgmap v586: 129 pgs: 129 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:06 smithi189 ceph-mon[34772]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:06 smithi089 ceph-mon[34499]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:06 smithi059 ceph-mon[30025]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:08 smithi189 ceph-mon[34772]: pgmap v588: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:08 smithi089 ceph-mon[34499]: pgmap v588: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:08 smithi059 ceph-mon[30025]: pgmap v588: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:10 smithi189 ceph-mon[34772]: pgmap v589: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:10 smithi089 ceph-mon[34499]: pgmap v589: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:10 smithi059 ceph-mon[30025]: pgmap v589: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:12 smithi189 ceph-mon[34772]: pgmap v590: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:04:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:12 smithi089 ceph-mon[34499]: pgmap v590: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:12 smithi059 ceph-mon[30025]: pgmap v590: 129 pgs: 129 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:04:15.010 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:14 smithi059 ceph-mon[30025]: pgmap v591: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:14 smithi189 ceph-mon[34772]: pgmap v591: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:14 smithi089 ceph-mon[34499]: pgmap v591: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:16.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:16 smithi059 ceph-mon[30025]: pgmap v592: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:16 smithi189 ceph-mon[34772]: pgmap v592: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:16 smithi089 ceph-mon[34499]: pgmap v592: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:19.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:18 smithi189 ceph-mon[34772]: pgmap v593: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:18 smithi089 ceph-mon[34499]: pgmap v593: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:19.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:18 smithi059 ceph-mon[30025]: pgmap v593: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:20 smithi189 ceph-mon[34772]: pgmap v594: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:20 smithi089 ceph-mon[34499]: pgmap v594: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:21.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:20 smithi059 ceph-mon[30025]: pgmap v594: 129 pgs: 129 active+clean; 3.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:22 smithi189 ceph-mon[34772]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:22 smithi089 ceph-mon[34499]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:23.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:22 smithi059 ceph-mon[30025]: pgmap v595: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:24.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:04:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:24 smithi189 ceph-mon[34772]: pgmap v596: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:24 smithi089 ceph-mon[34499]: pgmap v596: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:25.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:24 smithi059 ceph-mon[30025]: pgmap v596: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 -- 172.21.15.59:0/3755865253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc1049a0 msgr2=0x7f2dfc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 --2- 172.21.15.59:0/3755865253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc1089f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009860 tx=0x7f2de802a340 comp rx=0 tx=0).stop 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 -- 172.21.15.59:0/3755865253 shutdown_connections 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 --2- 172.21.15.59:0/3755865253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc109120 0x7f2dfc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 --2- 172.21.15.59:0/3755865253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc1089f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 --2- 172.21.15.59:0/3755865253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 -- 172.21.15.59:0/3755865253 >> 172.21.15.59:0/3755865253 conn(0x7f2dfc0ffa90 msgr2=0x7f2dfc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 -- 172.21.15.59:0/3755865253 shutdown_connections 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.876+0000 7f2e00e00640 1 -- 172.21.15.59:0/3755865253 wait complete. 2024-09-05T12:04:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 Processor -- start 2024-09-05T12:04:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 -- start start 2024-09-05T12:04:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc109120 0x7f2dfc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc114120 con 0x7f2dfc1049a0 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc113fa0 con 0x7f2dfc103ff0 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.878+0000 7f2e00e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1142a0 con 0x7f2dfc109120 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58682/0 (socket says 172.21.15.59:58682) 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc109120 0x7f2dfc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 -- 172.21.15.59:0/437651498 learned_addr learned my addr 172.21.15.59:0/437651498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 -- 172.21.15.59:0/437651498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc109120 msgr2=0x7f2dfc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc109120 0x7f2dfc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 -- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 msgr2=0x7f2dfc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 -- 172.21.15.59:0/437651498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc19f650 con 0x7f2dfc103ff0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9400640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.879+0000 7f2df9e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00e9c0 tx=0x7f2dec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.880+0000 7f2df2a00640 1 -- 172.21.15.59:0/437651498 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00ccc0 con 0x7f2dfc103ff0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.880+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc19f940 con 0x7f2dfc103ff0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.880+0000 7f2df2a00640 1 -- 172.21.15.59:0/437651498 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec005500 con 0x7f2dfc103ff0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.880+0000 7f2df2a00640 1 -- 172.21.15.59:0/437651498 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec010430 con 0x7f2dfc103ff0 2024-09-05T12:04:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.880+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1a7220 con 0x7f2dfc103ff0 2024-09-05T12:04:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 Processor -- start 2024-09-05T12:04:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2df2a00640 1 -- 172.21.15.59:0/437651498 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2dec0046c0 con 0x7f2dfc103ff0 2024-09-05T12:04:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 -- start start 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648b5a223e0 con 0x5648b5abc4b0 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5648b59c9d80 con 0x5648b5ac6d70 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.883+0000 7f2e03a22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5648b5a1e3a0 con 0x5648b5abbb90 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38464/0 (socket says 172.21.15.59:38464) 2024-09-05T12:04:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2dfa800640 1 -- 172.21.15.59:0/2582891338 learned_addr learned my addr 172.21.15.59:0/2582891338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46498/0 (socket says 172.21.15.59:46498) 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 -- 172.21.15.59:0/2582891338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 msgr2=0x5648b5abbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 -- 172.21.15.59:0/2582891338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 msgr2=0x5648b5ac6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 -- 172.21.15.59:0/2582891338 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b59e8ae0 con 0x5648b5ac6d70 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2dfa800640 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.884+0000 7f2df9400640 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac9160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de8009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2ce73af350d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2df2a00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2de7400640 1 -- 172.21.15.59:0/2582891338 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8041020 con 0x5648b5ac6d70 2024-09-05T12:04:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2df2a00640 1 -- 172.21.15.59:0/437651498 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2dec0cf7a0 con 0x7f2dfc103ff0 2024-09-05T12:04:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2de7400640 1 -- 172.21.15.59:0/2582891338 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8004240 con 0x5648b5ac6d70 2024-09-05T12:04:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2de7400640 1 -- 172.21.15.59:0/2582891338 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037040 con 0x5648b5ac6d70 2024-09-05T12:04:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.885+0000 7f2dfa800640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- 172.21.15.59:0/2582891338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 msgr2=0x5648b5ac9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac9160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de8009510 comp rx=0 tx=0).stop 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- 172.21.15.59:0/2582891338 shutdown_connections 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac9160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2582891338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- 172.21.15.59:0/2582891338 >> 172.21.15.59:0/2582891338 conn(0x5648b5ab8120 msgr2=0x5648b5ab8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2dfa800640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 0x7f2de00a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fd0 tx=0x7f2df4009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- 172.21.15.59:0/2582891338 shutdown_connections 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- 172.21.15.59:0/2582891338 wait complete. 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.886+0000 7f2e03a22740 1 -- start start 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 Processor -- start 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5ac3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648b5ac9980 con 0x5648b5abc4b0 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5648b59dd0f0 con 0x5648b5ac6d70 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5648b5ac9b00 con 0x5648b5abbb90 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.887+0000 7f2e03a22740 10 client.0 _ll_register_callbacks cb 0x5648b59ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5ac3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5ac3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38466/0 (socket says 172.21.15.59:38466) 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9e00640 1 -- 172.21.15.59:0/2385035155 learned_addr learned my addr 172.21.15.59:0/2385035155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2dfa800640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 -- 172.21.15.59:0/2385035155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 msgr2=0x5648b5ac3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5ac3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 -- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 msgr2=0x5648b5ac46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 -- 172.21.15.59:0/2385035155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b57933c0 con 0x5648b5abc4b0 2024-09-05T12:04:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2dfa800640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2df9400640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac4180 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f491e507da631951 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.888+0000 7f2ddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de804c070 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2de8003e30 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de8004240 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2de8058050 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2e03a22740 10 client.16362 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5648b5ad5610 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.889+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8004240 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2de803dad0 con 0x5648b5abc4b0 2024-09-05T12:04:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 10 client.16362 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 20 client.16362 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5ad6960 0x5648b5ad8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 4 client.16362 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b5a2f950 con 0x5648b5ad6960 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.890+0000 7f2df9e00640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5ad6960 0x5648b5ad8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.891+0000 7f2df9e00640 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5ad6960 0x5648b5ad8d50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fba803f7b94664 server_cookie=d970ac2902ae2e61 in_seq=0 out_seq=0 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.891+0000 7f2ddbe00640 10 client.16362 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2ddbe00640 1 -- 172.21.15.59:0/2385035155 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648b5a2f950 con 0x5648b5ad6960 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2ddbe00640 10 client.16362 handle_command_reply: tid=0 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 client.16362 shutdown 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 msgr2=0x5648b5ac4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac4180 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035040 tx=0x7f2de8004040 comp rx=0 tx=0).stop 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 shutdown_connections 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5ac6d70 0x5648b5ac46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5ad6960 0x5648b5ad8d50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5abc4b0 0x5648b5ac4180 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2ddbe00640 0 client.16362 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 --2- 172.21.15.59:0/2385035155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5ac3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 >> 172.21.15.59:0/2385035155 conn(0x5648b5ab8120 msgr2=0x5648b5abde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 shutdown_connections 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 1 -- 172.21.15.59:0/2385035155 wait complete. 2024-09-05T12:04:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.892+0000 7f2e03a22740 20 client.16362 trim_cache size 0 max 16384 2024-09-05T12:04:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.910+0000 7f2e03a22740 1 Processor -- start 2024-09-05T12:04:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.910+0000 7f2e03a22740 1 -- start start 2024-09-05T12:04:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abc4b0 0x5648b5ac0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ac13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1af110 con 0x5648b5ac6d70 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b1990 con 0x5648b5abc4b0 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2e03a22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1af780 con 0x5648b5abbb90 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ac13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ac13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38468/0 (socket says 172.21.15.59:38468) 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2dfa800640 1 -- 172.21.15.59:0/3067889706 learned_addr learned my addr 172.21.15.59:0/3067889706 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9400640 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abc4b0 0x5648b5ac0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 -- 172.21.15.59:0/3067889706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abc4b0 msgr2=0x5648b5ac0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abc4b0 0x5648b5ac0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 -- 172.21.15.59:0/3067889706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 msgr2=0x5648b5ac13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ac13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.911+0000 7f2df9e00640 1 -- 172.21.15.59:0/3067889706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b5abc2e0 con 0x5648b5abbb90 2024-09-05T12:04:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.912+0000 7f2df9e00640 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2dec098560 tx=0x7f2dec0d5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a716f75e0d3b53c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.912+0000 7f2de6a00640 1 -- 172.21.15.59:0/3067889706 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec09b6d0 con 0x5648b5abbb90 2024-09-05T12:04:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.912+0000 7f2de6a00640 1 -- 172.21.15.59:0/3067889706 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec09c290 con 0x5648b5abbb90 2024-09-05T12:04:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.912+0000 7f2de6a00640 1 -- 172.21.15.59:0/3067889706 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0056b0 con 0x5648b5abbb90 2024-09-05T12:04:25.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- 172.21.15.59:0/3067889706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 msgr2=0x5648b5abdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2dec098560 tx=0x7f2dec0d5530 comp rx=0 tx=0).stop 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- 172.21.15.59:0/3067889706 shutdown_connections 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ac13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abc4b0 0x5648b5ac0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 --2- 172.21.15.59:0/3067889706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abbb90 0x5648b5abdcf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- 172.21.15.59:0/3067889706 >> 172.21.15.59:0/3067889706 conn(0x5648b5ab8120 msgr2=0x5648b5abd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- 172.21.15.59:0/3067889706 shutdown_connections 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- 172.21.15.59:0/3067889706 wait complete. 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 -- start start 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.913+0000 7f2e03a22740 1 Processor -- start 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abc4b0 0x5648b5ada1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ada6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1afe50 con 0x5648b5ac6d70 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b12c0 con 0x5648b5abbb90 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b0520 con 0x5648b5abc4b0 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2e03a22740 10 client.0 _ll_register_callbacks cb 0x7f2df40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2dfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ada6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58728/0 (socket says 172.21.15.59:58728) 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9e00640 1 -- 172.21.15.59:0/1061157131 learned_addr learned my addr 172.21.15.59:0/1061157131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9400640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abc4b0 0x5648b5ada1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9e00640 1 -- 172.21.15.59:0/1061157131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abc4b0 msgr2=0x5648b5ada1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.914+0000 7f2df9e00640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abc4b0 0x5648b5ada1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2df9e00640 1 -- 172.21.15.59:0/1061157131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 msgr2=0x5648b5ada6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2df9e00640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ada6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2df9e00640 1 -- 172.21.15.59:0/1061157131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b5b40440 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2df9e00640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00c6a0 tx=0x7f2dec09b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf944f8ec597fe0a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2ddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec09b4f0 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec09c290 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2dec098200 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2dec09b6d0 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2e03a22740 10 client.26369 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.915+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5648b5b416f0 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec09c290 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2dec0ce030 con 0x5648b5abbb90 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 10 client.26369 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 10 client.26369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 20 client.26369 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5b41d00 0x5648b5b440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 4 client.26369 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:04:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648b5a29e60 con 0x5648b5b41d00 2024-09-05T12:04:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.916+0000 7f2df9400640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5b41d00 0x5648b5b440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2df9400640 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5b41d00 0x5648b5b440f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a065d8047b12b96 server_cookie=bd5f60d20c2083ca in_seq=0 out_seq=0 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2ddbe00640 10 client.26369 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2ddbe00640 1 -- 172.21.15.59:0/1061157131 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648b5a29e60 con 0x5648b5b41d00 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2ddbe00640 10 client.26369 handle_command_reply: tid=0 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 client.26369 shutdown 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 msgr2=0x5648b5ad9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00c6a0 tx=0x7f2dec09b960 comp rx=0 tx=0).stop 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 shutdown_connections 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5648b5b41d00 0x5648b5b440f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648b5ac6d70 0x5648b5ada6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5648b5abc4b0 0x5648b5ada1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 --2- 172.21.15.59:0/1061157131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5648b5abbb90 0x5648b5ad9c70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2ddbe00640 0 client.26369 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 >> 172.21.15.59:0/1061157131 conn(0x5648b5ab8120 msgr2=0x5648b5ad1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 shutdown_connections 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 1 -- 172.21.15.59:0/1061157131 wait complete. 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.917+0000 7f2e03a22740 20 client.26369 trim_cache size 0 max 16384 2024-09-05T12:04:25.923 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 msgr2=0x7f2de00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 0x7f2de00a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fd0 tx=0x7f2df4009500 comp rx=0 tx=0).stop 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 msgr2=0x7f2dfc19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00e9c0 tx=0x7f2dec00ee80 comp rx=0 tx=0).stop 2024-09-05T12:04:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 shutdown_connections 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2dfc109120 0x7f2dfc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dfc1049a0 0x7f2dfc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 --2- 172.21.15.59:0/437651498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2dfc103ff0 0x7f2dfc19a6a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 >> 172.21.15.59:0/437651498 conn(0x7f2dfc0ffa90 msgr2=0x7f2dfc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 shutdown_connections 2024-09-05T12:04:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:25.918+0000 7f2e00e00640 1 -- 172.21.15.59:0/437651498 wait complete. 2024-09-05T12:04:26.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:04:26.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:04:27.014 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:26 smithi059 ceph-mon[30025]: pgmap v597: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:26 smithi189 ceph-mon[34772]: pgmap v597: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:26 smithi089 ceph-mon[34499]: pgmap v597: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:04:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 -- 172.21.15.59:0/4217385327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684112870 msgr2=0x7f7684114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 --2- 172.21.15.59:0/4217385327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684112870 0x7f7684114c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f767c009ca0 tx=0x7f767c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 -- 172.21.15.59:0/4217385327 shutdown_connections 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 --2- 172.21.15.59:0/4217385327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684112870 0x7f7684114c60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 --2- 172.21.15.59:0/4217385327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684077040 0x7f7684112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 --2- 172.21.15.59:0/4217385327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f7684076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 -- 172.21.15.59:0/4217385327 >> 172.21.15.59:0/4217385327 conn(0x7f76840ff8d0 msgr2=0x7f7684101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 -- 172.21.15.59:0/4217385327 shutdown_connections 2024-09-05T12:04:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.002+0000 7f7683e00640 1 -- 172.21.15.59:0/4217385327 wait complete. 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 Processor -- start 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 -- start start 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f76841a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 0x7f76841a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684112870 0x7f768419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76841183c0 con 0x7f7684077040 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7684118240 con 0x7f7684112870 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.003+0000 7f7683e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7684118540 con 0x7f7684076720 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f767a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 0x7f76841a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f767a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 0x7f76841a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46520/0 (socket says 172.21.15.59:46520) 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f767a000640 1 -- 172.21.15.59:0/3733948812 learned_addr learned my addr 172.21.15.59:0/3733948812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f76841a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7683400640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684112870 0x7f768419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 -- 172.21.15.59:0/3733948812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684112870 msgr2=0x7f768419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684112870 0x7f768419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 -- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 msgr2=0x7f76841a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 0x7f76841a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 -- 172.21.15.59:0/3733948812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768419ef00 con 0x7f7684076720 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.004+0000 7f7682a00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f76841a45c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f767400e9f0 tx=0x7f767400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.005+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767400ccc0 con 0x7f7684076720 2024-09-05T12:04:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.005+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7674005020 con 0x7f7684076720 2024-09-05T12:04:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.005+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768419f1f0 con 0x7f7684076720 2024-09-05T12:04:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.005+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740049e0 con 0x7f7684076720 2024-09-05T12:04:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.005+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841ab540 con 0x7f7684076720 2024-09-05T12:04:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.006+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684077e60 con 0x7f7684076720 2024-09-05T12:04:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.008+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7674005560 con 0x7f7684076720 2024-09-05T12:04:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.009+0000 7f7688a00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 0x7f76680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.009+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f76740cf3a0 con 0x7f7684076720 2024-09-05T12:04:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.010+0000 7f767a000640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 0x7f76680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.010+0000 7f767a000640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 0x7f76680a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f768419fee0 tx=0x7f766c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.012+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76740051d0 con 0x7f7684076720 2024-09-05T12:04:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.234+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76841ab710 con 0x7f7684076720 2024-09-05T12:04:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.235+0000 7f7688a00640 1 -- 172.21.15.59:0/3733948812 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f767409be90 con 0x7f7684076720 2024-09-05T12:04:28.237 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:04:28.237 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:04:28.237 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.237+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 msgr2=0x7f76680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.237+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 0x7f76680a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f768419fee0 tx=0x7f766c005f90 comp rx=0 tx=0).stop 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.237+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 msgr2=0x7f76841a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.237+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f76841a45c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f767400e9f0 tx=0x7f767400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.237+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 shutdown_connections 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f76680a2c00 0x7f76680a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684112870 0x7f768419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7684077040 0x7f76841a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 --2- 172.21.15.59:0/3733948812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684076720 0x7f76841a45c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:28.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 >> 172.21.15.59:0/3733948812 conn(0x7f76840ff8d0 msgr2=0x7f7684101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:28.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 shutdown_connections 2024-09-05T12:04:28.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:28.238+0000 7f7683e00640 1 -- 172.21.15.59:0/3733948812 wait complete. 2024-09-05T12:04:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:28 smithi059 ceph-mon[30025]: pgmap v598: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:28 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3733948812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:04:29.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:04:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:28 smithi189 ceph-mon[34772]: pgmap v598: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:28 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3733948812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:04:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:28 smithi089 ceph-mon[34499]: pgmap v598: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:28 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3733948812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:04:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 -- 172.21.15.59:0/3773424240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16901049c0 msgr2=0x7f1690108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 --2- 172.21.15.59:0/3773424240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16901049c0 0x7f1690108a10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 -- 172.21.15.59:0/3773424240 shutdown_connections 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 --2- 172.21.15.59:0/3773424240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1690109140 0x7f1690110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 --2- 172.21.15.59:0/3773424240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16901049c0 0x7f1690108a10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 --2- 172.21.15.59:0/3773424240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f16901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 -- 172.21.15.59:0/3773424240 >> 172.21.15.59:0/3773424240 conn(0x7f16900ffa50 msgr2=0x7f1690101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 -- 172.21.15.59:0/3773424240 shutdown_connections 2024-09-05T12:04:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.068+0000 7f1695600640 1 -- 172.21.15.59:0/3773424240 wait complete. 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.069+0000 7f1695600640 1 Processor -- start 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.069+0000 7f1695600640 1 -- start start 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.069+0000 7f1695600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f169019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.069+0000 7f1695600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16901049c0 0x7f169019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f1695600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 0x7f169019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f1695600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114190 con 0x7f1690109140 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f1695600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114010 con 0x7f1690104010 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f1695600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1690114310 con 0x7f16901049c0 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f169019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 0x7f169019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f169019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58754/0 (socket says 172.21.15.59:58754) 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168ea00640 1 -- 172.21.15.59:0/1334289789 learned_addr learned my addr 172.21.15.59:0/1334289789 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f1686000640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16901049c0 0x7f169019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 -- 172.21.15.59:0/1334289789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16901049c0 msgr2=0x7f169019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16901049c0 0x7f169019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 -- 172.21.15.59:0/1334289789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 msgr2=0x7f169019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f169019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.070+0000 7f168f400640 1 -- 172.21.15.59:0/1334289789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019f6a0 con 0x7f1690109140 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f168f400640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 0x7f169019efc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1688009fd0 tx=0x7f168800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f1687e00640 1 -- 172.21.15.59:0/1334289789 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168800ce20 con 0x7f1690109140 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f1687e00640 1 -- 172.21.15.59:0/1334289789 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688005020 con 0x7f1690109140 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019f990 con 0x7f1690109140 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f1687e00640 1 -- 172.21.15.59:0/1334289789 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688010720 con 0x7f1690109140 2024-09-05T12:04:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.071+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a7270 con 0x7f1690109140 2024-09-05T12:04:30.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.074+0000 7f169822c740 1 Processor -- start 2024-09-05T12:04:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.074+0000 7f1687e00640 1 -- 172.21.15.59:0/1334289789 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1688005540 con 0x7f1690109140 2024-09-05T12:04:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.074+0000 7f169822c740 1 -- start start 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fe3590 0x5611a6fed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fede50 0x5611a6ff0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611a6f493e0 con 0x5611a6fe2c70 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5611a6ef0d80 con 0x5611a6fe3590 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f169822c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5611a6f453a0 con 0x5611a6fede50 2024-09-05T12:04:30.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46558/0 (socket says 172.21.15.59:46558) 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 -- 172.21.15.59:0/1801963724 learned_addr learned my addr 172.21.15.59:0/1801963724 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f168ea00640 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fe3590 0x5611a6fed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fede50 0x5611a6ff0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 -- 172.21.15.59:0/1801963724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fede50 msgr2=0x5611a6ff0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fede50 0x5611a6ff0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 -- 172.21.15.59:0/1801963724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fe3590 msgr2=0x5611a6fed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fe3590 0x5611a6fed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.075+0000 7f1686000640 1 -- 172.21.15.59:0/1801963724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a6f0fae0 con 0x5611a6fe2c70 2024-09-05T12:04:30.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f1686000640 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1678009230 tx=0x7f1678005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7cfc0d911d1975 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f1687e00640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 0x7f16740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f167f400640 1 -- 172.21.15.59:0/1801963724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678044020 con 0x5611a6fe2c70 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f167f400640 1 -- 172.21.15.59:0/1801963724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16780044a0 con 0x5611a6fe2c70 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f167f400640 1 -- 172.21.15.59:0/1801963724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678035040 con 0x5611a6fe2c70 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f1687e00640 1 -- 172.21.15.59:0/1334289789 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f16880cfa70 con 0x7f1690109140 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.076+0000 7f168ea00640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 0x7f16740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 -- 172.21.15.59:0/1801963724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 msgr2=0x5611a6fe3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1678009230 tx=0x7f1678005bc0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 -- 172.21.15.59:0/1801963724 shutdown_connections 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fede50 0x5611a6ff0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fe3590 0x5611a6fed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 --2- 172.21.15.59:0/1801963724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fe3050 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 -- 172.21.15.59:0/1801963724 >> 172.21.15.59:0/1801963724 conn(0x5611a6fdf200 msgr2=0x5611a6fdf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f168ea00640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 0x7f16740a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1680009620 tx=0x7f1680005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 -- 172.21.15.59:0/1801963724 shutdown_connections 2024-09-05T12:04:30.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.077+0000 7f169822c740 1 -- 172.21.15.59:0/1801963724 wait complete. 2024-09-05T12:04:30.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 -- start start 2024-09-05T12:04:30.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 Processor -- start 2024-09-05T12:04:30.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 0x5611a6feb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6feb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611a6ff0a10 con 0x5611a6fe2c70 2024-09-05T12:04:30.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5611a6f040f0 con 0x5611a6fede50 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.078+0000 7f169822c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5611a6ff0b90 con 0x5611a6fe3590 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f169822c740 10 client.0 _ll_register_callbacks cb 0x5611a6eebbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46572/0 (socket says 172.21.15.59:46572) 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 -- 172.21.15.59:0/3604137741 learned_addr learned my addr 172.21.15.59:0/3604137741 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168f400640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6feb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f1686000640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 0x5611a6feb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 -- 172.21.15.59:0/3604137741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 msgr2=0x5611a6feb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 0x5611a6feb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 -- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 msgr2=0x5611a6feb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6feb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168ea00640 1 -- 172.21.15.59:0/3604137741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a6cba3c0 con 0x5611a6fe2c70 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f1686000640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 0x5611a6feb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.079+0000 7f168f400640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6feb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f168ea00640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1680004640 tx=0x7f1680004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dc63db7acfee03 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f1673e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168001f640 con 0x5611a6fe2c70 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f168001f7e0 con 0x5611a6fe2c70 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16780044a0 con 0x5611a6fe2c70 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f168001f9e0 con 0x5611a6fe2c70 2024-09-05T12:04:30.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f169822c740 10 client.16386 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5611a6ffca80 con 0x5611a6fe2c70 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.080+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16780044a0 con 0x5611a6fe2c70 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1680027020 con 0x5611a6fe2c70 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 10 client.16386 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 20 client.16386 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a6ffdf40 0x5611a7000330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 4 client.16386 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a6f56950 con 0x5611a6ffdf40 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.081+0000 7f1686000640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a6ffdf40 0x5611a7000330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.082+0000 7f1686000640 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a6ffdf40 0x5611a7000330 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba8f9c816ad4fad server_cookie=5355fa87960f1dbf in_seq=0 out_seq=0 2024-09-05T12:04:30.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.082+0000 7f1673e00640 10 client.16386 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:30.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.082+0000 7f1673e00640 1 -- 172.21.15.59:0/3604137741 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611a6f56950 con 0x5611a6ffdf40 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.082+0000 7f1673e00640 10 client.16386 handle_command_reply: tid=0 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.082+0000 7f169822c740 1 client.16386 shutdown 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 msgr2=0x5611a6fead10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1680004640 tx=0x7f1680004730 comp rx=0 tx=0).stop 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 shutdown_connections 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6feb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe3590 0x5611a6feb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a6ffdf40 0x5611a7000330 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 --2- 172.21.15.59:0/3604137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe2c70 0x5611a6fead10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 >> 172.21.15.59:0/3604137741 conn(0x5611a6fdf200 msgr2=0x5611a6fe4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f1673e00640 0 client.16386 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 shutdown_connections 2024-09-05T12:04:30.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 1 -- 172.21.15.59:0/3604137741 wait complete. 2024-09-05T12:04:30.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.083+0000 7f169822c740 20 client.16386 trim_cache size 0 max 16384 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 Processor -- start 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 -- start start 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6fe5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a7005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a7006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16901aefc0 con 0x5611a6fe3590 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b1840 con 0x5611a6fede50 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.097+0000 7f169822c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16901af630 con 0x5611a6fe2c70 2024-09-05T12:04:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a7005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a7005be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46580/0 (socket says 172.21.15.59:46580) 2024-09-05T12:04:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a7006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f1686000640 1 -- 172.21.15.59:0/3724481753 learned_addr learned my addr 172.21.15.59:0/3724481753 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168ea00640 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6fe5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 -- 172.21.15.59:0/3724481753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 msgr2=0x5611a6fe5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6fe5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 -- 172.21.15.59:0/3724481753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 msgr2=0x5611a7005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a7005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 -- 172.21.15.59:0/3724481753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a6fdf030 con 0x5611a6fede50 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.098+0000 7f168f400640 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a7006120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f168809bdf0 tx=0x7f16880ccfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be33fc3395185353 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f167ea00640 1 -- 172.21.15.59:0/3724481753 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880a4ad0 con 0x5611a6fede50 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f167ea00640 1 -- 172.21.15.59:0/3724481753 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168809a160 con 0x5611a6fede50 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f167ea00640 1 -- 172.21.15.59:0/3724481753 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880a48a0 con 0x5611a6fede50 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 -- 172.21.15.59:0/3724481753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 msgr2=0x5611a7006120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a7006120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f168809bdf0 tx=0x7f16880ccfd0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 -- 172.21.15.59:0/3724481753 shutdown_connections 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a7006120 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a7005be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 --2- 172.21.15.59:0/3724481753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6fe5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 -- 172.21.15.59:0/3724481753 >> 172.21.15.59:0/3724481753 conn(0x5611a6fdf200 msgr2=0x5611a6fdf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 -- 172.21.15.59:0/3724481753 shutdown_connections 2024-09-05T12:04:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.099+0000 7f169822c740 1 -- 172.21.15.59:0/3724481753 wait complete. 2024-09-05T12:04:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 -- start start 2024-09-05T12:04:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 Processor -- start 2024-09-05T12:04:30.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6fe7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16901afd00 con 0x5611a6fe3590 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b1170 con 0x5611a6fede50 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.100+0000 7f169822c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b03d0 con 0x5611a6fe2c70 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f169822c740 10 client.0 _ll_register_callbacks cb 0x7f1680009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f1686000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46592/0 (socket says 172.21.15.59:46592) 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6fe7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38530/0 (socket says 172.21.15.59:38530) 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f1686000640 1 -- 172.21.15.59:0/1398027606 learned_addr learned my addr 172.21.15.59:0/1398027606 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 -- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 msgr2=0x5611a6fe7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6fe7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 -- 172.21.15.59:0/1398027606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 msgr2=0x5611a6ff6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f168ea00640 1 -- 172.21.15.59:0/1398027606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a6fda770 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.101+0000 7f1686000640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f168ea00640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1680019040 tx=0x7f1680003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d47c0d80e6d9c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f1673e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168003c030 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168809a160 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1680029650 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16800043e0 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f169822c740 10 client.26329 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5611a7068300 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.102+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168809a160 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1680027040 con 0x5611a6fe2c70 2024-09-05T12:04:30.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 10 client.26329 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 20 client.26329 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a7068b00 0x5611a706af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 4 client.26329 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611a6f50e60 con 0x5611a7068b00 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.103+0000 7f1686000640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a7068b00 0x5611a706af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f1686000640 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a7068b00 0x5611a706af20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3bdce7f75b5f48 server_cookie=f7851e98ab27b84 in_seq=0 out_seq=0 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f1673e00640 10 client.26329 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f1673e00640 1 -- 172.21.15.59:0/1398027606 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611a6f50e60 con 0x5611a7068b00 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f1673e00640 10 client.26329 handle_command_reply: tid=0 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 client.26329 shutdown 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 msgr2=0x5611a6ff5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1680019040 tx=0x7f1680003ed0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 shutdown_connections 2024-09-05T12:04:30.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5611a6fede50 0x5611a6fe7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5611a7068b00 0x5611a706af20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611a6fe3590 0x5611a6ff6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 --2- 172.21.15.59:0/1398027606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5611a6fe2c70 0x5611a6ff5de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 >> 172.21.15.59:0/1398027606 conn(0x5611a6fdf200 msgr2=0x5611a6fddb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f1673e00640 0 client.26329 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 shutdown_connections 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 1 -- 172.21.15.59:0/1398027606 wait complete. 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.104+0000 7f169822c740 20 client.26329 trim_cache size 0 max 16384 2024-09-05T12:04:30.110 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 msgr2=0x7f16740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 0x7f16740a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1680009620 tx=0x7f1680005f90 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 msgr2=0x7f169019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 0x7f169019efc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1688009fd0 tx=0x7f168800ef10 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 shutdown_connections 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f16740a2c00 0x7f16740a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1690109140 0x7f169019efc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16901049c0 0x7f169019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 --2- 172.21.15.59:0/1334289789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104010 0x7f169019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 >> 172.21.15.59:0/1334289789 conn(0x7f16900ffa50 msgr2=0x7f1690100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 shutdown_connections 2024-09-05T12:04:30.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:30.105+0000 7f1695600640 1 -- 172.21.15.59:0/1334289789 wait complete. 2024-09-05T12:04:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:30 smithi189 ceph-mon[34772]: pgmap v599: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:30 smithi089 ceph-mon[34499]: pgmap v599: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:30 smithi059 ceph-mon[30025]: pgmap v599: 129 pgs: 129 active+clean; 3.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:31.882 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:04:31.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:04:32.274 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:04:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 -- 172.21.15.59:0/3852993070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98103e30 msgr2=0x7f0b98104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 --2- 172.21.15.59:0/3852993070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98103e30 0x7f0b98104210 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 -- 172.21.15.59:0/3852993070 shutdown_connections 2024-09-05T12:04:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 --2- 172.21.15.59:0/3852993070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b98108f60 0x7f0b98110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 --2- 172.21.15.59:0/3852993070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b981047e0 0x7f0b98108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 --2- 172.21.15.59:0/3852993070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98103e30 0x7f0b98104210 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 -- 172.21.15.59:0/3852993070 >> 172.21.15.59:0/3852993070 conn(0x7f0b980ff8d0 msgr2=0x7f0b98101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.845+0000 7f0b9e600640 1 -- 172.21.15.59:0/3852993070 shutdown_connections 2024-09-05T12:04:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.846+0000 7f0b9e600640 1 -- 172.21.15.59:0/3852993070 wait complete. 2024-09-05T12:04:32.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.846+0000 7f0b9e600640 1 Processor -- start 2024-09-05T12:04:32.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 -- start start 2024-09-05T12:04:32.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 0x7f0b9819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b981047e0 0x7f0b9819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98114010 con 0x7f0b98108f60 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98113e90 con 0x7f0b98103e30 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98114190 con 0x7f0b981047e0 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40984/0 (socket says 172.21.15.59:40984) 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9dc00640 1 -- 172.21.15.59:0/458254588 learned_addr learned my addr 172.21.15.59:0/458254588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b9d200640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 0x7f0b9819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.847+0000 7f0b97e00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b981047e0 0x7f0b9819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 -- 172.21.15.59:0/458254588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b981047e0 msgr2=0x7f0b9819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b981047e0 0x7f0b9819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 -- 172.21.15.59:0/458254588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 msgr2=0x7f0b9819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 0x7f0b9819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 -- 172.21.15.59:0/458254588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9819f3e0 con 0x7f0b98108f60 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9d200640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 0x7f0b9819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9dc00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000eb10 tx=0x7f0b9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b95600640 1 -- 172.21.15.59:0/458254588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004b20 con 0x7f0b98108f60 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b95600640 1 -- 172.21.15.59:0/458254588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90004cc0 con 0x7f0b98108f60 2024-09-05T12:04:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9819f6d0 con 0x7f0b98108f60 2024-09-05T12:04:32.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.848+0000 7f0b95600640 1 -- 172.21.15.59:0/458254588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90010760 con 0x7f0b98108f60 2024-09-05T12:04:32.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.849+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b981a6fb0 con 0x7f0b98108f60 2024-09-05T12:04:32.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 Processor -- start 2024-09-05T12:04:32.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0b95600640 1 -- 172.21.15.59:0/458254588 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b90010900 con 0x7f0b98108f60 2024-09-05T12:04:32.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 -- start start 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d406dc0290 con 0x55d406e64e90 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d406d67b40 con 0x55d406e5a5d0 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.852+0000 7f0ba284f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d406dbc250 con 0x55d406e59cb0 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:33484/0 (socket says 172.21.15.59:33484) 2024-09-05T12:04:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 -- 172.21.15.59:0/335380196 learned_addr learned my addr 172.21.15.59:0/335380196 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b97e00640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b97e00640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 -- 172.21.15.59:0/335380196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 msgr2=0x55d406e5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 -- 172.21.15.59:0/335380196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 msgr2=0x55d406e67280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e67280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 -- 172.21.15.59:0/335380196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d406b316a0 con 0x55d406e5a5d0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b97e00640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b95600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 0x7f0b800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b9d200640 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb69d257d34f5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.853+0000 7f0b95600640 1 -- 172.21.15.59:0/458254588 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0b900cf990 con 0x7f0b98108f60 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0b97e00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 0x7f0b800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0b8a000640 1 -- 172.21.15.59:0/335380196 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c035040 con 0x55d406e5a5d0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0b8a000640 1 -- 172.21.15.59:0/335380196 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c004760 con 0x55d406e5a5d0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0b8a000640 1 -- 172.21.15.59:0/335380196 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0038e0 con 0x55d406e5a5d0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0ba284f740 1 -- 172.21.15.59:0/335380196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 msgr2=0x55d406e64950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0b97e00640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 0x7f0b800a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d406e575c0 tx=0x7f0b84013040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.854+0000 7f0ba284f740 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c005de0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 -- 172.21.15.59:0/335380196 shutdown_connections 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e64950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 --2- 172.21.15.59:0/335380196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 -- 172.21.15.59:0/335380196 >> 172.21.15.59:0/335380196 conn(0x55d406e56240 msgr2=0x55d406e56620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 -- 172.21.15.59:0/335380196 shutdown_connections 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 -- 172.21.15.59:0/335380196 wait complete. 2024-09-05T12:04:32.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 -- start start 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.855+0000 7f0ba284f740 1 Processor -- start 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e61a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e61fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d406e67aa0 con 0x55d406e64e90 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d406d7afa0 con 0x55d406e5a5d0 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d406e67c20 con 0x55d406e59cb0 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0ba284f740 10 client.0 _ll_register_callbacks cb 0x55d406aceba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0b9dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e61fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40650/0 (socket says 172.21.15.59:40650) 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0b9d200640 1 -- 172.21.15.59:0/645980421 learned_addr learned my addr 172.21.15.59:0/645980421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:32.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.856+0000 7f0b97e00640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e61a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 -- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 msgr2=0x55d406e61a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e61a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 -- 172.21.15.59:0/645980421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 msgr2=0x55d406e61fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e61fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 -- 172.21.15.59:0/645980421 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d406e627a0 con 0x55d406e59cb0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b9d200640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006010 tx=0x7f0b8c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1adfa10ab40d8b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c052060 con 0x55d406e59cb0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b8c004760 con 0x55d406e59cb0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.857+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0b8c005130 con 0x55d406e59cb0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c035070 con 0x55d406e59cb0 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 10 client.26341 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:32.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d406e67aa0 con 0x55d406e59cb0 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c004760 con 0x55d406e59cb0 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0b8c0315b0 con 0x55d406e59cb0 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 10 client.26341 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 20 client.26341 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.858+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406e74b00 0x55d406e76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.859+0000 7f0ba284f740 4 client.26341 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.859+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d406dcd800 con 0x55d406e74b00 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.859+0000 7f0b97e00640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406e74b00 0x55d406e76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.859+0000 7f0b97e00640 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406e74b00 0x55d406e76ef0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761f18b58ea3c65f server_cookie=cdfd17ac0e493f80 in_seq=0 out_seq=0 2024-09-05T12:04:32.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.859+0000 7f0b7ea00640 10 client.26341 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.860+0000 7f0b7ea00640 1 -- 172.21.15.59:0/645980421 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d406dcd800 con 0x55d406e74b00 2024-09-05T12:04:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.860+0000 7f0b7ea00640 10 client.26341 handle_command_reply: tid=0 2024-09-05T12:04:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.860+0000 7f0ba284f740 1 client.26341 shutdown 2024-09-05T12:04:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 msgr2=0x55d406e647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006010 tx=0x7f0b8c004b40 comp rx=0 tx=0).stop 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 shutdown_connections 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406e74b00 0x55d406e76ef0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e64e90 0x55d406e61fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e61a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 --2- 172.21.15.59:0/645980421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e59cb0 0x55d406e647b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0b7ea00640 0 client.26341 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 >> 172.21.15.59:0/645980421 conn(0x55d406e56240 msgr2=0x55d406e5bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 shutdown_connections 2024-09-05T12:04:32.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 1 -- 172.21.15.59:0/645980421 wait complete. 2024-09-05T12:04:32.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.861+0000 7f0ba284f740 20 client.26341 trim_cache size 0 max 16384 2024-09-05T12:04:32.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.879+0000 7f0ba284f740 1 Processor -- start 2024-09-05T12:04:32.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.879+0000 7f0ba284f740 1 -- start start 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e5a5d0 0x55d406e7c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e64e90 0x55d406e7cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981aef20 con 0x55d406e59cb0 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b17a0 con 0x55d406e64e90 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0ba284f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981af590 con 0x55d406e5a5d0 2024-09-05T12:04:32.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41026/0 (socket says 172.21.15.59:41026) 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 -- 172.21.15.59:0/1305146668 learned_addr learned my addr 172.21.15.59:0/1305146668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9dc00640 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e64e90 0x55d406e7cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b97e00640 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e5a5d0 0x55d406e7c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 -- 172.21.15.59:0/1305146668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e5a5d0 msgr2=0x55d406e7c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e5a5d0 0x55d406e7c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 -- 172.21.15.59:0/1305146668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e64e90 msgr2=0x55d406e7cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e64e90 0x55d406e7cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.880+0000 7f0b9d200640 1 -- 172.21.15.59:0/1305146668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d406e622e0 con 0x55d406e59cb0 2024-09-05T12:04:32.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.881+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02a910 tx=0x7f0b8c02ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc00f79956fef380 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.881+0000 7f0b89600640 1 -- 172.21.15.59:0/1305146668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c000e60 con 0x55d406e59cb0 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.881+0000 7f0b89600640 1 -- 172.21.15.59:0/1305146668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c009520 con 0x55d406e59cb0 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.881+0000 7f0b89600640 1 -- 172.21.15.59:0/1305146668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c052070 con 0x55d406e59cb0 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- 172.21.15.59:0/1305146668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 msgr2=0x55d406e55160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c02a910 tx=0x7f0b8c02ab70 comp rx=0 tx=0).stop 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- 172.21.15.59:0/1305146668 shutdown_connections 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e64e90 0x55d406e7cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e5a5d0 0x55d406e7c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1305146668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e55160 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- 172.21.15.59:0/1305146668 >> 172.21.15.59:0/1305146668 conn(0x55d406e56240 msgr2=0x55d406e54a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- 172.21.15.59:0/1305146668 shutdown_connections 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- 172.21.15.59:0/1305146668 wait complete. 2024-09-05T12:04:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.882+0000 7f0ba284f740 1 -- start start 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 Processor -- start 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e64e90 0x55d406e6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981afc60 con 0x55d406e59cb0 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b10d0 con 0x55d406e5a5d0 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b0330 con 0x55d406e64e90 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0ba284f740 10 client.0 _ll_register_callbacks cb 0x7f0b84009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0b9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41030/0 (socket says 172.21.15.59:41030) 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0b9d200640 1 -- 172.21.15.59:0/1141289154 learned_addr learned my addr 172.21.15.59:0/1141289154 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0b97e00640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.883+0000 7f0b9dc00640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e64e90 0x55d406e6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 -- 172.21.15.59:0/1141289154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e64e90 msgr2=0x55d406e6b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e64e90 0x55d406e6b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 -- 172.21.15.59:0/1141289154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 msgr2=0x55d406e6ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 -- 172.21.15.59:0/1141289154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d406ede840 con 0x55d406e5a5d0 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b97e00640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e6b3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55d406e630a0 tx=0x7f0b84005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975323f9c973bb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:04:32.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.884+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84005020 con 0x55d406e5a5d0 2024-09-05T12:04:32.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0b8401a070 con 0x55d406e5a5d0 2024-09-05T12:04:32.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b8c009520 con 0x55d406e5a5d0 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b8403c050 con 0x55d406e5a5d0 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0ba284f740 10 client.26411 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d406edfbe0 con 0x55d406e5a5d0 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c009520 con 0x55d406e5a5d0 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0b840035f0 con 0x55d406e5a5d0 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0ba284f740 10 client.26411 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.885+0000 7f0ba284f740 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0ba284f740 20 client.26411 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406ee0330 0x55d406ee2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0ba284f740 4 client.26411 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d406dc7d10 con 0x55d406ee0330 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406ee0330 0x55d406ee2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0b9d200640 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406ee0330 0x55d406ee2750 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c868f271a27fd6d server_cookie=811505a891fb2c28 in_seq=0 out_seq=0 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.886+0000 7f0b7ea00640 10 client.26411 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0b7ea00640 1 -- 172.21.15.59:0/1141289154 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d406dc7d10 con 0x55d406ee0330 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0b7ea00640 10 client.26411 handle_command_reply: tid=0 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 client.26411 shutdown 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 msgr2=0x55d406e6b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e6b3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55d406e630a0 tx=0x7f0b84005f90 comp rx=0 tx=0).stop 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 shutdown_connections 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d406e64e90 0x55d406e6b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d406e5a5d0 0x55d406e6b3a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d406ee0330 0x55d406ee2750 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 --2- 172.21.15.59:0/1141289154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d406e59cb0 0x55d406e6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 >> 172.21.15.59:0/1141289154 conn(0x55d406e56240 msgr2=0x55d406e55270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0b7ea00640 0 client.26411 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 shutdown_connections 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 1 -- 172.21.15.59:0/1141289154 wait complete. 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.888+0000 7f0ba284f740 20 client.26411 trim_cache size 0 max 16384 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:04:32.893 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:04:32.894 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "df3b7fa0-ec6e-4a83-99e7-c4b80ac3ff36", 2024-09-05T12:04:32.894 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:04:32.894 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:04:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 msgr2=0x7f0b800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 0x7f0b800a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55d406e575c0 tx=0x7f0b84013040 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 msgr2=0x7f0b9819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000eb10 tx=0x7f0b9000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 shutdown_connections 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0b800a2c00 0x7f0b800a50c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b98108f60 0x7f0b9819ed00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b981047e0 0x7f0b9819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 --2- 172.21.15.59:0/458254588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b98103e30 0x7f0b9819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.889+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 >> 172.21.15.59:0/458254588 conn(0x7f0b980ff8d0 msgr2=0x7f0b98100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.890+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 shutdown_connections 2024-09-05T12:04:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:32.890+0000 7f0b9e600640 1 -- 172.21.15.59:0/458254588 wait complete. 2024-09-05T12:04:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:32 smithi189 ceph-mon[34772]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:32 smithi089 ceph-mon[34499]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:33.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:32 smithi059 ceph-mon[30025]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:33.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:04:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:33 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:04:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:33 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:33 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:04:33.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:33 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:33.983 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:04:34.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:33 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:04:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:33 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:33 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:04:34.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:33 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:33 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:04:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:33 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:33 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:04:34.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:33 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:04:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 -- 172.21.15.59:0/533982777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 msgr2=0x7fa88c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 --2- 172.21.15.59:0/533982777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c110ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa884009be0 tx=0x7fa88402a6d0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 -- 172.21.15.59:0/533982777 shutdown_connections 2024-09-05T12:04:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 --2- 172.21.15.59:0/533982777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c110ca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 --2- 172.21.15.59:0/533982777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c1049a0 0x7fa88c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 --2- 172.21.15.59:0/533982777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c103ff0 0x7fa88c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 -- 172.21.15.59:0/533982777 >> 172.21.15.59:0/533982777 conn(0x7fa88c0ffa90 msgr2=0x7fa88c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 -- 172.21.15.59:0/533982777 shutdown_connections 2024-09-05T12:04:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.570+0000 7fa891400640 1 -- 172.21.15.59:0/533982777 wait complete. 2024-09-05T12:04:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.571+0000 7fa891400640 1 Processor -- start 2024-09-05T12:04:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.571+0000 7fa891400640 1 -- start start 2024-09-05T12:04:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c1049a0 0x7fa88c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c114150 con 0x7fa88c103ff0 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c113fd0 con 0x7fa88c1049a0 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa891400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1142d0 con 0x7fa88c109120 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa88a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa88a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41046/0 (socket says 172.21.15.59:41046) 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa88a800640 1 -- 172.21.15.59:0/1563479015 learned_addr learned my addr 172.21.15.59:0/1563479015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa889e00640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c1049a0 0x7fa88c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.572+0000 7fa88b200640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 -- 172.21.15.59:0/1563479015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 msgr2=0x7fa88c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 -- 172.21.15.59:0/1563479015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c1049a0 msgr2=0x7fa88c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c1049a0 0x7fa88c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 -- 172.21.15.59:0/1563479015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c19f560 con 0x7fa88c103ff0 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa88a800640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9f0 tx=0x7fa87c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa883400640 1 -- 172.21.15.59:0/1563479015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00ccc0 con 0x7fa88c103ff0 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c19f850 con 0x7fa88c103ff0 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa883400640 1 -- 172.21.15.59:0/1563479015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c005500 con 0x7fa88c103ff0 2024-09-05T12:04:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa883400640 1 -- 172.21.15.59:0/1563479015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c010690 con 0x7fa88c103ff0 2024-09-05T12:04:34.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.573+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c1a7130 con 0x7fa88c103ff0 2024-09-05T12:04:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa883400640 1 -- 172.21.15.59:0/1563479015 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa87c0046c0 con 0x7fa88c103ff0 2024-09-05T12:04:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 Processor -- start 2024-09-05T12:04:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 -- start start 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c7160af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c71618160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c715713e0 con 0x555c71615d70 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555c71518d80 con 0x555c7160b4b0 2024-09-05T12:04:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.577+0000 7fa893fd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c7156d3a0 con 0x555c7160ab90 2024-09-05T12:04:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c7160af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:33552/0 (socket says 172.21.15.59:33552) 2024-09-05T12:04:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 -- 172.21.15.59:0/543373252 learned_addr learned my addr 172.21.15.59:0/543373252 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa889e00640 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c71618160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 -- 172.21.15.59:0/543373252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 msgr2=0x555c7160af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c7160af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 -- 172.21.15.59:0/543373252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 msgr2=0x555c71618160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c71618160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.578+0000 7fa88a800640 1 -- 172.21.15.59:0/543373252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c71537ae0 con 0x555c7160b4b0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa88a800640 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d5a90 tx=0x7fa87c0d5b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601eb7bb80e8c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa883400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 0x7fa8700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa877400640 1 -- 172.21.15.59:0/543373252 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0de070 con 0x555c7160b4b0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa877400640 1 -- 172.21.15.59:0/543373252 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09c290 con 0x555c7160b4b0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa877400640 1 -- 172.21.15.59:0/543373252 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c004f20 con 0x555c7160b4b0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa883400640 1 -- 172.21.15.59:0/1563479015 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa87c0cf7a0 con 0x7fa88c103ff0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.579+0000 7fa88b200640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 0x7fa8700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 -- 172.21.15.59:0/543373252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 msgr2=0x555c71615830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0d5a90 tx=0x7fa87c0d5b80 comp rx=0 tx=0).stop 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 -- 172.21.15.59:0/543373252 shutdown_connections 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c71618160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c71615830 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 --2- 172.21.15.59:0/543373252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c7160af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 -- 172.21.15.59:0/543373252 >> 172.21.15.59:0/543373252 conn(0x555c71607120 msgr2=0x555c71607500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa88b200640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 0x7fa8700a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa884006010 tx=0x7fa884009840 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 -- 172.21.15.59:0/543373252 shutdown_connections 2024-09-05T12:04:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.580+0000 7fa893fd0740 1 -- 172.21.15.59:0/543373252 wait complete. 2024-09-05T12:04:34.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 -- start start 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 Processor -- start 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c716131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c71615d70 0x555c71613700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c716189e0 con 0x555c7160b4b0 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555c7152c0f0 con 0x555c71615d70 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c71618b60 con 0x555c7160ab90 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.581+0000 7fa893fd0740 10 client.0 _ll_register_callbacks cb 0x555c7151d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa889e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c716131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40688/0 (socket says 172.21.15.59:40688) 2024-09-05T12:04:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 -- 172.21.15.59:0/3898849239 learned_addr learned my addr 172.21.15.59:0/3898849239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88b200640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c71615d70 0x555c71613700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 -- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c71615d70 msgr2=0x555c71613700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c71615d70 0x555c71613700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 -- 172.21.15.59:0/3898849239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 msgr2=0x555c716131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c716131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 -- 172.21.15.59:0/3898849239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c712e23c0 con 0x555c7160ab90 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa88a800640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa87c097ff0 tx=0x7fa87c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea404eb79ee269f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.582+0000 7fa86be00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020ae0 con 0x555c7160ab90 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87c09c290 con 0x555c7160ab90 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa87c0ccf40 con 0x555c7160ab90 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa87c01f6c0 con 0x555c7160ab90 2024-09-05T12:04:34.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa893fd0740 10 client.26359 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555c716189e0 con 0x555c7160ab90 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.583+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09c290 con 0x555c7160ab90 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa87c0ce020 con 0x555c7160ab90 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 10 client.26359 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 10 client.26359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 20 client.26359 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71625840 0x555c71627c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 4 client.26359 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c7157e950 con 0x555c71625840 2024-09-05T12:04:34.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.584+0000 7fa889e00640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71625840 0x555c71627c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.585+0000 7fa889e00640 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71625840 0x555c71627c30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703f9d1d7fcb3e69 server_cookie=7a1e5d675694bbed in_seq=0 out_seq=0 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.585+0000 7fa86be00640 10 client.26359 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.585+0000 7fa86be00640 1 -- 172.21.15.59:0/3898849239 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c7157e950 con 0x555c71625840 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.585+0000 7fa86be00640 10 client.26359 handle_command_reply: tid=0 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 client.26359 shutdown 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 msgr2=0x555c71612c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa87c097ff0 tx=0x7fa87c00cf60 comp rx=0 tx=0).stop 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 shutdown_connections 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c71615d70 0x555c71613700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71625840 0x555c71627c30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c716131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3898849239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c71612c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa86be00640 0 client.26359 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 >> 172.21.15.59:0/3898849239 conn(0x555c71607120 msgr2=0x555c7160ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 shutdown_connections 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 1 -- 172.21.15.59:0/3898849239 wait complete. 2024-09-05T12:04:34.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.586+0000 7fa893fd0740 20 client.26359 trim_cache size 0 max 16384 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.604+0000 7fa893fd0740 1 Processor -- start 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.604+0000 7fa893fd0740 1 -- start start 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c716144e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c7162d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1af0d0 con 0x555c71615d70 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1b1950 con 0x555c7160b4b0 2024-09-05T12:04:34.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa893fd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1af740 con 0x555c7160ab90 2024-09-05T12:04:34.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa88b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa88b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41068/0 (socket says 172.21.15.59:41068) 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa88b200640 1 -- 172.21.15.59:0/3964213155 learned_addr learned my addr 172.21.15.59:0/3964213155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa88a800640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c716144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.605+0000 7fa889e00640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c7162d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 -- 172.21.15.59:0/3964213155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 msgr2=0x555c716144e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c716144e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 -- 172.21.15.59:0/3964213155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 msgr2=0x555c7162d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c7162d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 -- 172.21.15.59:0/3964213155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7160c110 con 0x555c71615d70 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa889e00640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c7162d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa88b200640 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa884033000 tx=0x7fa88403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521db01f6db6626f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa876a00640 1 -- 172.21.15.59:0/3964213155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884036040 con 0x555c71615d70 2024-09-05T12:04:34.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa876a00640 1 -- 172.21.15.59:0/3964213155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa884003c30 con 0x555c71615d70 2024-09-05T12:04:34.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.606+0000 7fa876a00640 1 -- 172.21.15.59:0/3964213155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8840319a0 con 0x555c71615d70 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- 172.21.15.59:0/3964213155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 msgr2=0x555c7162d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa884033000 tx=0x7fa88403f000 comp rx=0 tx=0).stop 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- 172.21.15.59:0/3964213155 shutdown_connections 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c71615d70 0x555c7162d9a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160b4b0 0x555c7162d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 --2- 172.21.15.59:0/3964213155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c7160ab90 0x555c716144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- 172.21.15.59:0/3964213155 >> 172.21.15.59:0/3964213155 conn(0x555c71607120 msgr2=0x555c71613460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- 172.21.15.59:0/3964213155 shutdown_connections 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- 172.21.15.59:0/3964213155 wait complete. 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.607+0000 7fa893fd0740 1 -- start start 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 Processor -- start 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160ab90 0x555c71619a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c71615d70 0x555c7161a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1afe10 con 0x555c7160b4b0 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1b1280 con 0x555c7160ab90 2024-09-05T12:04:34.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c1b04e0 con 0x555c71615d70 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa893fd0740 10 client.0 _ll_register_callbacks cb 0x7fa87c0d58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa889e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa889e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41072/0 (socket says 172.21.15.59:41072) 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.608+0000 7fa889e00640 1 -- 172.21.15.59:0/4127260970 learned_addr learned my addr 172.21.15.59:0/4127260970 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa88a800640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160ab90 0x555c71619a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa88b200640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c71615d70 0x555c7161a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 -- 172.21.15.59:0/4127260970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c71615d70 msgr2=0x555c7161a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c71615d70 0x555c7161a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 -- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160ab90 msgr2=0x555c71619a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160ab90 0x555c71619a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 -- 172.21.15.59:0/4127260970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7168f430 con 0x555c7160b4b0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa889e00640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x555c71612510 tx=0x7fa878010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dc75deac0efed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.609+0000 7fa86be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87801ebd0 con 0x555c7160b4b0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa884003c30 con 0x555c7160b4b0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa87801ed70 con 0x555c7160b4b0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa878034050 con 0x555c7160b4b0 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 10 client.16428 fetch_fsmap learned FSMap version 21 2024-09-05T12:04:34.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555c716906a0 con 0x555c7160b4b0 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa884003c30 con 0x555c7160b4b0 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa87c01cd50 con 0x555c7160b4b0 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 10 client.16428 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 20 client.16428 populate_metadata read hostname 'smithi059' 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71690df0 0x555c71693210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:04:34.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 4 client.16428 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.610+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c71578e60 con 0x555c71690df0 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.611+0000 7fa88a800640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71690df0 0x555c71693210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.611+0000 7fa88a800640 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71690df0 0x555c71693210 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9002f0a8a2022978 server_cookie=8d8c6937446feba2 in_seq=0 out_seq=0 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.611+0000 7fa86be00640 10 client.16428 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.611+0000 7fa86be00640 1 -- 172.21.15.59:0/4127260970 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555c71578e60 con 0x555c71690df0 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.611+0000 7fa86be00640 10 client.16428 handle_command_reply: tid=0 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 client.16428 shutdown 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 msgr2=0x555c71619fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x555c71612510 tx=0x7fa878010eb0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 shutdown_connections 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c71615d70 0x555c7161a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x555c71690df0 0x555c71693210 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c7160b4b0 0x555c71619fc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 --2- 172.21.15.59:0/4127260970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c7160ab90 0x555c71619a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 >> 172.21.15.59:0/4127260970 conn(0x555c71607120 msgr2=0x555c71620c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa86be00640 0 client.16428 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 shutdown_connections 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 1 -- 172.21.15.59:0/4127260970 wait complete. 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.612+0000 7fa893fd0740 20 client.16428 trim_cache size 0 max 16384 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:04:34.618 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: "df3b7fa0-ec6e-4a83-99e7-c4b80ac3ff36": { 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "df3b7fa0-ec6e-4a83-99e7-c4b80ac3ff36", 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:04:34.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 msgr2=0x7fa8700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 0x7fa8700a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa884006010 tx=0x7fa884009840 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 msgr2=0x7fa88c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9f0 tx=0x7fa87c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 shutdown_connections 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa8700a2c00 0x7fa8700a50c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa88c109120 0x7fa88c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa88c1049a0 0x7fa88c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 --2- 172.21.15.59:0/1563479015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa88c103ff0 0x7fa88c19a5b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 >> 172.21.15.59:0/1563479015 conn(0x7fa88c0ffa90 msgr2=0x7fa88c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 shutdown_connections 2024-09-05T12:04:34.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:04:34.613+0000 7fa891400640 1 -- 172.21.15.59:0/1563479015 wait complete. 2024-09-05T12:04:35.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:34 smithi059 ceph-mon[30025]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:34 smithi189 ceph-mon[34772]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:34 smithi089 ceph-mon[34499]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:35.424 INFO:tasks.cephfs.filesystem:scrub status for tag:df3b7fa0-ec6e-4a83-99e7-c4b80ac3ff36 - {'path': '/', 'tag': 'df3b7fa0-ec6e-4a83-99e7-c4b80ac3ff36', 'options': 'recursive,force'} 2024-09-05T12:04:36.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:35 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:04:36.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:35 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:04:36.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:35 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:04:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:36 smithi189 ceph-mon[34772]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:37.174 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:36 smithi089 ceph-mon[34499]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:36 smithi059 ceph-mon[30025]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:38 smithi189 ceph-mon[34772]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:38 smithi089 ceph-mon[34499]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:39.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:38 smithi059 ceph-mon[30025]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:40 smithi189 ceph-mon[34772]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:04:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:40 smithi089 ceph-mon[34499]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:04:41.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:40 smithi059 ceph-mon[30025]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:04:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:42 smithi189 ceph-mon[34772]: pgmap v605: 129 pgs: 129 active+clean; 3.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:42 smithi089 ceph-mon[34499]: pgmap v605: 129 pgs: 129 active+clean; 3.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:43.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:42 smithi059 ceph-mon[30025]: pgmap v605: 129 pgs: 129 active+clean; 3.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:44.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:04:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:04:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:04:44.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:43 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:04:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:04:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:04:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:04:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:43 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:04:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:04:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:04:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:04:44.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:43 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:189/600: open fd = 3 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:190/600: open fd = 3 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.885 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:191/600: open fd = 3 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:192/600: open fd = 3 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:193/600: open fd = 3 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.886 INFO:tasks.workunit.client.0.smithi059.stdout:194/600: open fd = 3 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:195/600: open fd = 3 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:196/600: open fd = 3 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:197/600: open fd = 3 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.887 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:198/600: open fd = 3 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:199/600: open fd = 3 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:200/600: open fd = 3 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.888 INFO:tasks.workunit.client.0.smithi059.stdout:201/600: open fd = 3 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:202/600: open fd = 3 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:203/600: open fd = 3 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:204/600: open fd = 3 2024-09-05T12:04:44.889 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:205/600: open fd = 3 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:206/600: open fd = 3 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:207/600: open fd = 3 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.890 INFO:tasks.workunit.client.0.smithi059.stdout:208/600: open fd = 3 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:209/600: open fd = 3 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:210/600: open fd = 3 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:211/600: open fd = 3 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.891 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:212/600: open fd = 3 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:213/600: open fd = 3 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:214/600: open fd = 3 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.892 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:215/600: open fd = 3 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:216/600: open fd = 3 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:217/600: open fd = 3 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.893 INFO:tasks.workunit.client.0.smithi059.stdout:218/600: open fd = 3 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:219/600: open fd = 3 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:220/600: open fd = 3 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.894 INFO:tasks.workunit.client.0.smithi059.stdout:221/600: open fd = 3 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:222/600: open fd = 3 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:223/600: open fd = 3 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:224/600: open fd = 3 2024-09-05T12:04:44.895 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:225/600: open fd = 3 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:226/600: open fd = 3 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:227/600: open fd = 3 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.896 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:228/600: open fd = 3 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:229/600: open fd = 3 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:230/600: open fd = 3 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.897 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:231/600: open fd = 3 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:232/600: open fd = 3 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:233/600: open fd = 3 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.898 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:234/600: open fd = 3 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:235/600: open fd = 3 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:236/600: open fd = 3 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.899 INFO:tasks.workunit.client.0.smithi059.stdout:237/600: open fd = 3 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:238/600: open fd = 3 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:239/600: open fd = 3 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:240/600: open fd = 3 2024-09-05T12:04:44.900 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:241/600: open fd = 3 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:242/600: open fd = 3 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:243/600: open fd = 3 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.901 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:244/600: open fd = 3 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:245/600: open fd = 3 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:246/600: open fd = 3 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.902 INFO:tasks.workunit.client.0.smithi059.stdout:247/600: open fd = 3 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:248/600: open fd = 3 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:249/600: open fd = 3 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:250/600: open fd = 3 2024-09-05T12:04:44.903 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:44.904 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:04:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:44 smithi189 ceph-mon[34772]: pgmap v606: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:45.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:44 smithi059 ceph-mon[30025]: pgmap v606: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:44 smithi089 ceph-mon[34499]: pgmap v606: 129 pgs: 129 active+clean; 3.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:46.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:46 smithi059 ceph-mon[30025]: pgmap v607: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:46 smithi189 ceph-mon[34772]: pgmap v607: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:46 smithi089 ceph-mon[34499]: pgmap v607: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:04:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:48 smithi189 ceph-mon[34772]: pgmap v608: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:48 smithi089 ceph-mon[34499]: pgmap v608: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:49.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:48 smithi059 ceph-mon[30025]: pgmap v608: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:04:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:50 smithi189 ceph-mon[34772]: pgmap v609: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:50 smithi089 ceph-mon[34499]: pgmap v609: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:50 smithi059 ceph-mon[30025]: pgmap v609: 129 pgs: 129 active+clean; 3.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:04:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:52 smithi189 ceph-mon[34772]: pgmap v610: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:52 smithi089 ceph-mon[34499]: pgmap v610: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:52 smithi059 ceph-mon[30025]: pgmap v610: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:04:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:54 smithi189 ceph-mon[34772]: pgmap v611: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:54 smithi089 ceph-mon[34499]: pgmap v611: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:55.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:54 smithi059 ceph-mon[30025]: pgmap v611: 129 pgs: 129 active+clean; 3.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:56 smithi189 ceph-mon[34772]: pgmap v612: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:56 smithi089 ceph-mon[34499]: pgmap v612: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:57.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:56 smithi059 ceph-mon[30025]: pgmap v612: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:04:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:04:58 smithi189 ceph-mon[34772]: pgmap v613: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:04:58 smithi089 ceph-mon[34499]: pgmap v613: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:04:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:04:58 smithi059 ceph-mon[30025]: pgmap v613: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:00 smithi189 ceph-mon[34772]: pgmap v614: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:05:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:00 smithi089 ceph-mon[34499]: pgmap v614: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:05:01.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:00 smithi059 ceph-mon[30025]: pgmap v614: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:05:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:02 smithi189 ceph-mon[34772]: pgmap v615: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:05:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:02 smithi089 ceph-mon[34499]: pgmap v615: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:05:03.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:02 smithi059 ceph-mon[30025]: pgmap v615: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:05:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:04 smithi189 ceph-mon[34772]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:04 smithi089 ceph-mon[34499]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:04 smithi059 ceph-mon[30025]: pgmap v616: 129 pgs: 129 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:05.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:05:05.819 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:06.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 -- 172.21.15.59:0/3772218038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2188074f40 msgr2=0x7f218810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 --2- 172.21.15.59:0/3772218038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2188074f40 0x7f218810de80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2174009900 tx=0x7f217402a480 comp rx=0 tx=0).stop 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 -- 172.21.15.59:0/3772218038 shutdown_connections 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 --2- 172.21.15.59:0/3772218038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f218810e3c0 0x7f2188110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 --2- 172.21.15.59:0/3772218038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2188074f40 0x7f218810de80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 --2- 172.21.15.59:0/3772218038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f2188074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 -- 172.21.15.59:0/3772218038 >> 172.21.15.59:0/3772218038 conn(0x7f21880fd8c0 msgr2=0x7f21880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 -- 172.21.15.59:0/3772218038 shutdown_connections 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.354+0000 7f218fe00640 1 -- 172.21.15.59:0/3772218038 wait complete. 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.355+0000 7f218fe00640 1 Processor -- start 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.355+0000 7f218fe00640 1 -- start start 2024-09-05T12:05:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f218819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2188074f40 0x7f218819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 0x7f218819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21881141b0 con 0x7f218810e3c0 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2188114030 con 0x7f2188074620 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2188114330 con 0x7f2188074f40 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 0x7f218819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f218819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 0x7f218819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35604/0 (socket says 172.21.15.59:35604) 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218de00640 1 -- 172.21.15.59:0/4231583902 learned_addr learned my addr 172.21.15.59:0/4231583902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218ca00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2188074f40 0x7f218819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218d400640 1 -- 172.21.15.59:0/4231583902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2188074f40 msgr2=0x7f218819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.356+0000 7f218d400640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2188074f40 0x7f218819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218d400640 1 -- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 msgr2=0x7f218819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218d400640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 0x7f218819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218d400640 1 -- 172.21.15.59:0/4231583902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218819f620 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218d400640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f218819a670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f217c00e9f0 tx=0x7f217c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f2186000640 1 -- 172.21.15.59:0/4231583902 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c00ccc0 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f2186000640 1 -- 172.21.15.59:0/4231583902 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f217c010910 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f2186000640 1 -- 172.21.15.59:0/4231583902 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c018960 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218819f910 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.357+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881a71f0 con 0x7f2188074620 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.360+0000 7f2192878740 1 Processor -- start 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.360+0000 7f2192878740 1 -- start start 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2186000640 1 -- 172.21.15.59:0/4231583902 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f217c010ac0 con 0x7f2188074620 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d770160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56553d6c93e0 con 0x56553d762b90 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56553d670d80 con 0x56553d7634b0 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f2192878740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56553d6c53a0 con 0x56553d76dd70 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35612/0 (socket says 172.21.15.59:35612) 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f218de00640 1 -- 172.21.15.59:0/1401636636 learned_addr learned my addr 172.21.15.59:0/1401636636 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f218ca00640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d770160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.361+0000 7f218d400640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 -- 172.21.15.59:0/1401636636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 msgr2=0x56553d770160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d770160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 -- 172.21.15.59:0/1401636636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 msgr2=0x56553d76d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 -- 172.21.15.59:0/1401636636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553d68fae0 con 0x56553d762b90 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218d400640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f218de00640 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2180011470 tx=0x7f2180011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ddb7bb271f2881 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f217a600640 1 -- 172.21.15.59:0/1401636636 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004b20 con 0x56553d762b90 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f217a600640 1 -- 172.21.15.59:0/1401636636 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2180004cc0 con 0x56553d762b90 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f217a600640 1 -- 172.21.15.59:0/1401636636 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800053e0 con 0x56553d762b90 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.362+0000 7f2186000640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 0x7f21700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2186000640 1 -- 172.21.15.59:0/4231583902 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f217c0cf840 con 0x7f2188074620 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 -- 172.21.15.59:0/1401636636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 msgr2=0x56553d762f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2180011470 tx=0x7f2180011930 comp rx=0 tx=0).stop 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f218d400640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 0x7f21700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 -- 172.21.15.59:0/1401636636 shutdown_connections 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d770160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 --2- 172.21.15.59:0/1401636636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d762f70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 -- 172.21.15.59:0/1401636636 >> 172.21.15.59:0/1401636636 conn(0x56553d75f120 msgr2=0x56553d75f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 -- 172.21.15.59:0/1401636636 shutdown_connections 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.363+0000 7f2192878740 1 -- 172.21.15.59:0/1401636636 wait complete. 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f218d400640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 0x7f21700a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f217c098140 tx=0x7f217c009700 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 -- start start 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 Processor -- start 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d762b90 0x56553d76ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56553d7709e0 con 0x56553d76dd70 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56553d6840f0 con 0x56553d7634b0 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56553d770b60 con 0x56553d762b90 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.364+0000 7f2192878740 10 client.0 _ll_register_callbacks cb 0x56553d6759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d762b90 0x56553d76ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35626/0 (socket says 172.21.15.59:35626) 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218de00640 1 -- 172.21.15.59:0/787413540 learned_addr learned my addr 172.21.15.59:0/787413540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 -- 172.21.15.59:0/787413540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d762b90 msgr2=0x56553d76ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d762b90 0x56553d76ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 -- 172.21.15.59:0/787413540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 msgr2=0x56553d76b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218ca00640 1 -- 172.21.15.59:0/787413540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553d43a3c0 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.365+0000 7f218de00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f218ca00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76b040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f217402aa70 tx=0x7f2174004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a057b20f3b1cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f216ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174035040 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2180004cc0 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2174005450 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f217404f050 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f2192878740 10 client.26441 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56553d77c580 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.366+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2180004cc0 con 0x56553d7634b0 2024-09-05T12:05:06.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2174003790 con 0x56553d7634b0 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 10 client.26441 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 20 client.26441 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d77d990 0x56553d77fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 4 client.26441 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56553d6d6950 con 0x56553d77d990 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.367+0000 7f218de00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d77d990 0x56553d77fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.368+0000 7f218de00640 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d77d990 0x56553d77fdb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f720e84e466721d2 server_cookie=267e6d655286f8fa in_seq=0 out_seq=0 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.368+0000 7f216ea00640 10 client.26441 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.368+0000 7f216ea00640 1 -- 172.21.15.59:0/787413540 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56553d6d6950 con 0x56553d77d990 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.368+0000 7f216ea00640 10 client.26441 handle_command_reply: tid=0 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.368+0000 7f2192878740 1 client.26441 shutdown 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 msgr2=0x56553d76b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76b040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f217402aa70 tx=0x7f2174004390 comp rx=0 tx=0).stop 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 shutdown_connections 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d77d990 0x56553d77fdb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d76dd70 0x56553d76b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76b040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 --2- 172.21.15.59:0/787413540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d762b90 0x56553d76ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f216ea00640 0 client.26441 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 >> 172.21.15.59:0/787413540 conn(0x56553d75f120 msgr2=0x56553d764e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 shutdown_connections 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 1 -- 172.21.15.59:0/787413540 wait complete. 2024-09-05T12:05:06.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.369+0000 7f2192878740 20 client.26441 trim_cache size 0 max 16384 2024-09-05T12:05:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.387+0000 7f2192878740 1 Processor -- start 2024-09-05T12:05:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.387+0000 7f2192878740 1 -- start start 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d785680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 0x56553d785bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 0x56553d76f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21881af140 con 0x56553d762b90 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b19c0 con 0x56553d76dd70 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f2192878740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21881af7b0 con 0x56553d7634b0 2024-09-05T12:05:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d785680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d785680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35642/0 (socket says 172.21.15.59:35642) 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218ca00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 0x56553d785bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218de00640 1 -- 172.21.15.59:0/602955727 learned_addr learned my addr 172.21.15.59:0/602955727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218de00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 0x56553d76f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.388+0000 7f218ca00640 1 -- 172.21.15.59:0/602955727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 msgr2=0x56553d76f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218ca00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 0x56553d76f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218ca00640 1 -- 172.21.15.59:0/602955727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 msgr2=0x56553d785680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218ca00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d785680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218ca00640 1 -- 172.21.15.59:0/602955727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56553d769050 con 0x56553d7634b0 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218de00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 0x56553d76f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:05:06.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f218ca00640 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 0x56553d785bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f2174005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c7954555168258 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f2179c00640 1 -- 172.21.15.59:0/602955727 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174059070 con 0x56553d7634b0 2024-09-05T12:05:06.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f2179c00640 1 -- 172.21.15.59:0/602955727 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2174041070 con 0x56553d7634b0 2024-09-05T12:05:06.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.389+0000 7f2179c00640 1 -- 172.21.15.59:0/602955727 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174035070 con 0x56553d7634b0 2024-09-05T12:05:06.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 -- 172.21.15.59:0/602955727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 msgr2=0x56553d785bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 0x56553d785bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f2174005a20 comp rx=0 tx=0).stop 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 -- 172.21.15.59:0/602955727 shutdown_connections 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d76dd70 0x56553d76f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d7634b0 0x56553d785bc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 --2- 172.21.15.59:0/602955727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d785680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 -- 172.21.15.59:0/602955727 >> 172.21.15.59:0/602955727 conn(0x56553d75f120 msgr2=0x56553d75e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 -- 172.21.15.59:0/602955727 shutdown_connections 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.390+0000 7f2192878740 1 -- 172.21.15.59:0/602955727 wait complete. 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 -- start start 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 Processor -- start 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d76ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d76f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21881afe80 con 0x56553d762b90 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b12f0 con 0x56553d7634b0 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b0550 con 0x56553d76dd70 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.391+0000 7f2192878740 10 client.0 _ll_register_callbacks cb 0x7f2174005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d76ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d76ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35658/0 (socket says 172.21.15.59:35658) 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218de00640 1 -- 172.21.15.59:0/3465115564 learned_addr learned my addr 172.21.15.59:0/3465115564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218de00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d76f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 -- 172.21.15.59:0/3465115564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 msgr2=0x56553d76f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d76f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 -- 172.21.15.59:0/3465115564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 msgr2=0x56553d76ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d76ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 -- 172.21.15.59:0/3465115564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56553d774180 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218de00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d76f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f218ca00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76f280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56553d66bcf0 tx=0x7f2174005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9223ba4886a953e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.392+0000 7f216ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174059070 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2174003800 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2174041070 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2174035040 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 10 client.26444 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56553d7e87d0 con 0x56553d7634b0 2024-09-05T12:05:06.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2174041070 con 0x56553d7634b0 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f217405c020 con 0x56553d7634b0 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 10 client.26444 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.393+0000 7f2192878740 20 client.26444 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d7e8ed0 0x56553d7eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f2192878740 4 client.26444 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56553d6d0e60 con 0x56553d7e8ed0 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f218de00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d7e8ed0 0x56553d7eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f218de00640 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d7e8ed0 0x56553d7eb2c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fdfd9cb43298044 server_cookie=96f957fd3c1c0af in_seq=0 out_seq=0 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f216ea00640 10 client.26444 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f216ea00640 1 -- 172.21.15.59:0/3465115564 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56553d6d0e60 con 0x56553d7e8ed0 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f216ea00640 10 client.26444 handle_command_reply: tid=0 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.394+0000 7f2192878740 1 client.26444 shutdown 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 msgr2=0x56553d76f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76f280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56553d66bcf0 tx=0x7f2174005b50 comp rx=0 tx=0).stop 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 shutdown_connections 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56553d76dd70 0x56553d76f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56553d7634b0 0x56553d76f280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56553d7e8ed0 0x56553d7eb2c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 --2- 172.21.15.59:0/3465115564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56553d762b90 0x56553d76ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 >> 172.21.15.59:0/3465115564 conn(0x56553d75f120 msgr2=0x56553d7645e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f216ea00640 0 client.26444 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 shutdown_connections 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 1 -- 172.21.15.59:0/3465115564 wait complete. 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.395+0000 7f2192878740 20 client.26444 trim_cache size 0 max 16384 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:05:06.401 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:06.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 msgr2=0x7f21700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 0x7f21700a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f217c098140 tx=0x7f217c009700 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 msgr2=0x7f218819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f218819a670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f217c00e9f0 tx=0x7f217c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 shutdown_connections 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f21700a2b80 0x7f21700a5040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218810e3c0 0x7f218819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2188074f40 0x7f218819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 --2- 172.21.15.59:0/4231583902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2188074620 0x7f218819a670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 >> 172.21.15.59:0/4231583902 conn(0x7f21880fd8c0 msgr2=0x7f21880fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 shutdown_connections 2024-09-05T12:05:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:06.396+0000 7f218fe00640 1 -- 172.21.15.59:0/4231583902 wait complete. 2024-09-05T12:05:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:06 smithi189 ceph-mon[34772]: pgmap v617: 129 pgs: 129 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:06 smithi089 ceph-mon[34499]: pgmap v617: 129 pgs: 129 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:06 smithi059 ceph-mon[30025]: pgmap v617: 129 pgs: 129 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:07.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:05:07.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3358580607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc0108f60 msgr2=0x7f0bc0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3358580607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc0108f60 0x7f0bc0110ae0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009ca0 tx=0x7f0bb802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3358580607 shutdown_connections 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3358580607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc0108f60 0x7f0bc0110ae0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3358580607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc01047e0 0x7f0bc0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3358580607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3358580607 >> 172.21.15.59:0/3358580607 conn(0x7f0bc00ff910 msgr2=0x7f0bc0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:08.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3358580607 shutdown_connections 2024-09-05T12:05:08.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.195+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3358580607 wait complete. 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.196+0000 7f0bc4c00640 1 Processor -- start 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.196+0000 7f0bc4c00640 1 -- start start 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc01047e0 0x7f0bc019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc0108f60 0x7f0bc019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0113fc0 con 0x7f0bc01047e0 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0113e40 con 0x7f0bc0103e30 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bc4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0114140 con 0x7f0bc0108f60 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bb6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc01047e0 0x7f0bc019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bbf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bbf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46856/0 (socket says 172.21.15.59:46856) 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bbfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc0108f60 0x7f0bc019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bbf400640 1 -- 172.21.15.59:0/3442333404 learned_addr learned my addr 172.21.15.59:0/3442333404 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.197+0000 7f0bbf400640 1 -- 172.21.15.59:0/3442333404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc0108f60 msgr2=0x7f0bc019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbf400640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc0108f60 0x7f0bc019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbf400640 1 -- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc01047e0 msgr2=0x7f0bc019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbf400640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc01047e0 0x7f0bc019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbf400640 1 -- 172.21.15.59:0/3442333404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc019f3f0 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbf400640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9f0 tx=0x7f0bb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000cd80 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc019f6e0 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0010910 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00188f0 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.198+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a6fc0 con 0x7f0bc0103e30 2024-09-05T12:05:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.199+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0105310 con 0x7f0bc0103e30 2024-09-05T12:05:08.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.201+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0bb0010ac0 con 0x7f0bc0103e30 2024-09-05T12:05:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.203+0000 7f0bbcc00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 0x7f0ba40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.203+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0bb00cfc40 con 0x7f0bc0103e30 2024-09-05T12:05:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.203+0000 7f0bb6a00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 0x7f0ba40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.204+0000 7f0bb6a00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 0x7f0ba40a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0100180 tx=0x7f0ba8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.205+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bb009c0e0 con 0x7f0bc0103e30 2024-09-05T12:05:08.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.429+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bc010c140 con 0x7f0bc0103e30 2024-09-05T12:05:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.430+0000 7f0bbcc00640 1 -- 172.21.15.59:0/3442333404 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f0bb00a03e0 con 0x7f0bc0103e30 2024-09-05T12:05:08.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:08.432 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:08.433 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 msgr2=0x7f0ba40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 0x7f0ba40a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0100180 tx=0x7f0ba8005f90 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 msgr2=0x7f0bc019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9f0 tx=0x7f0bb000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 shutdown_connections 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0ba40a2b80 0x7f0ba40a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc0108f60 0x7f0bc019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc01047e0 0x7f0bc019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 --2- 172.21.15.59:0/3442333404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc0103e30 0x7f0bc019a440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:08.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.433+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 >> 172.21.15.59:0/3442333404 conn(0x7f0bc00ff910 msgr2=0x7f0bc0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:08.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.434+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 shutdown_connections 2024-09-05T12:05:08.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:08.434+0000 7f0bc4c00640 1 -- 172.21.15.59:0/3442333404 wait complete. 2024-09-05T12:05:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:08 smithi189 ceph-mon[34772]: pgmap v618: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:08 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3442333404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:09.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:08 smithi059 ceph-mon[30025]: pgmap v618: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:09.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3442333404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:09.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:05:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:08 smithi089 ceph-mon[34499]: pgmap v618: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:08 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3442333404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:09.404 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:05:09.404 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-09-05T12:05:09.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 -- 172.21.15.59:0/4029659201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce41049c0 msgr2=0x7fdce4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 --2- 172.21.15.59:0/4029659201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce41049c0 0x7fdce4108a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 -- 172.21.15.59:0/4029659201 shutdown_connections 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 --2- 172.21.15.59:0/4029659201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4109140 0x7fdce4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 --2- 172.21.15.59:0/4029659201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce41049c0 0x7fdce4108a10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 --2- 172.21.15.59:0/4029659201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce4104010 0x7fdce41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 -- 172.21.15.59:0/4029659201 >> 172.21.15.59:0/4029659201 conn(0x7fdce40ffa50 msgr2=0x7fdce4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 -- 172.21.15.59:0/4029659201 shutdown_connections 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.153+0000 7fdcec400640 1 -- 172.21.15.59:0/4029659201 wait complete. 2024-09-05T12:05:10.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 Processor -- start 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 -- start start 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 0x7fdce419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 0x7fdce419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce4109140 0x7fdce419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4114140 con 0x7fdce4109140 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4113fc0 con 0x7fdce4104010 2024-09-05T12:05:10.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.154+0000 7fdcec400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41142c0 con 0x7fdce41049c0 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 0x7fdce419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 0x7fdce419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdcea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce4109140 0x7fdce419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 0x7fdce419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34176/0 (socket says 172.21.15.59:34176) 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9000640 1 -- 172.21.15.59:0/4048788090 learned_addr learned my addr 172.21.15.59:0/4048788090 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:10.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 -- 172.21.15.59:0/4048788090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 msgr2=0x7fdce419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 0x7fdce419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 -- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce4109140 msgr2=0x7fdce419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce4109140 0x7fdce419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 -- 172.21.15.59:0/4048788090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce419f660 con 0x7fdce4104010 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.155+0000 7fdce9a00640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 0x7fdce419a6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800e9f0 tx=0x7fdcd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.156+0000 7fdce2000640 1 -- 172.21.15.59:0/4048788090 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd800ccc0 con 0x7fdce4104010 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.156+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce419f950 con 0x7fdce4104010 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.156+0000 7fdce2000640 1 -- 172.21.15.59:0/4048788090 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd8010910 con 0x7fdce4104010 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.156+0000 7fdce2000640 1 -- 172.21.15.59:0/4048788090 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8018960 con 0x7fdce4104010 2024-09-05T12:05:10.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.156+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a7230 con 0x7fdce4104010 2024-09-05T12:05:10.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.159+0000 7fdce2000640 1 -- 172.21.15.59:0/4048788090 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdcd8010ac0 con 0x7fdce4104010 2024-09-05T12:05:10.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.160+0000 7fdceee87740 1 Processor -- start 2024-09-05T12:05:10.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdce2000640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 0x7fdccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 -- start start 2024-09-05T12:05:10.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdce2000640 1 -- 172.21.15.59:0/4048788090 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdcd80cf840 con 0x7fdce4104010 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdcea400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 0x7fdccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5596d0b7a3e0 con 0x5596d0c13c70 2024-09-05T12:05:10.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5596d0b21d80 con 0x5596d0c1ee50 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdceee87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5596d0b763a0 con 0x5596d0c14590 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.161+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46880/0 (socket says 172.21.15.59:46880) 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 -- 172.21.15.59:0/1587376027 learned_addr learned my addr 172.21.15.59:0/1587376027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35708/0 (socket says 172.21.15.59:35708) 2024-09-05T12:05:10.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdce9000640 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 -- 172.21.15.59:0/1587376027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 msgr2=0x5596d0c1e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 -- 172.21.15.59:0/1587376027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 msgr2=0x5596d0c14050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 -- 172.21.15.59:0/1587376027 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d0b40ae0 con 0x5596d0c1ee50 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdce9a00640 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.162+0000 7fdcea400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 0x7fdccc0a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009fd0 tx=0x7fdcdc013b80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.163+0000 7fdcea400640 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00f790 tx=0x7fdcdc01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293ce4a4949af2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.163+0000 7fdcd6a00640 1 -- 172.21.15.59:0/1587376027 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004050 con 0x5596d0c1ee50 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.163+0000 7fdcd6a00640 1 -- 172.21.15.59:0/1587376027 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc0046d0 con 0x5596d0c1ee50 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.163+0000 7fdcd6a00640 1 -- 172.21.15.59:0/1587376027 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc020bd0 con 0x5596d0c1ee50 2024-09-05T12:05:10.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- 172.21.15.59:0/1587376027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 msgr2=0x5596d0c21240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00f790 tx=0x7fdcdc01f000 comp rx=0 tx=0).stop 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- 172.21.15.59:0/1587376027 shutdown_connections 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c21240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 --2- 172.21.15.59:0/1587376027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- 172.21.15.59:0/1587376027 >> 172.21.15.59:0/1587376027 conn(0x5596d0c10200 msgr2=0x5596d0c105e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- 172.21.15.59:0/1587376027 shutdown_connections 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- 172.21.15.59:0/1587376027 wait complete. 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.164+0000 7fdceee87740 1 -- start start 2024-09-05T12:05:10.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 Processor -- start 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5596d0c219c0 con 0x5596d0c13c70 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5596d0b350f0 con 0x5596d0c1ee50 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5596d0c21b40 con 0x5596d0c14590 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdceee87740 10 client.0 _ll_register_callbacks cb 0x5596d0b1cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:10.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.165+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35712/0 (socket says 172.21.15.59:35712) 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 -- 172.21.15.59:0/1701611996 learned_addr learned my addr 172.21.15.59:0/1701611996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdcea400640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9000640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 -- 172.21.15.59:0/1701611996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 msgr2=0x5596d0c1c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 -- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 msgr2=0x5596d0c1c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c1c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 -- 172.21.15.59:0/1701611996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d08eb3c0 con 0x5596d0c13c70 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdce9a00640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80986b0 tx=0x7fdcd80a48a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe280749f3669c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdccb400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.166+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd809b130 con 0x5596d0c13c70 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdcd809b2d0 con 0x5596d0c13c70 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc0046d0 con 0x5596d0c13c70 2024-09-05T12:05:10.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdcd807ab50 con 0x5596d0c13c70 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdceee87740 10 client.16464 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5596d0c2db40 con 0x5596d0c13c70 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.167+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc0046d0 con 0x5596d0c13c70 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdcd809aba0 con 0x5596d0c13c70 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 10 client.16464 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 20 client.16464 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c2ee90 0x5596d0c312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 4 client.16464 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:10.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d0b87950 con 0x5596d0c2ee90 2024-09-05T12:05:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.168+0000 7fdce9000640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c2ee90 0x5596d0c312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.169+0000 7fdce9000640 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c2ee90 0x5596d0c312a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c5113cbacbf62c server_cookie=9da8f3e107a50f4d in_seq=0 out_seq=0 2024-09-05T12:05:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.169+0000 7fdccb400640 10 client.16464 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdccb400640 1 -- 172.21.15.59:0/1701611996 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596d0b87950 con 0x5596d0c2ee90 2024-09-05T12:05:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdccb400640 10 client.16464 handle_command_reply: tid=0 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 client.16464 shutdown 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 msgr2=0x5596d0c1be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80986b0 tx=0x7fdcd80a48a0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 shutdown_connections 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c1c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c14590 0x5596d0c1c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c2ee90 0x5596d0c312a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 --2- 172.21.15.59:0/1701611996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c13c70 0x5596d0c1be10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdccb400640 0 client.16464 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 >> 172.21.15.59:0/1701611996 conn(0x5596d0c10200 msgr2=0x5596d0c15f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 shutdown_connections 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.170+0000 7fdceee87740 1 -- 172.21.15.59:0/1701611996 wait complete. 2024-09-05T12:05:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.171+0000 7fdceee87740 20 client.16464 trim_cache size 0 max 16384 2024-09-05T12:05:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.184+0000 7fdceee87740 1 Processor -- start 2024-09-05T12:05:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.184+0000 7fdceee87740 1 -- start start 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c14590 0x5596d0c370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c1ee50 0x5596d0c19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41af050 con 0x5596d0c1ee50 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b18d0 con 0x5596d0c14590 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdceee87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41af6c0 con 0x5596d0c13c70 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34196/0 (socket says 172.21.15.59:34196) 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdce9a00640 1 -- 172.21.15.59:0/2050592751 learned_addr learned my addr 172.21.15.59:0/2050592751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdce9000640 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c14590 0x5596d0c370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.185+0000 7fdcea400640 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c1ee50 0x5596d0c19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 -- 172.21.15.59:0/2050592751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c14590 msgr2=0x5596d0c370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c14590 0x5596d0c370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 -- 172.21.15.59:0/2050592751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c1ee50 msgr2=0x5596d0c19070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c1ee50 0x5596d0c19070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 -- 172.21.15.59:0/2050592751 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d0c318f0 con 0x5596d0c13c70 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdce9a00640 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8098660 tx=0x7fdcd80a47d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beca6612b8c00b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdcd6000640 1 -- 172.21.15.59:0/2050592751 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80a12e0 con 0x5596d0c13c70 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdcd6000640 1 -- 172.21.15.59:0/2050592751 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd80a1480 con 0x5596d0c13c70 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.186+0000 7fdcd6000640 1 -- 172.21.15.59:0/2050592751 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd809b390 con 0x5596d0c13c70 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 -- 172.21.15.59:0/2050592751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 msgr2=0x5596d0c36b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8098660 tx=0x7fdcd80a47d0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 -- 172.21.15.59:0/2050592751 shutdown_connections 2024-09-05T12:05:10.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c1ee50 0x5596d0c19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c14590 0x5596d0c370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 --2- 172.21.15.59:0/2050592751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c36b90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.187+0000 7fdceee87740 1 -- 172.21.15.59:0/2050592751 >> 172.21.15.59:0/2050592751 conn(0x5596d0c10200 msgr2=0x5596d0c1c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.188+0000 7fdceee87740 1 -- 172.21.15.59:0/2050592751 shutdown_connections 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.188+0000 7fdceee87740 1 -- 172.21.15.59:0/2050592751 wait complete. 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.188+0000 7fdceee87740 1 -- start start 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.188+0000 7fdceee87740 1 Processor -- start 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.188+0000 7fdceee87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c14590 0x5596d0c23110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41afd90 con 0x5596d0c14590 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b1200 con 0x5596d0c1ee50 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b0460 con 0x5596d0c13c70 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdceee87740 10 client.0 _ll_register_callbacks cb 0x7fdcd8098010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:34198/0 (socket says 172.21.15.59:34198) 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 -- 172.21.15.59:0/3815053262 learned_addr learned my addr 172.21.15.59:0/3815053262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdcea400640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9000640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c14590 0x5596d0c23110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 -- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 msgr2=0x5596d0c23650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c23650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 -- 172.21.15.59:0/3815053262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c14590 msgr2=0x5596d0c23110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c14590 0x5596d0c23110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.189+0000 7fdce9a00640 1 -- 172.21.15.59:0/3815053262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d0c98c10 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdce9a00640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcd809b9e0 tx=0x7fdcd809add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5fd5df3427fe7f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdccb400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80a12e0 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd80a1480 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdcd8010430 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdcd80a4620 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdceee87740 10 client.26404 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.190+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5596d0c98c10 con 0x5596d0c13c70 2024-09-05T12:05:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.191+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd80a1480 con 0x5596d0c13c70 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.191+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdcd8081070 con 0x5596d0c13c70 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.191+0000 7fdceee87740 10 client.26404 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.191+0000 7fdceee87740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.191+0000 7fdceee87740 20 client.26404 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c9a150 0x5596d0c9c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdceee87740 4 client.26404 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596d0b81e60 con 0x5596d0c9a150 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdce9000640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c9a150 0x5596d0c9c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdce9000640 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c9a150 0x5596d0c9c540 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6433985adde208af server_cookie=e78cb6086fd6b79b in_seq=0 out_seq=0 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.192+0000 7fdccb400640 10 client.26404 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdccb400640 1 -- 172.21.15.59:0/3815053262 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596d0b81e60 con 0x5596d0c9a150 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdccb400640 10 client.26404 handle_command_reply: tid=0 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 client.26404 shutdown 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 msgr2=0x5596d0c22bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcd809b9e0 tx=0x7fdcd809add0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 shutdown_connections 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596d0c1ee50 0x5596d0c23650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5596d0c9a150 0x5596d0c9c540 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5596d0c14590 0x5596d0c23110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 --2- 172.21.15.59:0/3815053262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596d0c13c70 0x5596d0c22bd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 >> 172.21.15.59:0/3815053262 conn(0x5596d0c10200 msgr2=0x5596d0c1a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdccb400640 0 client.26404 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 shutdown_connections 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.193+0000 7fdceee87740 1 -- 172.21.15.59:0/3815053262 wait complete. 2024-09-05T12:05:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.194+0000 7fdceee87740 20 client.26404 trim_cache size 0 max 16384 2024-09-05T12:05:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:05:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.194+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 msgr2=0x7fdccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.194+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 0x7fdccc0a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009fd0 tx=0x7fdcdc013b80 comp rx=0 tx=0).stop 2024-09-05T12:05:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 msgr2=0x7fdce419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 0x7fdce419a6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800e9f0 tx=0x7fdcd800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 shutdown_connections 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdccc0a2b80 0x7fdccc0a5040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdce4109140 0x7fdce419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdce41049c0 0x7fdce419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 --2- 172.21.15.59:0/4048788090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdce4104010 0x7fdce419a6b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 >> 172.21.15.59:0/4048788090 conn(0x7fdce40ffa50 msgr2=0x7fdce4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 shutdown_connections 2024-09-05T12:05:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:10.195+0000 7fdcec400640 1 -- 172.21.15.59:0/4048788090 wait complete. 2024-09-05T12:05:10.217 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:11.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:10 smithi059 ceph-mon[30025]: pgmap v619: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:10 smithi089 ceph-mon[34499]: pgmap v619: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:11.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:10 smithi189 ceph-mon[34772]: pgmap v619: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 -- 172.21.15.59:0/443778093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec1047e0 msgr2=0x7f8eec108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 --2- 172.21.15.59:0/443778093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec1047e0 0x7f8eec108830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009900 tx=0x7f8edc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 -- 172.21.15.59:0/443778093 shutdown_connections 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 --2- 172.21.15.59:0/443778093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec108f60 0x7f8eec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 --2- 172.21.15.59:0/443778093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec1047e0 0x7f8eec108830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 --2- 172.21.15.59:0/443778093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec103e30 0x7f8eec104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.547+0000 7f8ef3200640 1 -- 172.21.15.59:0/443778093 >> 172.21.15.59:0/443778093 conn(0x7f8eec0ff8d0 msgr2=0x7f8eec101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.548+0000 7f8ef3200640 1 -- 172.21.15.59:0/443778093 shutdown_connections 2024-09-05T12:05:11.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.548+0000 7f8ef3200640 1 -- 172.21.15.59:0/443778093 wait complete. 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 Processor -- start 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 -- start start 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 0x7f8eec19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 0x7f8eec19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec108f60 0x7f8eec19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec113fd0 con 0x7f8eec1047e0 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec113e50 con 0x7f8eec103e30 2024-09-05T12:05:11.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec114150 con 0x7f8eec108f60 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.549+0000 7f8ef1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 0x7f8eec19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 0x7f8eec19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 0x7f8eec19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38514/0 (socket says 172.21.15.59:38514) 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1400640 1 -- 172.21.15.59:0/930186368 learned_addr learned my addr 172.21.15.59:0/930186368 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef2800640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec108f60 0x7f8eec19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 -- 172.21.15.59:0/930186368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec108f60 msgr2=0x7f8eec19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec108f60 0x7f8eec19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 -- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 msgr2=0x7f8eec19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 0x7f8eec19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 -- 172.21.15.59:0/930186368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec19f440 con 0x7f8eec103e30 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.550+0000 7f8ef1e00640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 0x7f8eec19a490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e9f0 tx=0x7f8ee000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.551+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000cd80 con 0x7f8eec103e30 2024-09-05T12:05:11.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.551+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee00035f0 con 0x7f8eec103e30 2024-09-05T12:05:11.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.551+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec19f730 con 0x7f8eec103e30 2024-09-05T12:05:11.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.551+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0010690 con 0x7f8eec103e30 2024-09-05T12:05:11.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.551+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1a7010 con 0x7f8eec103e30 2024-09-05T12:05:11.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.552+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec104210 con 0x7f8eec103e30 2024-09-05T12:05:11.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.554+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ee0003b30 con 0x7f8eec103e30 2024-09-05T12:05:11.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.556+0000 7f8ee6a00640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 0x7f8ed40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:11.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.556+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8ee00cf6c0 con 0x7f8eec103e30 2024-09-05T12:05:11.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.557+0000 7f8ef1400640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 0x7f8ed40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.558+0000 7f8ef1400640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 0x7f8ed40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0096c0 tx=0x7f8edc0046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.558+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ee009c1b0 con 0x7f8eec103e30 2024-09-05T12:05:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.780+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eec19b7f0 con 0x7f8eec103e30 2024-09-05T12:05:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.782+0000 7f8ee6a00640 1 -- 172.21.15.59:0/930186368 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f8ee009bab0 con 0x7f8eec103e30 2024-09-05T12:05:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:11.784 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:11.784 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:11.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.784+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 msgr2=0x7f8ed40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:11.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.784+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 0x7f8ed40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0096c0 tx=0x7f8edc0046c0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.784+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 msgr2=0x7f8eec19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.784+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 0x7f8eec19a490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e9f0 tx=0x7f8ee000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 shutdown_connections 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8ed40a2c00 0x7f8ed40a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8eec108f60 0x7f8eec19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8eec1047e0 0x7f8eec19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 --2- 172.21.15.59:0/930186368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eec103e30 0x7f8eec19a490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 >> 172.21.15.59:0/930186368 conn(0x7f8eec0ff8d0 msgr2=0x7f8eec100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 shutdown_connections 2024-09-05T12:05:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:11.785+0000 7f8ef3200640 1 -- 172.21.15.59:0/930186368 wait complete. 2024-09-05T12:05:11.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:05:11.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:05:12.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:11 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/930186368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:12.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:11 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/930186368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:12.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:11 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/930186368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:12.518 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-09-05T12:05:12.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-09-05T12:05:12.658 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:13.142 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:12 smithi059 ceph-mon[30025]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:05:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:12 smithi089 ceph-mon[34499]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 -- 172.21.15.59:0/1837303363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec1079c0 msgr2=0x7fdfec107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 --2- 172.21.15.59:0/1837303363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec1079c0 0x7fdfec107da0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009930 tx=0x7fdfe002a420 comp rx=0 tx=0).stop 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 -- 172.21.15.59:0/1837303363 shutdown_connections 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 --2- 172.21.15.59:0/1837303363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec102360 0x7fdfec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 --2- 172.21.15.59:0/1837303363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 0x7fdfec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 --2- 172.21.15.59:0/1837303363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec1079c0 0x7fdfec107da0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 -- 172.21.15.59:0/1837303363 >> 172.21.15.59:0/1837303363 conn(0x7fdfec0fd740 msgr2=0x7fdfec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 -- 172.21.15.59:0/1837303363 shutdown_connections 2024-09-05T12:05:13.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.259+0000 7fdff2400640 1 -- 172.21.15.59:0/1837303363 wait complete. 2024-09-05T12:05:13.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.260+0000 7fdff2400640 1 Processor -- start 2024-09-05T12:05:13.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 -- start start 2024-09-05T12:05:13.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 0x7fdfec19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec1079c0 0x7fdfec19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec111e10 con 0x7fdfec102360 2024-09-05T12:05:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec111c90 con 0x7fdfec1019c0 2024-09-05T12:05:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdff2400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec111f90 con 0x7fdfec1079c0 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38542/0 (socket says 172.21.15.59:38542) 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.261+0000 7fdfebe00640 1 -- 172.21.15.59:0/650294830 learned_addr learned my addr 172.21.15.59:0/650294830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdff1a00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec1079c0 0x7fdfec19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdff1000640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 0x7fdfec19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 -- 172.21.15.59:0/650294830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec1079c0 msgr2=0x7fdfec19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec1079c0 0x7fdfec19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 -- 172.21.15.59:0/650294830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 msgr2=0x7fdfec19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 0x7fdfec19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 -- 172.21.15.59:0/650294830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec19f400 con 0x7fdfec102360 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.262+0000 7fdfebe00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.263+0000 7fdfe9600640 1 -- 172.21.15.59:0/650294830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800cd80 con 0x7fdfec102360 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.263+0000 7fdfe9600640 1 -- 172.21.15.59:0/650294830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd8010910 con 0x7fdfec102360 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.263+0000 7fdfe9600640 1 -- 172.21.15.59:0/650294830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8018990 con 0x7fdfec102360 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.263+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec19f6f0 con 0x7fdfec102360 2024-09-05T12:05:13.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.263+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1a6fd0 con 0x7fdfec102360 2024-09-05T12:05:13.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.266+0000 7fdfe9600640 1 -- 172.21.15.59:0/650294830 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfd8010ac0 con 0x7fdfec102360 2024-09-05T12:05:13.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.266+0000 7fdff663b740 1 Processor -- start 2024-09-05T12:05:13.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.266+0000 7fdff663b740 1 -- start start 2024-09-05T12:05:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 0x55da57e2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e34950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 0x55da57e37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da57d90290 con 0x55da57e29cb0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55da57d37b40 con 0x55da57e34e90 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff663b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da57d8c250 con 0x55da57e2a5d0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 0x55da57e2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 0x55da57e2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38552/0 (socket says 172.21.15.59:38552) 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e34950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdff1a00640 1 -- 172.21.15.59:0/540975325 learned_addr learned my addr 172.21.15.59:0/540975325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.267+0000 7fdfebe00640 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 0x55da57e37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 -- 172.21.15.59:0/540975325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 msgr2=0x55da57e37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 0x55da57e37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 -- 172.21.15.59:0/540975325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 msgr2=0x55da57e2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 0x55da57e2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 -- 172.21.15.59:0/540975325 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da57b016a0 con 0x55da57e2a5d0 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfebe00640 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 0x55da57e37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdff1000640 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e34950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002a940 tx=0x7fdfe0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab32859c570c997 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfe9600640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 0x7fdfd40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfde000640 1 -- 172.21.15.59:0/540975325 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0035040 con 0x55da57e2a5d0 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfde000640 1 -- 172.21.15.59:0/540975325 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe0005520 con 0x55da57e2a5d0 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfde000640 1 -- 172.21.15.59:0/540975325 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00035f0 con 0x55da57e2a5d0 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.268+0000 7fdfe9600640 1 -- 172.21.15.59:0/650294830 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdfd80cf720 con 0x7fdfec102360 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.269+0000 7fdff1a00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 0x7fdfd40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.269+0000 7fdff663b740 1 -- 172.21.15.59:0/540975325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 msgr2=0x55da57e34950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.269+0000 7fdff1a00640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 0x7fdfd40a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4011630 tx=0x7fdfe4009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.269+0000 7fdff663b740 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e34950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002a940 tx=0x7fdfe0005de0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 -- 172.21.15.59:0/540975325 shutdown_connections 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e34e90 0x55da57e37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e34950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 --2- 172.21.15.59:0/540975325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e29cb0 0x55da57e2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 -- 172.21.15.59:0/540975325 >> 172.21.15.59:0/540975325 conn(0x55da57e26240 msgr2=0x55da57e26620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 -- 172.21.15.59:0/540975325 shutdown_connections 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 -- 172.21.15.59:0/540975325 wait complete. 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.270+0000 7fdff663b740 1 -- start start 2024-09-05T12:05:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 Processor -- start 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e34e90 0x55da57e32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55da57e37aa0 con 0x55da57e34e90 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55da57d4afa0 con 0x55da57e29cb0 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.271+0000 7fdff663b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da57e37c20 con 0x55da57e2a5d0 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff663b740 10 client.0 _ll_register_callbacks cb 0x55da57a9eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59162/0 (socket says 172.21.15.59:59162) 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 -- 172.21.15.59:0/1739673705 learned_addr learned my addr 172.21.15.59:0/1739673705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdfebe00640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1a00640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e34e90 0x55da57e32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 -- 172.21.15.59:0/1739673705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 msgr2=0x55da57e31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 -- 172.21.15.59:0/1739673705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e34e90 msgr2=0x55da57e32000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e34e90 0x55da57e32000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.272+0000 7fdff1000640 1 -- 172.21.15.59:0/1739673705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da57e32830 con 0x55da57e29cb0 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdff1000640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002aa10 tx=0x7fdfe00048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d5a35c705d5769 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdfd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0052060 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe0005520 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdfe0004390 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdfe0035070 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdff663b740 10 client.26486 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.273+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da57e439e0 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe0005520 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdfe0031980 con 0x55da57e29cb0 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 10 client.26486 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 20 client.26486 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e44d30 0x55da57e47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 4 client.26486 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.274+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da57d9d800 con 0x55da57e44d30 2024-09-05T12:05:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.275+0000 7fdfebe00640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e44d30 0x55da57e47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.275+0000 7fdfebe00640 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e44d30 0x55da57e47120 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbfd1333783a8c8 server_cookie=238f8642b9317e7f in_seq=0 out_seq=0 2024-09-05T12:05:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.275+0000 7fdfd2a00640 10 client.26486 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdfd2a00640 1 -- 172.21.15.59:0/1739673705 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da57d9d800 con 0x55da57e44d30 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdfd2a00640 10 client.26486 handle_command_reply: tid=0 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 client.26486 shutdown 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 msgr2=0x55da57e347f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002aa10 tx=0x7fdfe00048f0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 shutdown_connections 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e44d30 0x55da57e47120 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e34e90 0x55da57e32000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 --2- 172.21.15.59:0/1739673705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e347f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdfd2a00640 0 client.26486 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.276+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 >> 172.21.15.59:0/1739673705 conn(0x55da57e26240 msgr2=0x55da57e2bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.277+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 shutdown_connections 2024-09-05T12:05:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.277+0000 7fdff663b740 1 -- 172.21.15.59:0/1739673705 wait complete. 2024-09-05T12:05:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.277+0000 7fdff663b740 20 client.26486 trim_cache size 0 max 16384 2024-09-05T12:05:13.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.297+0000 7fdff663b740 1 Processor -- start 2024-09-05T12:05:13.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.297+0000 7fdff663b740 1 -- start start 2024-09-05T12:05:13.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e3d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1aeed0 con 0x55da57e26240 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1b1750 con 0x55da57e29cb0 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff663b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1af540 con 0x55da57e2a5d0 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38568/0 (socket says 172.21.15.59:38568) 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff1000640 1 -- 172.21.15.59:0/3870361845 learned_addr learned my addr 172.21.15.59:0/3870361845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdfebe00640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff1a00640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e3d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.298+0000 7fdff1000640 1 -- 172.21.15.59:0/3870361845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 msgr2=0x55da57e3d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdff1000640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e3d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdff1000640 1 -- 172.21.15.59:0/3870361845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 msgr2=0x55da57e3ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdff1000640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdff1000640 1 -- 172.21.15.59:0/3870361845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da57e273b0 con 0x55da57e26240 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdfebe00640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdff1000640 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00092a0 tx=0x7fdfe0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2e42a7e7b6c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdfdd600640 1 -- 172.21.15.59:0/3870361845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0052070 con 0x55da57e26240 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdfdd600640 1 -- 172.21.15.59:0/3870361845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe00036b0 con 0x55da57e26240 2024-09-05T12:05:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.299+0000 7fdfdd600640 1 -- 172.21.15.59:0/3870361845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003c40 con 0x55da57e26240 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 -- 172.21.15.59:0/3870361845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 msgr2=0x55da57e30950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00092a0 tx=0x7fdfe0005fd0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 -- 172.21.15.59:0/3870361845 shutdown_connections 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e2a5d0 0x55da57e3d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 --2- 172.21.15.59:0/3870361845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e26240 0x55da57e30950 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.300+0000 7fdff663b740 1 -- 172.21.15.59:0/3870361845 >> 172.21.15.59:0/3870361845 conn(0x55da57e33f30 msgr2=0x55da57e30240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.301+0000 7fdff663b740 1 -- 172.21.15.59:0/3870361845 shutdown_connections 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.301+0000 7fdff663b740 1 -- 172.21.15.59:0/3870361845 wait complete. 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.301+0000 7fdff663b740 1 -- start start 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 Processor -- start 2024-09-05T12:05:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e26240 0x55da57e3d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1afc10 con 0x55da57e2a5d0 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1b1080 con 0x55da57e29cb0 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec1b02e0 con 0x55da57e26240 2024-09-05T12:05:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff663b740 10 client.0 _ll_register_callbacks cb 0x55da57e32a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.302+0000 7fdff1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38584/0 (socket says 172.21.15.59:38584) 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 -- 172.21.15.59:0/3989478458 learned_addr learned my addr 172.21.15.59:0/3989478458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1000640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e26240 0x55da57e3d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdfebe00640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 -- 172.21.15.59:0/3989478458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e26240 msgr2=0x55da57e3d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e26240 0x55da57e3d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 -- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 msgr2=0x55da57e3d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 -- 172.21.15.59:0/3989478458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da57e3bf30 con 0x55da57e2a5d0 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdfebe00640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdff1a00640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003e90 tx=0x7fdfe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f560b5196aa433b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.303+0000 7fdfd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe402e060 con 0x55da57e2a5d0 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdfe400f560 con 0x55da57e2a5d0 2024-09-05T12:05:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe00036b0 con 0x55da57e2a5d0 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdfe4035050 con 0x55da57e2a5d0 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdff663b740 10 client.16494 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55da57eb0400 con 0x55da57e2a5d0 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.304+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe00036b0 con 0x55da57e2a5d0 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdfe4028070 con 0x55da57e2a5d0 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 10 client.16494 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 20 client.16494 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e34e90 0x55da57eb2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 4 client.16494 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da57d97d10 con 0x55da57e34e90 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.305+0000 7fdff1000640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e34e90 0x55da57eb2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.306+0000 7fdff1000640 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e34e90 0x55da57eb2bc0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac664fd830715a77 server_cookie=5bb9fe04b0c16122 in_seq=0 out_seq=0 2024-09-05T12:05:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.306+0000 7fdfd2a00640 10 client.16494 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdfd2a00640 1 -- 172.21.15.59:0/3989478458 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da57d97d10 con 0x55da57e34e90 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdfd2a00640 10 client.16494 handle_command_reply: tid=0 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 client.16494 shutdown 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 msgr2=0x55da57e3b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003e90 tx=0x7fdfe4004a40 comp rx=0 tx=0).stop 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 shutdown_connections 2024-09-05T12:05:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55da57e34e90 0x55da57eb2bc0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55da57e2a5d0 0x55da57e3b700 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da57e29cb0 0x55da57e3d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 --2- 172.21.15.59:0/3989478458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da57e26240 0x55da57e3d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdfd2a00640 0 client.16494 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 >> 172.21.15.59:0/3989478458 conn(0x55da57e33f30 msgr2=0x55da57e30a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 shutdown_connections 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.307+0000 7fdff663b740 1 -- 172.21.15.59:0/3989478458 wait complete. 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.308+0000 7fdff663b740 20 client.16494 trim_cache size 0 max 16384 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:13.310 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "0852cfe1-691f-4fb9-994c-e4b7627ad228", 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.308+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 msgr2=0x7fdfd40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.308+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 0x7fdfd40a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4011630 tx=0x7fdfe4009500 comp rx=0 tx=0).stop 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.308+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 msgr2=0x7fdfec19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.308+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800e9f0 tx=0x7fdfd800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 shutdown_connections 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdfd40a2c00 0x7fdfd40a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfec1079c0 0x7fdfec19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfec102360 0x7fdfec19a990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 --2- 172.21.15.59:0/650294830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfec1019c0 0x7fdfec19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 >> 172.21.15.59:0/650294830 conn(0x7fdfec0fd740 msgr2=0x7fdfec0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 shutdown_connections 2024-09-05T12:05:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:13.309+0000 7fdff2400640 1 -- 172.21.15.59:0/650294830 wait complete. 2024-09-05T12:05:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:13.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:12 smithi189 ceph-mon[34772]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:05:14.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 -- 172.21.15.59:0/4053833559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 msgr2=0x7f06a80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 --2- 172.21.15.59:0/4053833559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a80749a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0698009860 tx=0x7f069802a3d0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 -- 172.21.15.59:0/4053833559 shutdown_connections 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 --2- 172.21.15.59:0/4053833559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 0x7f06a8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 --2- 172.21.15.59:0/4053833559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 --2- 172.21.15.59:0/4053833559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a80749a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 -- 172.21.15.59:0/4053833559 >> 172.21.15.59:0/4053833559 conn(0x7f06a80fd8b0 msgr2=0x7f06a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 -- 172.21.15.59:0/4053833559 shutdown_connections 2024-09-05T12:05:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.484+0000 7f06af800640 1 -- 172.21.15.59:0/4053833559 wait complete. 2024-09-05T12:05:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 Processor -- start 2024-09-05T12:05:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 -- start start 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a8107a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 0x7f06a8102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8114190 con 0x7f06a8074ee0 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8114010 con 0x7f06a80745c0 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.486+0000 7f06af800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8114310 con 0x7f06a810e430 2024-09-05T12:05:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f069fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a8107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59208/0 (socket says 172.21.15.59:59208) 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f069fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a8107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38602/0 (socket says 172.21.15.59:38602) 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 -- 172.21.15.59:0/642303394 learned_addr learned my addr 172.21.15.59:0/642303394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ad800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 0x7f06a8102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 -- 172.21.15.59:0/642303394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 msgr2=0x7f06a8102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 0x7f06a8102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 -- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 msgr2=0x7f06a8107a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a8107a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.487+0000 7f06ace00640 1 -- 172.21.15.59:0/642303394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a8102f00 con 0x7f06a80745c0 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06ace00640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f069802a8f0 tx=0x7f0698030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06a6000640 1 -- 172.21.15.59:0/642303394 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698040020 con 0x7f06a80745c0 2024-09-05T12:05:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06a6000640 1 -- 172.21.15.59:0/642303394 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0698004460 con 0x7f06a80745c0 2024-09-05T12:05:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a8103190 con 0x7f06a80745c0 2024-09-05T12:05:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06a6000640 1 -- 172.21.15.59:0/642303394 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06980046c0 con 0x7f06a80745c0 2024-09-05T12:05:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.488+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a810d410 con 0x7f06a80745c0 2024-09-05T12:05:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.491+0000 7f06b2207740 1 Processor -- start 2024-09-05T12:05:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06a6000640 1 -- 172.21.15.59:0/642303394 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0698043020 con 0x7f06a80745c0 2024-09-05T12:05:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 -- start start 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0c6541260 con 0x55c0c65dce00 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c0c64e8500 con 0x55c0c65e76c0 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06b2207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0c653d220 con 0x55c0c65dc4e0 2024-09-05T12:05:14.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.492+0000 7f06ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38616/0 (socket says 172.21.15.59:38616) 2024-09-05T12:05:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ace00640 1 -- 172.21.15.59:0/3108961889 learned_addr learned my addr 172.21.15.59:0/3108961889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f069fe00640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 -- 172.21.15.59:0/3108961889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 msgr2=0x55c0c65e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 -- 172.21.15.59:0/3108961889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 msgr2=0x55c0c65e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 -- 172.21.15.59:0/3108961889 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c62b2ba0 con 0x55c0c65dc4e0 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ace00640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f069fe00640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.493+0000 7f06ad800640 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65dc8c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f06a8108150 tx=0x7f06a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5543f4046bbb575 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f069e000640 1 -- 172.21.15.59:0/3108961889 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004d20 con 0x55c0c65dc4e0 2024-09-05T12:05:14.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f06a6000640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f069e000640 1 -- 172.21.15.59:0/3108961889 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a0004050 con 0x55c0c65dc4e0 2024-09-05T12:05:14.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f069e000640 1 -- 172.21.15.59:0/3108961889 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0005280 con 0x55c0c65dc4e0 2024-09-05T12:05:14.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f06a6000640 1 -- 172.21.15.59:0/642303394 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f06980f03d0 con 0x7f06a80745c0 2024-09-05T12:05:14.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.494+0000 7f06ace00640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- 172.21.15.59:0/3108961889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 msgr2=0x55c0c65dc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65dc8c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f06a8108150 tx=0x7f06a0011810 comp rx=0 tx=0).stop 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- 172.21.15.59:0/3108961889 shutdown_connections 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 --2- 172.21.15.59:0/3108961889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65dc8c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- 172.21.15.59:0/3108961889 >> 172.21.15.59:0/3108961889 conn(0x55c0c65d8a70 msgr2=0x55c0c65d8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06ace00640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 0x7f068c0a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0698005e00 tx=0x7f06980f3e10 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- 172.21.15.59:0/3108961889 shutdown_connections 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- 172.21.15.59:0/3108961889 wait complete. 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.495+0000 7f06b2207740 1 -- start start 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 Processor -- start 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0c65ea290 con 0x55c0c65e76c0 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c0c64fbbd0 con 0x55c0c65dc4e0 2024-09-05T12:05:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0c65ea410 con 0x55c0c65dce00 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f06b2207740 10 client.0 _ll_register_callbacks cb 0x55c0c62500a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f069fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f069fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:48548/0 (socket says 172.21.15.59:48548) 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.496+0000 7f069fe00640 1 -- 172.21.15.59:0/1209265582 learned_addr learned my addr 172.21.15.59:0/1209265582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f06ad800640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f06ad800640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 -- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 msgr2=0x55c0c65e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 -- 172.21.15.59:0/1209265582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 msgr2=0x55c0c65e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 -- 172.21.15.59:0/1209265582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c65f3700 con 0x55c0c65dce00 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f069fe00640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06900116a0 tx=0x7f0690011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f805298a55d41ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f0696a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.497+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069001e020 con 0x55c0c65dce00 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a0004050 con 0x55c0c65dce00 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f069000fbf0 con 0x55c0c65dce00 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0690026050 con 0x55c0c65dce00 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f06b2207740 10 client.26440 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0c65f6140 con 0x55c0c65dce00 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a0004050 con 0x55c0c65dce00 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0690019070 con 0x55c0c65dce00 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f06b2207740 10 client.26440 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.498+0000 7f06b2207740 10 client.26440 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.499+0000 7f06b2207740 20 client.26440 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.499+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c65f7610 0x55c0c65f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.499+0000 7f06b2207740 4 client.26440 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.499+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c654e990 con 0x55c0c65f7610 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.499+0000 7f06ad800640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c65f7610 0x55c0c65f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.500+0000 7f06ad800640 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c65f7610 0x55c0c65f9a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acc21e334599367 server_cookie=73a52637f04d730e in_seq=0 out_seq=0 2024-09-05T12:05:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.500+0000 7f0696a00640 10 client.26440 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.500+0000 7f0696a00640 1 -- 172.21.15.59:0/1209265582 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0c654e990 con 0x55c0c65f7610 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.500+0000 7f0696a00640 10 client.26440 handle_command_reply: tid=0 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.500+0000 7f06b2207740 1 client.26440 shutdown 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 msgr2=0x55c0c65e4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06900116a0 tx=0x7f0690011b60 comp rx=0 tx=0).stop 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 shutdown_connections 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c65f7610 0x55c0c65f9a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65e4b10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 --2- 172.21.15.59:0/1209265582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f0696a00640 0 client.26440 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 >> 172.21.15.59:0/1209265582 conn(0x55c0c65d8a70 msgr2=0x55c0c65d7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 shutdown_connections 2024-09-05T12:05:14.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 1 -- 172.21.15.59:0/1209265582 wait complete. 2024-09-05T12:05:14.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.501+0000 7f06b2207740 20 client.26440 trim_cache size 0 max 16384 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.519+0000 7f06b2207740 1 Processor -- start 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.519+0000 7f06b2207740 1 -- start start 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.519+0000 7f06b2207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.519+0000 7f06b2207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06b2207740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65ff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06b2207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81bbcd0 con 0x55c0c65dce00 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06b2207740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81be550 con 0x55c0c65e76c0 2024-09-05T12:05:14.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06b2207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81bc340 con 0x55c0c65dc4e0 2024-09-05T12:05:14.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38644/0 (socket says 172.21.15.59:38644) 2024-09-05T12:05:14.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06ad800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 -- 172.21.15.59:0/4045011334 learned_addr learned my addr 172.21.15.59:0/4045011334 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f06ad800640 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 -- 172.21.15.59:0/4045011334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 msgr2=0x55c0c65e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 -- 172.21.15.59:0/4045011334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 msgr2=0x55c0c65ff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65ff300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.520+0000 7f069fe00640 1 -- 172.21.15.59:0/4045011334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c65d2fe0 con 0x55c0c65dce00 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.521+0000 7f069fe00640 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f0690001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899f6da7dd350346 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.521+0000 7f069d600640 1 -- 172.21.15.59:0/4045011334 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069002e020 con 0x55c0c65dce00 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.521+0000 7f069d600640 1 -- 172.21.15.59:0/4045011334 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0690019650 con 0x55c0c65dce00 2024-09-05T12:05:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.521+0000 7f069d600640 1 -- 172.21.15.59:0/4045011334 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069001e040 con 0x55c0c65dce00 2024-09-05T12:05:14.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 -- 172.21.15.59:0/4045011334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 msgr2=0x55c0c65d7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f0690001e70 comp rx=0 tx=0).stop 2024-09-05T12:05:14.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 -- 172.21.15.59:0/4045011334 shutdown_connections 2024-09-05T12:05:14.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65e76c0 0x55c0c65ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65dce00 0x55c0c65d7ac0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 --2- 172.21.15.59:0/4045011334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dc4e0 0x55c0c65e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 -- 172.21.15.59:0/4045011334 >> 172.21.15.59:0/4045011334 conn(0x55c0c65d8a70 msgr2=0x55c0c65d73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 -- 172.21.15.59:0/4045011334 shutdown_connections 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.522+0000 7f06b2207740 1 -- 172.21.15.59:0/4045011334 wait complete. 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 -- start start 2024-09-05T12:05:14.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 Processor -- start 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81bca10 con 0x55c0c65e76c0 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81bde80 con 0x55c0c65dc4e0 2024-09-05T12:05:14.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81bd0e0 con 0x55c0c65dce00 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.523+0000 7f06b2207740 10 client.0 _ll_register_callbacks cb 0x7f0690009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59248/0 (socket says 172.21.15.59:59248) 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 -- 172.21.15.59:0/91944938 learned_addr learned my addr 172.21.15.59:0/91944938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f069fe00640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 -- 172.21.15.59:0/91944938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 msgr2=0x55c0c65eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 -- 172.21.15.59:0/91944938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 msgr2=0x55c0c65ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 -- 172.21.15.59:0/91944938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c66613d0 con 0x55c0c65dc4e0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f069fe00640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06ad800640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005fd0 tx=0x7f06a0004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87efbe80f9ff79f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:14.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f0696a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a001d040 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.524+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0690019650 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f06a0019ad0 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f06a0019cd0 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 10 client.26510 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0c6662670 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0690019650 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f06a0023040 con 0x55c0c65dc4e0 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 10 client.26510 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 10 client.26510 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 20 client.26510 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c6662c10 0x55c0c6665000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 4 client.26510 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-09-05T12:05:14.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.525+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x55c0c6548ce0 con 0x55c0c6662c10 2024-09-05T12:05:14.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.526+0000 7f069fe00640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c6662c10 0x55c0c6665000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:14.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.526+0000 7f069fe00640 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c6662c10 0x55c0c6665000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a368be6255b5e322 server_cookie=9221f7a41f8ce2d8 in_seq=0 out_seq=0 2024-09-05T12:05:14.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.526+0000 7f0696a00640 10 client.26510 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f0696a00640 1 -- 172.21.15.59:0/91944938 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c0c6548ce0 con 0x55c0c6662c10 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f0696a00640 10 client.26510 handle_command_reply: tid=0 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 client.26510 shutdown 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 msgr2=0x55c0c65eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06a0005fd0 tx=0x7f06a0004840 comp rx=0 tx=0).stop 2024-09-05T12:05:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 shutdown_connections 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c0c6662c10 0x55c0c6665000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0c65e76c0 0x55c0c65ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0c65dce00 0x55c0c65eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 --2- 172.21.15.59:0/91944938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c0c65dc4e0 0x55c0c65eb4a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f0696a00640 0 client.26510 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 >> 172.21.15.59:0/91944938 conn(0x55c0c65d8a70 msgr2=0x55c0c65ddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 shutdown_connections 2024-09-05T12:05:14.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.535+0000 7f06b2207740 1 -- 172.21.15.59:0/91944938 wait complete. 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.536+0000 7f06b2207740 20 client.26510 trim_cache size 0 max 16384 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 26, 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:05:14.539 INFO:teuthology.orchestra.run.smithi059.stdout: "b8c9f4a": { 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 26, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 146.2, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:05:14.540 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:14.541 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.536+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 msgr2=0x7f068c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.536+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 0x7f068c0a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0698005e00 tx=0x7f06980f3e10 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.536+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 msgr2=0x7f06a8107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.536+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f069802a8f0 tx=0x7f0698030650 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 shutdown_connections 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06a810e430 0x7f06a8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a8074ee0 0x7f06a8107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 --2- 172.21.15.59:0/642303394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06a80745c0 0x7f06a8107520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 >> 172.21.15.59:0/642303394 conn(0x7f06a80fd8b0 msgr2=0x7f06a80fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 shutdown_connections 2024-09-05T12:05:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:14.537+0000 7f06af800640 1 -- 172.21.15.59:0/642303394 wait complete. 2024-09-05T12:05:15.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:14 smithi059 ceph-mon[30025]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:15.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:14 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:05:15.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:14 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:14 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:05:15.148 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:14 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:14 smithi089 ceph-mon[34499]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:14 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:05:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:14 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:14 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:05:15.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:14 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.286 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:15.315 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b8c9f4a', quiesce took 5.9 seconds. Will release after: 20.25 2024-09-05T12:05:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:14 smithi189 ceph-mon[34772]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:14 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:05:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:14 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:14 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:05:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:14 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 -- 172.21.15.59:0/958736829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 msgr2=0x7fdbd80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 --2- 172.21.15.59:0/958736829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd80748f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc802a480 comp rx=0 tx=0).stop 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 -- 172.21.15.59:0/958736829 shutdown_connections 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 --2- 172.21.15.59:0/958736829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 0x7fdbd8110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 --2- 172.21.15.59:0/958736829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 --2- 172.21.15.59:0/958736829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd80748f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 -- 172.21.15.59:0/958736829 >> 172.21.15.59:0/958736829 conn(0x7fdbd80fd8c0 msgr2=0x7fdbd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 -- 172.21.15.59:0/958736829 shutdown_connections 2024-09-05T12:05:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.860+0000 7fdbde000640 1 -- 172.21.15.59:0/958736829 wait complete. 2024-09-05T12:05:15.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.861+0000 7fdbde000640 1 Processor -- start 2024-09-05T12:05:15.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.861+0000 7fdbde000640 1 -- start start 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 0x7fdbd819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81141c0 con 0x7fdbd8074510 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8114040 con 0x7fdbd810e440 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbde000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd8114340 con 0x7fdbd8074e30 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38666/0 (socket says 172.21.15.59:38666) 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbd7400640 1 -- 172.21.15.59:0/2288313126 learned_addr learned my addr 172.21.15.59:0/2288313126 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.862+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 0x7fdbd819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 -- 172.21.15.59:0/2288313126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 msgr2=0x7fdbd819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 0x7fdbd819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 -- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 msgr2=0x7fdbd819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 -- 172.21.15.59:0/2288313126 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd819f670 con 0x7fdbd8074e30 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd819ac00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000e9f0 tx=0x7fdbc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbd4c00640 1 -- 172.21.15.59:0/2288313126 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000ccc0 con 0x7fdbd8074e30 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd819f960 con 0x7fdbd8074e30 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbd4c00640 1 -- 172.21.15.59:0/2288313126 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc0005020 con 0x7fdbd8074e30 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.863+0000 7fdbd4c00640 1 -- 172.21.15.59:0/2288313126 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00049e0 con 0x7fdbd8074e30 2024-09-05T12:05:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.864+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd81a7240 con 0x7fdbd8074e30 2024-09-05T12:05:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.866+0000 7fdbd4c00640 1 -- 172.21.15.59:0/2288313126 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbc0005560 con 0x7fdbd8074e30 2024-09-05T12:05:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 Processor -- start 2024-09-05T12:05:15.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 -- start start 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46337830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca462933e0 con 0x55ca46337d70 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ca4623ad80 con 0x55ca4632d4b0 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca4628f3a0 con 0x55ca4632cb90 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38674/0 (socket says 172.21.15.59:38674) 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbcfe00640 1 -- 172.21.15.59:0/163298194 learned_addr learned my addr 172.21.15.59:0/163298194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:15.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.867+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7400640 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46337830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 -- 172.21.15.59:0/163298194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 msgr2=0x55ca46337830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46337830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 -- 172.21.15.59:0/163298194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 msgr2=0x55ca4633a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 -- 172.21.15.59:0/163298194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca46259ae0 con 0x55ca4632cb90 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd4c00640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 0x7fdbbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.868+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632cf70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005fd0 tx=0x7fdbd0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ccedbdece65edd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbd4c00640 1 -- 172.21.15.59:0/2288313126 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdbc00cf380 con 0x7fdbd8074e30 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbc7e00640 1 -- 172.21.15.59:0/163298194 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0004b20 con 0x55ca4632cb90 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbc7e00640 1 -- 172.21.15.59:0/163298194 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd0004cc0 con 0x55ca4632cb90 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbc7e00640 1 -- 172.21.15.59:0/163298194 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000e7f0 con 0x55ca4632cb90 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 0x7fdbbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 -- 172.21.15.59:0/163298194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 msgr2=0x55ca4632cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632cf70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0005fd0 tx=0x7fdbd0011810 comp rx=0 tx=0).stop 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 0x7fdbbc0a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80096c0 tx=0x7fdbc80046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 -- 172.21.15.59:0/163298194 shutdown_connections 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46337830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/163298194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632cf70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.869+0000 7fdbe09d7740 1 -- 172.21.15.59:0/163298194 >> 172.21.15.59:0/163298194 conn(0x55ca46329120 msgr2=0x55ca46329500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.870+0000 7fdbe09d7740 1 -- 172.21.15.59:0/163298194 shutdown_connections 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.870+0000 7fdbe09d7740 1 -- 172.21.15.59:0/163298194 wait complete. 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.870+0000 7fdbe09d7740 1 -- start start 2024-09-05T12:05:15.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.870+0000 7fdbe09d7740 1 Processor -- start 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46335200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca4633a9e0 con 0x55ca46337d70 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ca4624e0f0 con 0x55ca4632d4b0 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca4633ab60 con 0x55ca4632cb90 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbe09d7740 10 client.0 _ll_register_callbacks cb 0x55ca4623f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38680/0 (socket says 172.21.15.59:38680) 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:48606/0 (socket says 172.21.15.59:48606) 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbd7e00640 1 -- 172.21.15.59:0/2104590669 learned_addr learned my addr 172.21.15.59:0/2104590669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.871+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46335200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 -- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 msgr2=0x55ca46335200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46335200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 -- 172.21.15.59:0/2104590669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 msgr2=0x55ca46335740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 -- 172.21.15.59:0/2104590669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca460043c0 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8004770 tx=0x7fdbc8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd9a7c755c96ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.872+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc804e070 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd0004cc0 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdbc80037f0 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdbc805a050 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 10 client.26458 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ca46346740 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd0004cc0 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdbc804aaf0 con 0x55ca4632cb90 2024-09-05T12:05:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 10 client.26458 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 20 client.26458 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.873+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca46347b50 0x55ca46349f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.874+0000 7fdbe09d7740 4 client.26458 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.874+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca462a0950 con 0x55ca46347b50 2024-09-05T12:05:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.874+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca46347b50 0x55ca46349f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.874+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca46347b50 0x55ca46349f40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72975dc094e5e40 server_cookie=7857229bdfe798c7 in_seq=0 out_seq=0 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.874+0000 7fdbc4c00640 10 client.26458 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbc4c00640 1 -- 172.21.15.59:0/2104590669 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca462a0950 con 0x55ca46347b50 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbc4c00640 10 client.26458 handle_command_reply: tid=0 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 client.26458 shutdown 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 msgr2=0x55ca46334cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8004770 tx=0x7fdbc8009260 comp rx=0 tx=0).stop 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 shutdown_connections 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca46347b50 0x55ca46349f40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca46335740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca46335200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2104590669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca46334cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbc4c00640 0 client.26458 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.875+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 >> 172.21.15.59:0/2104590669 conn(0x55ca46329120 msgr2=0x55ca4632ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.876+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 shutdown_connections 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.876+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2104590669 wait complete. 2024-09-05T12:05:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.876+0000 7fdbe09d7740 20 client.26458 trim_cache size 0 max 16384 2024-09-05T12:05:15.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 Processor -- start 2024-09-05T12:05:15.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 -- start start 2024-09-05T12:05:15.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4634f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81af0c0 con 0x55ca46337d70 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b1940 con 0x55ca4632d4b0 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81af730 con 0x55ca4632cb90 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38682/0 (socket says 172.21.15.59:38682) 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.894+0000 7fdbd7e00640 1 -- 172.21.15.59:0/2682568452 learned_addr learned my addr 172.21.15.59:0/2682568452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4634f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 -- 172.21.15.59:0/2682568452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 msgr2=0x55ca4634f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4634f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 -- 172.21.15.59:0/2682568452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 msgr2=0x55ca4634fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 -- 172.21.15.59:0/2682568452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4634a5a0 con 0x55ca4632cb90 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4634f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.895+0000 7fdbd7400640 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632ecf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdbc803d450 tx=0x7fdbc8031a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc2fa5b407b4c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbc7400640 1 -- 172.21.15.59:0/2682568452 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc804e050 con 0x55ca4632cb90 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbc7400640 1 -- 172.21.15.59:0/2682568452 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc803d6d0 con 0x55ca4632cb90 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbc7400640 1 -- 172.21.15.59:0/2682568452 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8035040 con 0x55ca4632cb90 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2682568452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 msgr2=0x55ca4632ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632ecf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdbc803d450 tx=0x7fdbc8031a40 comp rx=0 tx=0).stop 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2682568452 shutdown_connections 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4634fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4634f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/2682568452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca4632ecf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2682568452 >> 172.21.15.59:0/2682568452 conn(0x55ca46329120 msgr2=0x55ca4632e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2682568452 shutdown_connections 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.896+0000 7fdbe09d7740 1 -- 172.21.15.59:0/2682568452 wait complete. 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 -- start start 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 Processor -- start 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca463406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4633ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81afe00 con 0x55ca46337d70 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b1270 con 0x55ca4632d4b0 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.897+0000 7fdbe09d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd81b04d0 con 0x55ca4632cb90 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbe09d7740 10 client.0 _ll_register_callbacks cb 0x7fdbc80042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca463406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca463406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:48626/0 (socket says 172.21.15.59:48626) 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbd7400640 1 -- 172.21.15.59:0/1943938749 learned_addr learned my addr 172.21.15.59:0/1943938749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4633ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 -- 172.21.15.59:0/1943938749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 msgr2=0x55ca463406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca463406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 -- 172.21.15.59:0/1943938749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 msgr2=0x55ca4633c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbcfe00640 1 -- 172.21.15.59:0/1943938749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca463b17c0 con 0x55ca4632d4b0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.898+0000 7fdbd7e00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbcfe00640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4633ba20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0098140 tx=0x7fdbc00d2dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ab6b3a936fb870 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc001c040 con 0x55ca4632d4b0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdbc00ccb20 con 0x55ca4632d4b0 2024-09-05T12:05:15.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc803d6d0 con 0x55ca4632d4b0 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdbc00051d0 con 0x55ca4632d4b0 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 10 client.26528 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ca463b29d0 con 0x55ca4632d4b0 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc803d6d0 con 0x55ca4632d4b0 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdbc0097de0 con 0x55ca4632d4b0 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 10 client.26528 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 20 client.26528 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca463b3180 0x55ca463b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 4 client.26528 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:05:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.899+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca4629ae60 con 0x55ca463b3180 2024-09-05T12:05:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.900+0000 7fdbd7400640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca463b3180 0x55ca463b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.900+0000 7fdbd7400640 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca463b3180 0x55ca463b55a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc9510356a9a879 server_cookie=d75a5f5ffef11f23 in_seq=0 out_seq=0 2024-09-05T12:05:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.900+0000 7fdbc4c00640 10 client.26528 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbc4c00640 1 -- 172.21.15.59:0/1943938749 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca4629ae60 con 0x55ca463b3180 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbc4c00640 10 client.26528 handle_command_reply: tid=0 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 client.26528 shutdown 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 msgr2=0x55ca4633ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4633ba20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0098140 tx=0x7fdbc00d2dc0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 shutdown_connections 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ca463b3180 0x55ca463b55a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca46337d70 0x55ca4633c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ca4632d4b0 0x55ca4633ba20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbc4c00640 0 client.26528 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 --2- 172.21.15.59:0/1943938749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca4632cb90 0x55ca463406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 >> 172.21.15.59:0/1943938749 conn(0x55ca46329120 msgr2=0x55ca4632e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 shutdown_connections 2024-09-05T12:05:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 1 -- 172.21.15.59:0/1943938749 wait complete. 2024-09-05T12:05:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.901+0000 7fdbe09d7740 20 client.26528 trim_cache size 0 max 16384 2024-09-05T12:05:15.907 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:15.907 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:05:15.907 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:05:15.907 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 msgr2=0x7fdbbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 0x7fdbbc0a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80096c0 tx=0x7fdbc80046c0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 msgr2=0x7fdbd819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd819ac00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000e9f0 tx=0x7fdbc000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 shutdown_connections 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdbbc0a2c00 0x7fdbbc0a50c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbd810e440 0x7fdbd819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbd8074e30 0x7fdbd819ac00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 --2- 172.21.15.59:0/2288313126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbd8074510 0x7fdbd819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 >> 172.21.15.59:0/2288313126 conn(0x7fdbd80fd8c0 msgr2=0x7fdbd80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 shutdown_connections 2024-09-05T12:05:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:15.902+0000 7fdbde000640 1 -- 172.21.15.59:0/2288313126 wait complete. 2024-09-05T12:05:15.909 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:15 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:05:16.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:15 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:05:16.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:15 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:05:16.688 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:05:16.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:16.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:16 smithi059 ceph-mon[30025]: pgmap v622: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:17.071 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:16 smithi089 ceph-mon[34499]: pgmap v622: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:17.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:16 smithi189 ceph-mon[34772]: pgmap v622: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 -- 172.21.15.59:0/3172643794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc103e30 msgr2=0x7f6bdc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 --2- 172.21.15.59:0/3172643794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc103e30 0x7f6bdc104210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009900 tx=0x7f6bd002a410 comp rx=0 tx=0).stop 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 -- 172.21.15.59:0/3172643794 shutdown_connections 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 --2- 172.21.15.59:0/3172643794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc108f60 0x7f6bdc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 --2- 172.21.15.59:0/3172643794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 --2- 172.21.15.59:0/3172643794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc103e30 0x7f6bdc104210 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 -- 172.21.15.59:0/3172643794 >> 172.21.15.59:0/3172643794 conn(0x7f6bdc0ff8d0 msgr2=0x7f6bdc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.625+0000 7f6be2800640 1 -- 172.21.15.59:0/3172643794 shutdown_connections 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.626+0000 7f6be2800640 1 -- 172.21.15.59:0/3172643794 wait complete. 2024-09-05T12:05:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.626+0000 7f6be2800640 1 Processor -- start 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 -- start start 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc108f60 0x7f6bdc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc114010 con 0x7f6bdc108f60 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc113e90 con 0x7f6bdc103e30 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be2800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc114190 con 0x7f6bdc1047e0 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc108f60 0x7f6bdc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:59326/0 (socket says 172.21.15.59:59326) 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be1400640 1 -- 172.21.15.59:0/280461474 learned_addr learned my addr 172.21.15.59:0/280461474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.627+0000 7f6be0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:48658/0 (socket says 172.21.15.59:48658) 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 -- 172.21.15.59:0/280461474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 msgr2=0x7f6bdc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 -- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc108f60 msgr2=0x7f6bdc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc108f60 0x7f6bdc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 -- 172.21.15.59:0/280461474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc19f440 con 0x7f6bdc103e30 2024-09-05T12:05:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.628+0000 7f6be1400640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0004770 tx=0x7f6bd0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.629+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd003f070 con 0x7f6bdc103e30 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.629+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc19f6d0 con 0x7f6bdc103e30 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.629+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd0031ca0 con 0x7f6bdc103e30 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.629+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0035030 con 0x7f6bdc103e30 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.629+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a7010 con 0x7f6bdc103e30 2024-09-05T12:05:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.630+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc104210 con 0x7f6bdc103e30 2024-09-05T12:05:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.632+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6bd0042070 con 0x7f6bdc103e30 2024-09-05T12:05:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.633+0000 7f6bda000640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 0x7f6bc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.633+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6bd00efff0 con 0x7f6bdc103e30 2024-09-05T12:05:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.633+0000 7f6be0a00640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 0x7f6bc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:17.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.634+0000 7f6be0a00640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 0x7f6bc40a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0ffcf0 tx=0x7f6bc8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.636+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bd00bc590 con 0x7f6bdc103e30 2024-09-05T12:05:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.857+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bdc10c140 con 0x7f6bdc103e30 2024-09-05T12:05:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.858+0000 7f6bda000640 1 -- 172.21.15.59:0/280461474 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f6bd00c0890 con 0x7f6bdc103e30 2024-09-05T12:05:17.859 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:17.860 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.860+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 msgr2=0x7f6bc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.860+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 0x7f6bc40a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0ffcf0 tx=0x7f6bc8005f90 comp rx=0 tx=0).stop 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.860+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 msgr2=0x7f6bdc19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.860+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0004770 tx=0x7f6bd0003c10 comp rx=0 tx=0).stop 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 shutdown_connections 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6bc40a2b80 0x7f6bc40a5040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bdc108f60 0x7f6bdc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bdc1047e0 0x7f6bdc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 --2- 172.21.15.59:0/280461474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6bdc103e30 0x7f6bdc19a490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 >> 172.21.15.59:0/280461474 conn(0x7f6bdc0ff8d0 msgr2=0x7f6bdc100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 shutdown_connections 2024-09-05T12:05:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:17.861+0000 7f6be2800640 1 -- 172.21.15.59:0/280461474 wait complete. 2024-09-05T12:05:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:17 smithi089 ceph-mon[34499]: pgmap v623: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:17 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/280461474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:17 smithi059 ceph-mon[30025]: pgmap v623: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:17 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/280461474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:18.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:17 smithi189 ceph-mon[34772]: pgmap v623: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:05:18.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:17 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/280461474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:18.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:05:18.970 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 -- 172.21.15.59:0/3039482807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 msgr2=0x7fe2180749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 --2- 172.21.15.59:0/3039482807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe2180749a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009be0 tx=0x7fe20c02a750 comp rx=0 tx=0).stop 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 -- 172.21.15.59:0/3039482807 shutdown_connections 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 --2- 172.21.15.59:0/3039482807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe21810e240 0x7fe218110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 --2- 172.21.15.59:0/3039482807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe218074ee0 0x7fe21810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 --2- 172.21.15.59:0/3039482807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe2180749a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 -- 172.21.15.59:0/3039482807 >> 172.21.15.59:0/3039482807 conn(0x7fe2180fd700 msgr2=0x7fe2180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 -- 172.21.15.59:0/3039482807 shutdown_connections 2024-09-05T12:05:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.542+0000 7fe21ee00640 1 -- 172.21.15.59:0/3039482807 wait complete. 2024-09-05T12:05:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.543+0000 7fe21ee00640 1 Processor -- start 2024-09-05T12:05:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 -- start start 2024-09-05T12:05:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe21819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 0x7fe21819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe218113f40 con 0x7fe21810e240 2024-09-05T12:05:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe218113dc0 con 0x7fe218074ee0 2024-09-05T12:05:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.544+0000 7fe21ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181140c0 con 0x7fe2180745c0 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.545+0000 7fe21e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.545+0000 7fe21e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38734/0 (socket says 172.21.15.59:38734) 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.545+0000 7fe21e400640 1 -- 172.21.15.59:0/763256845 learned_addr learned my addr 172.21.15.59:0/763256845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.545+0000 7fe21da00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe21819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.545+0000 7fe21d000640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 0x7fe21819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 -- 172.21.15.59:0/763256845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 msgr2=0x7fe21819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe21819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 -- 172.21.15.59:0/763256845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 msgr2=0x7fe21819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 0x7fe21819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 -- 172.21.15.59:0/763256845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21819f450 con 0x7fe21810e240 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21d000640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 0x7fe21819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.546+0000 7fe21e400640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e970 tx=0x7fe21000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.547+0000 7fe216000640 1 -- 172.21.15.59:0/763256845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000ccc0 con 0x7fe21810e240 2024-09-05T12:05:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.547+0000 7fe216000640 1 -- 172.21.15.59:0/763256845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe210005500 con 0x7fe21810e240 2024-09-05T12:05:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.547+0000 7fe216000640 1 -- 172.21.15.59:0/763256845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210010690 con 0x7fe21810e240 2024-09-05T12:05:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.547+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21819f740 con 0x7fe21810e240 2024-09-05T12:05:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.547+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a7020 con 0x7fe21810e240 2024-09-05T12:05:19.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.550+0000 7fe216000640 1 -- 172.21.15.59:0/763256845 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe2100046c0 con 0x7fe21810e240 2024-09-05T12:05:19.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.550+0000 7fe223001740 1 Processor -- start 2024-09-05T12:05:19.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.550+0000 7fe223001740 1 -- start start 2024-09-05T12:05:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1808fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e1809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e17fed3e0 con 0x562e1808ff70 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562e17f94d80 con 0x562e1809a830 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe223001740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562e17fe93a0 con 0x562e1808f650 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38740/0 (socket says 172.21.15.59:38740) 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe21da00640 1 -- 172.21.15.59:0/2260574812 learned_addr learned my addr 172.21.15.59:0/2260574812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe21d000640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1808fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.551+0000 7fe21d000640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e1809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 -- 172.21.15.59:0/2260574812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 msgr2=0x562e1808fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1808fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 -- 172.21.15.59:0/2260574812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 msgr2=0x562e1809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e1809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 -- 172.21.15.59:0/2260574812 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e17fb3ae0 con 0x562e1808ff70 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21d000640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e1809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe216000640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 0x7fe2000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe216000640 1 -- 172.21.15.59:0/763256845 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe2100cf830 con 0x7fe21810e240 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21da00640 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009be0 tx=0x7fe20c005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503676900e6b4e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe21d000640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 0x7fe2000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.552+0000 7fe20aa00640 1 -- 172.21.15.59:0/2260574812 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c042030 con 0x562e1808ff70 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe20aa00640 1 -- 172.21.15.59:0/2260574812 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c037040 con 0x562e1808ff70 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe20aa00640 1 -- 172.21.15.59:0/2260574812 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c036070 con 0x562e1808ff70 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe223001740 1 -- 172.21.15.59:0/2260574812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 msgr2=0x562e1809a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe223001740 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009be0 tx=0x7fe20c005c60 comp rx=0 tx=0).stop 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe21d000640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 0x7fe2000a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe204005ab0 tx=0x7fe204009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.553+0000 7fe223001740 1 -- 172.21.15.59:0/2260574812 shutdown_connections 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e1809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808ff70 0x562e1809a2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 --2- 172.21.15.59:0/2260574812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1808fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 -- 172.21.15.59:0/2260574812 >> 172.21.15.59:0/2260574812 conn(0x562e1808bbe0 msgr2=0x562e1808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 -- 172.21.15.59:0/2260574812 shutdown_connections 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 -- 172.21.15.59:0/2260574812 wait complete. 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 -- start start 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.554+0000 7fe223001740 1 Processor -- start 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e18097c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1809a830 0x562e180981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e1809d3f0 con 0x562e1808f650 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562e17fa80f0 con 0x562e1808ff70 2024-09-05T12:05:19.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562e1809d570 con 0x562e1809a830 2024-09-05T12:05:19.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.555+0000 7fe223001740 10 client.0 _ll_register_callbacks cb 0x562e17f8fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:19.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38756/0 (socket says 172.21.15.59:38756) 2024-09-05T12:05:19.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 -- 172.21.15.59:0/1789190675 learned_addr learned my addr 172.21.15.59:0/1789190675 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:19.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 -- 172.21.15.59:0/1789190675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1809a830 msgr2=0x562e180981c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:05:19.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1809a830 0x562e180981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 -- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 msgr2=0x562e18097c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:05:19.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e18097c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.556+0000 7fe21da00640 1 -- 172.21.15.59:0/1789190675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e17d5e3c0 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.557+0000 7fe21da00640 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02ac70 tx=0x7fe20c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c812ba4d6ad722e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.557+0000 7fe1ff400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.557+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c042040 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.557+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe20c0305a0 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c037040 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe20c058050 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe223001740 10 client.16548 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562e180a9430 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c037040 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.558+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe20c03c040 con 0x562e1808f650 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 10 client.16548 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 20 client.16548 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:19.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e180aa780 0x562e180acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 4 client.16548 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:19.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e17ffa950 con 0x562e180aa780 2024-09-05T12:05:19.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.559+0000 7fe21d000640 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e180aa780 0x562e180acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.560+0000 7fe21d000640 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e180aa780 0x562e180acb70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682f57cf57a001f2 server_cookie=b79e8627327bc775 in_seq=0 out_seq=0 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.560+0000 7fe1ff400640 10 client.16548 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.560+0000 7fe1ff400640 1 -- 172.21.15.59:0/1789190675 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e17ffa950 con 0x562e180aa780 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.560+0000 7fe1ff400640 10 client.16548 handle_command_reply: tid=0 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.560+0000 7fe223001740 1 client.16548 shutdown 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 msgr2=0x562e18097740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02ac70 tx=0x7fe20c02a750 comp rx=0 tx=0).stop 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 shutdown_connections 2024-09-05T12:05:19.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1809a830 0x562e180981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e18097c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e180aa780 0x562e180acb70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe1ff400640 0 client.16548 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 --2- 172.21.15.59:0/1789190675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e18097740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 >> 172.21.15.59:0/1789190675 conn(0x562e1808bbe0 msgr2=0x562e180918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 shutdown_connections 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 1 -- 172.21.15.59:0/1789190675 wait complete. 2024-09-05T12:05:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.561+0000 7fe223001740 20 client.16548 trim_cache size 0 max 16384 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.575+0000 7fe223001740 1 Processor -- start 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.575+0000 7fe223001740 1 -- start start 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.575+0000 7fe223001740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e1807e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.575+0000 7fe223001740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.575+0000 7fe223001740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e180b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe223001740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181aaa00 con 0x562e1808f650 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe223001740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181ad280 con 0x562e1809a830 2024-09-05T12:05:19.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe223001740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181ab070 con 0x562e1808ff70 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e1807e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:48712/0 (socket says 172.21.15.59:48712) 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21d000640 1 -- 172.21.15.59:0/397086948 learned_addr learned my addr 172.21.15.59:0/397086948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e180b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 -- 172.21.15.59:0/397086948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 msgr2=0x562e180b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 -- 172.21.15.59:0/397086948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 msgr2=0x562e1807e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e1807e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21e400640 1 -- 172.21.15.59:0/397086948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e180966f0 con 0x562e1809a830 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21d000640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.576+0000 7fe21da00640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e1807e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.577+0000 7fe21e400640 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e180b26e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe21009bbb0 tx=0x7fe21009b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e694a2d2364c455b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.577+0000 7fe20a000640 1 -- 172.21.15.59:0/397086948 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21009c320 con 0x562e1809a830 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.577+0000 7fe20a000640 1 -- 172.21.15.59:0/397086948 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21009bc20 con 0x562e1809a830 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.577+0000 7fe20a000640 1 -- 172.21.15.59:0/397086948 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100a4660 con 0x562e1809a830 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- 172.21.15.59:0/397086948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 msgr2=0x562e180b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e180b26e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe21009bbb0 tx=0x7fe21009b760 comp rx=0 tx=0).stop 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- 172.21.15.59:0/397086948 shutdown_connections 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1809a830 0x562e180b26e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808ff70 0x562e180b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 --2- 172.21.15.59:0/397086948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1808f650 0x562e1807e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- 172.21.15.59:0/397086948 >> 172.21.15.59:0/397086948 conn(0x562e1808bbe0 msgr2=0x562e180918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- 172.21.15.59:0/397086948 shutdown_connections 2024-09-05T12:05:19.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- 172.21.15.59:0/397086948 wait complete. 2024-09-05T12:05:19.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 -- start start 2024-09-05T12:05:19.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.578+0000 7fe223001740 1 Processor -- start 2024-09-05T12:05:19.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1809e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e1809eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181ab740 con 0x562e1809a830 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181acbb0 con 0x562e1808ff70 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181abe10 con 0x562e1808f650 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe223001740 10 client.0 _ll_register_callbacks cb 0x7fe204009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe21e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe21da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1809e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe21e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38776/0 (socket says 172.21.15.59:38776) 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe21d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e1809eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.579+0000 7fe21e400640 1 -- 172.21.15.59:0/3708552192 learned_addr learned my addr 172.21.15.59:0/3708552192 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 -- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 msgr2=0x562e1809eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e1809eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 -- 172.21.15.59:0/3708552192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 msgr2=0x562e1809f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 -- 172.21.15.59:0/3708552192 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e181142d0 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21d000640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e1809eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21e400640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe21da00640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1809e630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009be0 tx=0x7fe20c005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843d941ebc9abf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe1ff400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c042040 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21009bc20 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe20c0303a0 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.580+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe20c065050 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 10 client.26485 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562e18115550 con 0x562e1808f650 2024-09-05T12:05:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe21009bc20 con 0x562e1808f650 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe20c05b030 con 0x562e1808f650 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 10 client.26485 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 20 client.26485 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e18115c10 0x562e18118000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 4 client.26485 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.581+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e17ff4e60 con 0x562e18115c10 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe21d000640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e18115c10 0x562e18118000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe21d000640 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e18115c10 0x562e18118000 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b60c554ef16444 server_cookie=ae9043c7c865b010 in_seq=0 out_seq=0 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe1ff400640 10 client.26485 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe1ff400640 1 -- 172.21.15.59:0/3708552192 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e17ff4e60 con 0x562e18115c10 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe1ff400640 10 client.26485 handle_command_reply: tid=0 2024-09-05T12:05:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe223001740 1 client.26485 shutdown 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 msgr2=0x562e1809e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.582+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1809e630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009be0 tx=0x7fe20c005cf0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 shutdown_connections 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562e18115c10 0x562e18118000 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e1809a830 0x562e1809f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1808ff70 0x562e1809eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe1ff400640 0 client.26485 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 --2- 172.21.15.59:0/3708552192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e1808f650 0x562e1809e630 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 >> 172.21.15.59:0/3708552192 conn(0x562e1808bbe0 msgr2=0x562e18097700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 shutdown_connections 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 1 -- 172.21.15.59:0/3708552192 wait complete. 2024-09-05T12:05:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.583+0000 7fe223001740 20 client.26485 trim_cache size 0 max 16384 2024-09-05T12:05:19.589 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 msgr2=0x7fe2000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 0x7fe2000a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe204005ab0 tx=0x7fe204009260 comp rx=0 tx=0).stop 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 msgr2=0x7fe21819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e970 tx=0x7fe21000ee30 comp rx=0 tx=0).stop 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 shutdown_connections 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2000a2c00 0x7fe2000a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe21810e240 0x7fe21819ed70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe218074ee0 0x7fe21819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 --2- 172.21.15.59:0/763256845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2180745c0 0x7fe21819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 >> 172.21.15.59:0/763256845 conn(0x7fe2180fd700 msgr2=0x7fe2180feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 shutdown_connections 2024-09-05T12:05:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:19.584+0000 7fe21ee00640 1 -- 172.21.15.59:0/763256845 wait complete. 2024-09-05T12:05:20.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:20 smithi059 ceph-mon[30025]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:20 smithi089 ceph-mon[34499]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:20 smithi189 ceph-mon[34772]: pgmap v624: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:05:21.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:05:21.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:05:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:22 smithi059 ceph-mon[30025]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:22.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:22 smithi089 ceph-mon[34499]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 -- 172.21.15.59:0/2602040625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c112a30 msgr2=0x7f6b9c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 --2- 172.21.15.59:0/2602040625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c112a30 0x7f6b9c114e20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009ca0 tx=0x7f6b9402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 -- 172.21.15.59:0/2602040625 shutdown_connections 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 --2- 172.21.15.59:0/2602040625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c112a30 0x7f6b9c114e20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 --2- 172.21.15.59:0/2602040625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c077040 0x7f6b9c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 --2- 172.21.15.59:0/2602040625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c076720 0x7f6b9c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 -- 172.21.15.59:0/2602040625 >> 172.21.15.59:0/2602040625 conn(0x7f6b9c0ffa90 msgr2=0x7f6b9c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 -- 172.21.15.59:0/2602040625 shutdown_connections 2024-09-05T12:05:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.462+0000 7f6ba2000640 1 -- 172.21.15.59:0/2602040625 wait complete. 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.463+0000 7f6ba2000640 1 Processor -- start 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 -- start start 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c077040 0x7f6b9c1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c112a30 0x7f6b9c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1185d0 con 0x7f6b9c076720 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c118450 con 0x7f6b9c112a30 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6ba2000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c118750 con 0x7f6b9c077040 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39194/0 (socket says 172.21.15.59:39194) 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6b9b400640 1 -- 172.21.15.59:0/4202073304 learned_addr learned my addr 172.21.15.59:0/4202073304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.464+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c077040 0x7f6b9c1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9be00640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c112a30 0x7f6b9c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 -- 172.21.15.59:0/4202073304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c077040 msgr2=0x7f6b9c1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c077040 0x7f6b9c1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 -- 172.21.15.59:0/4202073304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c112a30 msgr2=0x7f6b9c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c112a30 0x7f6b9c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 -- 172.21.15.59:0/4202073304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c19efd0 con 0x7f6b9c076720 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00e9f0 tx=0x7f6b8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.465+0000 7f6b93e00640 1 -- 172.21.15.59:0/4202073304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c00cd80 con 0x7f6b9c076720 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.466+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c19f2c0 con 0x7f6b9c076720 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.466+0000 7f6b93e00640 1 -- 172.21.15.59:0/4202073304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c005500 con 0x7f6b9c076720 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.466+0000 7f6b93e00640 1 -- 172.21.15.59:0/4202073304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c010690 con 0x7f6b9c076720 2024-09-05T12:05:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.466+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1ab5b0 con 0x7f6b9c076720 2024-09-05T12:05:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 Processor -- start 2024-09-05T12:05:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 -- start start 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6b93e00640 1 -- 172.21.15.59:0/4202073304 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b8c0046c0 con 0x7f6b9c076720 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11febea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.470+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c11f52290 con 0x561c11febac0 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561c11ef9b40 con 0x561c11fec3e0 2024-09-05T12:05:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c11f4e250 con 0x561c11ff6ca0 2024-09-05T12:05:22.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11febea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53014/0 (socket says 172.21.15.59:53014) 2024-09-05T12:05:22.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11febea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39198/0 (socket says 172.21.15.59:39198) 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 -- 172.21.15.59:0/3724605282 learned_addr learned my addr 172.21.15.59:0/3724605282 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 -- 172.21.15.59:0/3724605282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 msgr2=0x561c11ff9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 -- 172.21.15.59:0/3724605282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 msgr2=0x561c11febea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11febea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.471+0000 7f6b9b400640 1 -- 172.21.15.59:0/3724605282 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c11cc36a0 con 0x561c11fec3e0 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b9b400640 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0d5a80 tx=0x7f6b8c0d5b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa92bae98445170 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b87e00640 1 -- 172.21.15.59:0/3724605282 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0de070 con 0x561c11fec3e0 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b87e00640 1 -- 172.21.15.59:0/3724605282 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c0da440 con 0x561c11fec3e0 2024-09-05T12:05:22.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b87e00640 1 -- 172.21.15.59:0/3724605282 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0d12d0 con 0x561c11fec3e0 2024-09-05T12:05:22.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b93e00640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 0x7f6b800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.472+0000 7f6b93e00640 1 -- 172.21.15.59:0/4202073304 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6b8c0cf7d0 con 0x7f6b9c076720 2024-09-05T12:05:22.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6b9be00640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 0x7f6b800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 -- 172.21.15.59:0/3724605282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 msgr2=0x561c11ff6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0d5a80 tx=0x7f6b8c0d5b70 comp rx=0 tx=0).stop 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 -- 172.21.15.59:0/3724605282 shutdown_connections 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff6760 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/3724605282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11febea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 -- 172.21.15.59:0/3724605282 >> 172.21.15.59:0/3724605282 conn(0x561c11fe8050 msgr2=0x561c11fe8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 -- 172.21.15.59:0/3724605282 shutdown_connections 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6ba4aae740 1 -- 172.21.15.59:0/3724605282 wait complete. 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.473+0000 7f6b9be00640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 0x7f6b800a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b94005e30 tx=0x7f6b94005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 -- start start 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 Processor -- start 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c11ff9910 con 0x561c11febac0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561c11f0cfa0 con 0x561c11fec3e0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c11ff9a90 con 0x561c11ff6ca0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.474+0000 7f6ba4aae740 10 client.0 _ll_register_callbacks cb 0x561c11c60ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39208/0 (socket says 172.21.15.59:39208) 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 -- 172.21.15.59:0/4054699593 learned_addr learned my addr 172.21.15.59:0/4054699593 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9be00640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 -- 172.21.15.59:0/4054699593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 msgr2=0x561c11ff3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 -- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 msgr2=0x561c11ff3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 -- 172.21.15.59:0/4054699593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c11ff4540 con 0x561c11febac0 2024-09-05T12:05:22.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9be00640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:22.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b9b400640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0985f0 tx=0x7f6b8c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10e1fa548494716 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.475+0000 7f6b84c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0de070 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6b8c09be80 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c0da440 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c0b9050 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6ba4aae740 10 client.16566 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c120057c0 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c0da440 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.476+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6b8c0a4400 con 0x561c11febac0 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 10 client.16566 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:22.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 20 client.16566 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c12006b10 0x561c12008f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 4 client.16566 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c11f5f800 con 0x561c12006b10 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.477+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c12006b10 0x561c12008f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.478+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c12006b10 0x561c12008f00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f825578ad6bbd14 server_cookie=209138bfba050a8f in_seq=0 out_seq=0 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.478+0000 7f6b84c00640 10 client.16566 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.478+0000 7f6b84c00640 1 -- 172.21.15.59:0/4054699593 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c11f5f800 con 0x561c12006b10 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.478+0000 7f6b84c00640 10 client.16566 handle_command_reply: tid=0 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 client.16566 shutdown 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 msgr2=0x561c11ff6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0985f0 tx=0x7f6b8c00cf40 comp rx=0 tx=0).stop 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 shutdown_connections 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11ff6ca0 0x561c11ff3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11fec3e0 0x561c11ff3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c12006b10 0x561c12008f00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/4054699593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11febac0 0x561c11ff6550 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 >> 172.21.15.59:0/4054699593 conn(0x561c11fe8050 msgr2=0x561c11fedd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6b84c00640 0 client.16566 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 shutdown_connections 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 1 -- 172.21.15.59:0/4054699593 wait complete. 2024-09-05T12:05:22.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.479+0000 7f6ba4aae740 20 client.16566 trim_cache size 0 max 16384 2024-09-05T12:05:22.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 Processor -- start 2024-09-05T12:05:22.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 -- start start 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 0x561c11ff5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11fec3e0 0x561c1200bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 0x561c1200c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b3510 con 0x561c11ff6ca0 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b5d90 con 0x561c11febac0 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b3b80 con 0x561c11fec3e0 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.498+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 0x561c1200c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 0x561c11ff5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 0x561c1200c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39224/0 (socket says 172.21.15.59:39224) 2024-09-05T12:05:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9be00640 1 -- 172.21.15.59:0/608059862 learned_addr learned my addr 172.21.15.59:0/608059862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11fec3e0 0x561c1200bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 -- 172.21.15.59:0/608059862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11fec3e0 msgr2=0x561c1200bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11fec3e0 0x561c1200bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 -- 172.21.15.59:0/608059862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 msgr2=0x561c1200c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 0x561c1200c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 -- 172.21.15.59:0/608059862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c11fe7e80 con 0x561c11febac0 2024-09-05T12:05:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.499+0000 7f6b9b400640 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 0x561c11ff5770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c09bbb0 tx=0x7f6b8c0d5b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7385cdecdd4c52f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6b87400640 1 -- 172.21.15.59:0/608059862 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0de070 con 0x561c11febac0 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6b87400640 1 -- 172.21.15.59:0/608059862 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c0d5320 con 0x561c11febac0 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6b87400640 1 -- 172.21.15.59:0/608059862 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c097ec0 con 0x561c11febac0 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 -- 172.21.15.59:0/608059862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 msgr2=0x561c11ff5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 0x561c11ff5770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c09bbb0 tx=0x7f6b8c0d5b70 comp rx=0 tx=0).stop 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 -- 172.21.15.59:0/608059862 shutdown_connections 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11ff6ca0 0x561c1200c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11fec3e0 0x561c1200bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/608059862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11febac0 0x561c11ff5770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 -- 172.21.15.59:0/608059862 >> 172.21.15.59:0/608059862 conn(0x561c11fe8050 msgr2=0x561c11ff5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 -- 172.21.15.59:0/608059862 shutdown_connections 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.500+0000 7f6ba4aae740 1 -- 172.21.15.59:0/608059862 wait complete. 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.501+0000 7f6ba4aae740 1 -- start start 2024-09-05T12:05:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.501+0000 7f6ba4aae740 1 Processor -- start 2024-09-05T12:05:22.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 0x561c12065920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 0x561c12065e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b4250 con 0x561c11fec3e0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b56c0 con 0x561c11ff6ca0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b4920 con 0x561c11febac0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6ba4aae740 10 client.0 _ll_register_callbacks cb 0x7f6b8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 0x561c12065920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43996/0 (socket says 172.21.15.59:43996) 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9b400640 1 -- 172.21.15.59:0/2544278753 learned_addr learned my addr 172.21.15.59:0/2544278753 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9be00640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 0x561c12065e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 -- 172.21.15.59:0/2544278753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 msgr2=0x561c120653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 -- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 msgr2=0x561c12065e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 0x561c12065e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9aa00640 1 -- 172.21.15.59:0/2544278753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c120706b0 con 0x561c11fec3e0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9b400640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.502+0000 7f6b9be00640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 0x561c12065e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6b9aa00640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 0x561c12065920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x561c11ff4e40 tx=0x7f6b88004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d59bd0aa7d7103 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6b84c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88005020 con 0x561c11fec3e0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6b880051c0 con 0x561c11fec3e0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c0d5320 con 0x561c11fec3e0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6b8802f050 con 0x561c11fec3e0 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6ba4aae740 10 client.16572 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:22.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.503+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c12071a30 con 0x561c11fec3e0 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c0d5320 con 0x561c11fec3e0 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6b88020410 con 0x561c11fec3e0 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 10 client.16572 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 20 client.16572 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c120721d0 0x561c120745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 4 client.16572 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c11ff40d0 con 0x561c120721d0 2024-09-05T12:05:22.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.504+0000 7f6b9b400640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c120721d0 0x561c120745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:22.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.505+0000 7f6b9b400640 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c120721d0 0x561c120745f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449fb90691db1c5e server_cookie=771975647470a25 in_seq=0 out_seq=0 2024-09-05T12:05:22.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.505+0000 7f6b84c00640 10 client.16572 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:22.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6b84c00640 1 -- 172.21.15.59:0/2544278753 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c11ff40d0 con 0x561c120721d0 2024-09-05T12:05:22.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6b84c00640 10 client.16572 handle_command_reply: tid=0 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 client.16572 shutdown 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 msgr2=0x561c12065920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 0x561c12065920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x561c11ff4e40 tx=0x7f6b88004260 comp rx=0 tx=0).stop 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 shutdown_connections 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c11ff6ca0 0x561c12065e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561c120721d0 0x561c120745f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c11fec3e0 0x561c12065920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 --2- 172.21.15.59:0/2544278753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c11febac0 0x561c120653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6b84c00640 0 client.16572 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.506+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 >> 172.21.15.59:0/2544278753 conn(0x561c11fe8050 msgr2=0x561c11fe71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 shutdown_connections 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba4aae740 1 -- 172.21.15.59:0/2544278753 wait complete. 2024-09-05T12:05:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba4aae740 20 client.16572 trim_cache size 0 max 16384 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "36ef12ed-6409-4b1e-8c8e-50e24c36ab43", 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 msgr2=0x7f6b800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 0x7f6b800a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b94005e30 tx=0x7f6b94005d80 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 msgr2=0x7f6b9c1a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00e9f0 tx=0x7f6b8c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 shutdown_connections 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.507+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6b800a2c00 0x7f6b800a50c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c112a30 0x7f6b9c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b9c077040 0x7f6b9c1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 --2- 172.21.15.59:0/4202073304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b9c076720 0x7f6b9c1a46e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 >> 172.21.15.59:0/4202073304 conn(0x7f6b9c0ffa90 msgr2=0x7f6b9c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 shutdown_connections 2024-09-05T12:05:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:22.508+0000 7f6ba2000640 1 -- 172.21.15.59:0/4202073304 wait complete. 2024-09-05T12:05:22.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:22 smithi189 ceph-mon[34772]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:05:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:23 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:05:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:23 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:23 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:05:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:23 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:05:23.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:23 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:05:23.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:23 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:23 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:05:23.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:23 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:23 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:05:23.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:23 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:23 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:05:23.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:23 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:05:23.767 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:24.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:24 smithi059 ceph-mon[30025]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-05T12:05:24.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 -- 172.21.15.59:0/422242834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48103e30 msgr2=0x7efd48104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 --2- 172.21.15.59:0/422242834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48103e30 0x7efd48104210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009900 tx=0x7efd3c02a410 comp rx=0 tx=0).stop 2024-09-05T12:05:24.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 -- 172.21.15.59:0/422242834 shutdown_connections 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 --2- 172.21.15.59:0/422242834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd48108f60 0x7efd48110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 --2- 172.21.15.59:0/422242834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd481047e0 0x7efd48108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 --2- 172.21.15.59:0/422242834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48103e30 0x7efd48104210 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 -- 172.21.15.59:0/422242834 >> 172.21.15.59:0/422242834 conn(0x7efd480ff8d0 msgr2=0x7efd48101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 -- 172.21.15.59:0/422242834 shutdown_connections 2024-09-05T12:05:24.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.302+0000 7efd4de00640 1 -- 172.21.15.59:0/422242834 wait complete. 2024-09-05T12:05:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.303+0000 7efd4de00640 1 Processor -- start 2024-09-05T12:05:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.303+0000 7efd4de00640 1 -- start start 2024-09-05T12:05:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd481047e0 0x7efd4819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48108f60 0x7efd4819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd48113fd0 con 0x7efd48108f60 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd48113e50 con 0x7efd48103e30 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd48114150 con 0x7efd481047e0 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53056/0 (socket says 172.21.15.59:53056) 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4ca00640 1 -- 172.21.15.59:0/108747328 learned_addr learned my addr 172.21.15.59:0/108747328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd4d400640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48108f60 0x7efd4819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.304+0000 7efd47e00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd481047e0 0x7efd4819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 -- 172.21.15.59:0/108747328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd481047e0 msgr2=0x7efd4819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd481047e0 0x7efd4819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 -- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48108f60 msgr2=0x7efd4819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48108f60 0x7efd4819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 -- 172.21.15.59:0/108747328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4819f4c0 con 0x7efd48103e30 2024-09-05T12:05:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4ca00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd3c004770 tx=0x7efd3c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd45600640 1 -- 172.21.15.59:0/108747328 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c03f070 con 0x7efd48103e30 2024-09-05T12:05:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd45600640 1 -- 172.21.15.59:0/108747328 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3c031ca0 con 0x7efd48103e30 2024-09-05T12:05:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4819f750 con 0x7efd48103e30 2024-09-05T12:05:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd45600640 1 -- 172.21.15.59:0/108747328 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c035030 con 0x7efd48103e30 2024-09-05T12:05:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.305+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd481a7090 con 0x7efd48103e30 2024-09-05T12:05:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.308+0000 7efd45600640 1 -- 172.21.15.59:0/108747328 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd3c042070 con 0x7efd48103e30 2024-09-05T12:05:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 Processor -- start 2024-09-05T12:05:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 -- start start 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1dfe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e01320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5591f1d5a3e0 con 0x5591f1df4670 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5591f1d01d80 con 0x5591f1df3d50 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd51dba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591f1d563a0 con 0x5591f1dfef30 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.309+0000 7efd4ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1dfe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53058/0 (socket says 172.21.15.59:53058) 2024-09-05T12:05:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 -- 172.21.15.59:0/2121959481 learned_addr learned my addr 172.21.15.59:0/2121959481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd47e00640 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e01320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 -- 172.21.15.59:0/2121959481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 msgr2=0x5591f1e01320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e01320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 -- 172.21.15.59:0/2121959481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 msgr2=0x5591f1dfe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1dfe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 -- 172.21.15.59:0/2121959481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f1d20ae0 con 0x5591f1df3d50 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.310+0000 7efd4d400640 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd40009f90 tx=0x7efd40010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e44c2d5efa5631 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd39c00640 1 -- 172.21.15.59:0/2121959481 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40011700 con 0x5591f1df3d50 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd39c00640 1 -- 172.21.15.59:0/2121959481 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40011d80 con 0x5591f1df3d50 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd39c00640 1 -- 172.21.15.59:0/2121959481 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40018380 con 0x5591f1df3d50 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd45600640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 0x7efd300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd45600640 1 -- 172.21.15.59:0/108747328 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd3c0ef740 con 0x7efd48103e30 2024-09-05T12:05:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd4ca00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 0x7efd300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 -- 172.21.15.59:0/2121959481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 msgr2=0x5591f1df4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd40009f90 tx=0x7efd40010860 comp rx=0 tx=0).stop 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 -- 172.21.15.59:0/2121959481 shutdown_connections 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e01320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1dfe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 --2- 172.21.15.59:0/2121959481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1df4130 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.311+0000 7efd51dba740 1 -- 172.21.15.59:0/2121959481 >> 172.21.15.59:0/2121959481 conn(0x5591f1df02e0 msgr2=0x5591f1df06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.312+0000 7efd51dba740 1 -- 172.21.15.59:0/2121959481 shutdown_connections 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.312+0000 7efd51dba740 1 -- 172.21.15.59:0/2121959481 wait complete. 2024-09-05T12:05:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.312+0000 7efd4ca00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 0x7efd300a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b8040 tx=0x7efd3c0f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.312+0000 7efd51dba740 1 -- start start 2024-09-05T12:05:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.312+0000 7efd51dba740 1 Processor -- start 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1dfbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1dfc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5591f1e01ba0 con 0x5591f1dfef30 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5591f1d150f0 con 0x5591f1df3d50 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591f1e01d20 con 0x5591f1df4670 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd51dba740 10 client.0 _ll_register_callbacks cb 0x5591f1d069e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1dfbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1dfbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53060/0 (socket says 172.21.15.59:53060) 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd47e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1dfc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd4d400640 1 -- 172.21.15.59:0/2177874728 learned_addr learned my addr 172.21.15.59:0/2177874728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.313+0000 7efd4d400640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 -- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 msgr2=0x5591f1dfbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1dfbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 -- 172.21.15.59:0/2177874728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 msgr2=0x5591f1dfc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 -- 172.21.15.59:0/2177874728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f1acb3c0 con 0x5591f1df4670 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd4d400640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd47e00640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1dfc390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efd340126d0 tx=0x7efd34012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd5bd18f38ed34f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd2e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400f920 con 0x5591f1df4670 2024-09-05T12:05:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd40011d80 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd3400fac0 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.314+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd3400fcc0 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 10 client.26515 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5591f1e0d960 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40011d80 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd3401c420 con 0x5591f1df4670 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 10 client.26515 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 10 client.26515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 20 client.26515 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e0ee10 0x5591f1e11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 4 client.26515 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.315+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f1d67950 con 0x5591f1e0ee10 2024-09-05T12:05:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.316+0000 7efd4d400640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e0ee10 0x5591f1e11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.316+0000 7efd4d400640 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e0ee10 0x5591f1e11230 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301c24384eac167b server_cookie=c56cc640bae7ab24 in_seq=0 out_seq=0 2024-09-05T12:05:24.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.316+0000 7efd2e000640 10 client.26515 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.317+0000 7efd2e000640 1 -- 172.21.15.59:0/2177874728 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591f1d67950 con 0x5591f1e0ee10 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.317+0000 7efd2e000640 10 client.26515 handle_command_reply: tid=0 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.317+0000 7efd51dba740 1 client.26515 shutdown 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 msgr2=0x5591f1dfc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1dfc390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efd340126d0 tx=0x7efd34012b90 comp rx=0 tx=0).stop 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 shutdown_connections 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e0ee10 0x5591f1e11230 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1dfc390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd2e000640 0 client.26515 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 --2- 172.21.15.59:0/2177874728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1dfbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 >> 172.21.15.59:0/2177874728 conn(0x5591f1df02e0 msgr2=0x5591f1df6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 shutdown_connections 2024-09-05T12:05:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 1 -- 172.21.15.59:0/2177874728 wait complete. 2024-09-05T12:05:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.318+0000 7efd51dba740 20 client.26515 trim_cache size 0 max 16384 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.336+0000 7efd51dba740 1 Processor -- start 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.336+0000 7efd51dba740 1 -- start start 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.336+0000 7efd51dba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e16ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.336+0000 7efd51dba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1e17020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.336+0000 7efd51dba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd51dba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd481aeef0 con 0x5591f1dfef30 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd51dba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b1770 con 0x5591f1df3d50 2024-09-05T12:05:24.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd51dba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd481af560 con 0x5591f1df4670 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39258/0 (socket says 172.21.15.59:39258) 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 -- 172.21.15.59:0/2088182708 learned_addr learned my addr 172.21.15.59:0/2088182708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd47e00640 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1e17020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e16ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 -- 172.21.15.59:0/2088182708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 msgr2=0x5591f1e17020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1e17020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 -- 172.21.15.59:0/2088182708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 msgr2=0x5591f1e16ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e16ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.337+0000 7efd4d400640 1 -- 172.21.15.59:0/2088182708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f1df6bc0 con 0x5591f1dfef30 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd4d400640 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd40009d50 tx=0x7efd400096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5646d910b18fd103 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd39200640 1 -- 172.21.15.59:0/2088182708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40019b60 con 0x5591f1dfef30 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd39200640 1 -- 172.21.15.59:0/2088182708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40020040 con 0x5591f1dfef30 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd39200640 1 -- 172.21.15.59:0/2088182708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40005300 con 0x5591f1dfef30 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 -- 172.21.15.59:0/2088182708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 msgr2=0x5591f1dfd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd40009d50 tx=0x7efd400096a0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 -- 172.21.15.59:0/2088182708 shutdown_connections 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1dfef30 0x5591f1dfd6b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1df4670 0x5591f1e17020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 --2- 172.21.15.59:0/2088182708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e16ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.338+0000 7efd51dba740 1 -- 172.21.15.59:0/2088182708 >> 172.21.15.59:0/2088182708 conn(0x5591f1df02e0 msgr2=0x5591f1dfaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.339+0000 7efd51dba740 1 -- 172.21.15.59:0/2088182708 shutdown_connections 2024-09-05T12:05:24.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.339+0000 7efd51dba740 1 -- 172.21.15.59:0/2088182708 wait complete. 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.339+0000 7efd51dba740 1 -- start start 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.339+0000 7efd51dba740 1 Processor -- start 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd481afc30 con 0x5591f1df4670 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b10a0 con 0x5591f1df3d50 2024-09-05T12:05:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b0300 con 0x5591f1dfef30 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd51dba740 10 client.0 _ll_register_callbacks cb 0x7efd34009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd4d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39268/0 (socket says 172.21.15.59:39268) 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 -- 172.21.15.59:0/202467158 learned_addr learned my addr 172.21.15.59:0/202467158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd4d400640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 -- 172.21.15.59:0/202467158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 msgr2=0x5591f1e026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 -- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 msgr2=0x5591f1e072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.340+0000 7efd47e00640 1 -- 172.21.15.59:0/202467158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f1e02f20 con 0x5591f1df4670 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd47e00640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5591f1dfb6e0 tx=0x7efd3401c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596b19e327364895 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd2e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3402f060 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd40020040 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd3401cd10 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd3400fe60 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd51dba740 10 client.16590 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.341+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5591f1e796c0 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40020040 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd34003ab0 con 0x5591f1df4670 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 10 client.16590 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 20 client.16590 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e79de0 0x5591f1e7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 4 client.16590 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591f1d61e60 con 0x5591f1e79de0 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.342+0000 7efd4d400640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e79de0 0x5591f1e7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd4d400640 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e79de0 0x5591f1e7c200 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227e072e761b859f server_cookie=cf4623b44e8f14c7 in_seq=0 out_seq=0 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd2e000640 10 client.16590 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd2e000640 1 -- 172.21.15.59:0/202467158 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5591f1d61e60 con 0x5591f1e79de0 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd2e000640 10 client.16590 handle_command_reply: tid=0 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 client.16590 shutdown 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 msgr2=0x5591f1e07830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5591f1dfb6e0 tx=0x7efd3401c6d0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 shutdown_connections 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f1dfef30 0x5591f1e026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5591f1e79de0 0x5591f1e7c200 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5591f1df4670 0x5591f1e07830 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 --2- 172.21.15.59:0/202467158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5591f1df3d50 0x5591f1e072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd2e000640 0 client.16590 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.343+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 >> 172.21.15.59:0/202467158 conn(0x5591f1df02e0 msgr2=0x5591f1deeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 shutdown_connections 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd51dba740 1 -- 172.21.15.59:0/202467158 wait complete. 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd51dba740 20 client.16590 trim_cache size 0 max 16384 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:24.349 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: "36ef12ed-6409-4b1e-8c8e-50e24c36ab43": { 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "36ef12ed-6409-4b1e-8c8e-50e24c36ab43", 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:24.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 msgr2=0x7efd300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 0x7efd300a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b8040 tx=0x7efd3c0f5530 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 msgr2=0x7efd4819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.344+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd3c004770 tx=0x7efd3c003c10 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 shutdown_connections 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd300a2c00 0x7efd300a50c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48108f60 0x7efd4819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd481047e0 0x7efd4819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 --2- 172.21.15.59:0/108747328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd48103e30 0x7efd4819a510 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 >> 172.21.15.59:0/108747328 conn(0x7efd480ff8d0 msgr2=0x7efd48100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 shutdown_connections 2024-09-05T12:05:24.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:24.345+0000 7efd4de00640 1 -- 172.21.15.59:0/108747328 wait complete. 2024-09-05T12:05:24.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:24 smithi089 ceph-mon[34499]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-05T12:05:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:24 smithi189 ceph-mon[34772]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-05T12:05:25.159 INFO:tasks.cephfs.filesystem:scrub status for tag:36ef12ed-6409-4b1e-8c8e-50e24c36ab43 - {'path': '/', 'tag': '36ef12ed-6409-4b1e-8c8e-50e24c36ab43', 'options': 'recursive,force'} 2024-09-05T12:05:25.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:25 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:05:25.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:25 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:05:25.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:25 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:05:26.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:26 smithi089 ceph-mon[34499]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 KiB/s wr, 5 op/s 2024-09-05T12:05:26.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:26 smithi059 ceph-mon[30025]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 KiB/s wr, 5 op/s 2024-09-05T12:05:26.490 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:26 smithi189 ceph-mon[34772]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.7 KiB/s wr, 5 op/s 2024-09-05T12:05:28.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:28 smithi089 ceph-mon[34499]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:05:28.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:28 smithi059 ceph-mon[30025]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:05:28.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:28 smithi189 ceph-mon[34772]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:05:30.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:30 smithi089 ceph-mon[34499]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:30.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:30 smithi059 ceph-mon[30025]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:30.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:30 smithi189 ceph-mon[34772]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:32.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:32 smithi089 ceph-mon[34499]: pgmap v630: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:32.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:32 smithi059 ceph-mon[30025]: pgmap v630: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:32.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:32 smithi189 ceph-mon[34772]: pgmap v630: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:34.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:34 smithi089 ceph-mon[34499]: pgmap v631: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:34.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:34 smithi059 ceph-mon[30025]: pgmap v631: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:34 smithi189 ceph-mon[34772]: pgmap v631: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:05:35.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:35.952 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:36.423 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:36 smithi059 ceph-mon[30025]: pgmap v632: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:36.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:36 smithi089 ceph-mon[34499]: pgmap v632: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 -- 172.21.15.59:0/2076628761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 msgr2=0x7fca1c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 --2- 172.21.15.59:0/2076628761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c1089f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fca04009900 tx=0x7fca0402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 -- 172.21.15.59:0/2076628761 shutdown_connections 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 --2- 172.21.15.59:0/2076628761 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c109120 0x7fca1c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 --2- 172.21.15.59:0/2076628761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c1089f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 --2- 172.21.15.59:0/2076628761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c103ff0 0x7fca1c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 -- 172.21.15.59:0/2076628761 >> 172.21.15.59:0/2076628761 conn(0x7fca1c0ffa90 msgr2=0x7fca1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.533+0000 7fca21200640 1 -- 172.21.15.59:0/2076628761 shutdown_connections 2024-09-05T12:05:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.534+0000 7fca21200640 1 -- 172.21.15.59:0/2076628761 wait complete. 2024-09-05T12:05:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.534+0000 7fca21200640 1 Processor -- start 2024-09-05T12:05:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 -- start start 2024-09-05T12:05:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 0x7fca1c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c109120 0x7fca1c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c114140 con 0x7fca1c1049a0 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c113fc0 con 0x7fca1c103ff0 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca21200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1142c0 con 0x7fca1c109120 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca11400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca19e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 0x7fca1c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca11400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53378/0 (socket says 172.21.15.59:53378) 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.535+0000 7fca11400640 1 -- 172.21.15.59:0/2723409148 learned_addr learned my addr 172.21.15.59:0/2723409148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca1a800640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c109120 0x7fca1c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 -- 172.21.15.59:0/2723409148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c109120 msgr2=0x7fca1c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c109120 0x7fca1c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 -- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 msgr2=0x7fca1c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 -- 172.21.15.59:0/2723409148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c19f5d0 con 0x7fca1c103ff0 2024-09-05T12:05:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca11400640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca19e00640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 0x7fca1c19a620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d720 tx=0x7fca0c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.536+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c017070 con 0x7fca1c103ff0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.537+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c19f8c0 con 0x7fca1c103ff0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.537+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c00de30 con 0x7fca1c103ff0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.537+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0047a0 con 0x7fca1c103ff0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.537+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1a71a0 con 0x7fca1c103ff0 2024-09-05T12:05:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.538+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1c1043d0 con 0x7fca1c103ff0 2024-09-05T12:05:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.540+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca0c0099d0 con 0x7fca1c103ff0 2024-09-05T12:05:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.541+0000 7fca13400640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 0x7fca000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.541+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca0c0cf270 con 0x7fca1c103ff0 2024-09-05T12:05:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.541+0000 7fca11400640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 0x7fca000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.542+0000 7fca11400640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 0x7fca000a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fca04031550 tx=0x7fca04005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.543+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca0c020020 con 0x7fca1c103ff0 2024-09-05T12:05:36.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:36 smithi189 ceph-mon[34772]: pgmap v632: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-05T12:05:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.763+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca1c10c360 con 0x7fca1c103ff0 2024-09-05T12:05:36.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.765+0000 7fca13400640 1 -- 172.21.15.59:0/2723409148 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fca0c09bd60 con 0x7fca1c103ff0 2024-09-05T12:05:36.767 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:36.767 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:36.768 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 msgr2=0x7fca000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 0x7fca000a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fca04031550 tx=0x7fca04005a20 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 msgr2=0x7fca1c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 0x7fca1c19a620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d720 tx=0x7fca0c00dbe0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 shutdown_connections 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca000a2b80 0x7fca000a5040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c109120 0x7fca1c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca1c1049a0 0x7fca1c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 --2- 172.21.15.59:0/2723409148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca1c103ff0 0x7fca1c19a620 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 >> 172.21.15.59:0/2723409148 conn(0x7fca1c0ffa90 msgr2=0x7fca1c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 shutdown_connections 2024-09-05T12:05:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:36.767+0000 7fca21200640 1 -- 172.21.15.59:0/2723409148 wait complete. 2024-09-05T12:05:37.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:37 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2723409148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:37.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:37 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2723409148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:37.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:37 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2723409148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:37.609 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id b8c9f4a --release --await' 2024-09-05T12:05:37.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id b8c9f4a --release --await 2024-09-05T12:05:37.986 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:38.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:38 smithi059 ceph-mon[30025]: pgmap v633: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:38.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:38 smithi089 ceph-mon[34499]: pgmap v633: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 -- 172.21.15.59:0/3545762565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24481049a0 msgr2=0x7f24481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 --2- 172.21.15.59:0/3545762565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24481049a0 0x7f24481089f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2434009860 tx=0x7f243402a340 comp rx=0 tx=0).stop 2024-09-05T12:05:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 -- 172.21.15.59:0/3545762565 shutdown_connections 2024-09-05T12:05:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 --2- 172.21.15.59:0/3545762565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2448109120 0x7f2448110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 --2- 172.21.15.59:0/3545762565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24481049a0 0x7f24481089f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 --2- 172.21.15.59:0/3545762565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448103ff0 0x7f24481043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.531+0000 7f244d000640 1 -- 172.21.15.59:0/3545762565 >> 172.21.15.59:0/3545762565 conn(0x7f24480ffa90 msgr2=0x7f2448101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.532+0000 7f244d000640 1 -- 172.21.15.59:0/3545762565 shutdown_connections 2024-09-05T12:05:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.532+0000 7f244d000640 1 -- 172.21.15.59:0/3545762565 wait complete. 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 Processor -- start 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 -- start start 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24481049a0 0x7f244819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448109120 0x7f244819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2448114140 con 0x7f24481049a0 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2448113fc0 con 0x7f2448103ff0 2024-09-05T12:05:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f244d000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24481142c0 con 0x7f2448109120 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.533+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42670/0 (socket says 172.21.15.59:42670) 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 -- 172.21.15.59:0/187038266 learned_addr learned my addr 172.21.15.59:0/187038266 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445400640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24481049a0 0x7f244819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2446800640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448109120 0x7f244819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 -- 172.21.15.59:0/187038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448109120 msgr2=0x7f244819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448109120 0x7f244819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 -- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24481049a0 msgr2=0x7f244819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24481049a0 0x7f244819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 -- 172.21.15.59:0/187038266 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244819f5e0 con 0x7f2448103ff0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.534+0000 7f2445e00640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f243800e9f0 tx=0x7f243800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.535+0000 7f243ea00640 1 -- 172.21.15.59:0/187038266 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243800cd80 con 0x7f2448103ff0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.535+0000 7f243ea00640 1 -- 172.21.15.59:0/187038266 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438005500 con 0x7f2448103ff0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.535+0000 7f243ea00640 1 -- 172.21.15.59:0/187038266 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438010690 con 0x7f2448103ff0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.535+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244819f8d0 con 0x7f2448103ff0 2024-09-05T12:05:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.535+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481a71b0 con 0x7f2448103ff0 2024-09-05T12:05:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 Processor -- start 2024-09-05T12:05:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f243ea00640 1 -- 172.21.15.59:0/187038266 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24380046c0 con 0x7f2448103ff0 2024-09-05T12:05:38.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 -- start start 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78c310 0x55efdb78c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb7974f0 0x55efdb7998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55efdb6f2650 con 0x55efdb78cc30 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55efdb699f10 con 0x55efdb7974f0 2024-09-05T12:05:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.538+0000 7f244fc44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55efdb6ee610 con 0x55efdb78c310 2024-09-05T12:05:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2446800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78c310 0x55efdb78c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53412/0 (socket says 172.21.15.59:53412) 2024-09-05T12:05:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 -- 172.21.15.59:0/399690237 learned_addr learned my addr 172.21.15.59:0/399690237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445400640 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb7974f0 0x55efdb7998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 -- 172.21.15.59:0/399690237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78c310 msgr2=0x55efdb78c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78c310 0x55efdb78c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 -- 172.21.15.59:0/399690237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb7974f0 msgr2=0x55efdb7998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb7974f0 0x55efdb7998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.539+0000 7f2445e00640 1 -- 172.21.15.59:0/399690237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efdb401580 con 0x55efdb78cc30 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f2445e00640 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f24380d5a90 tx=0x7f24380d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67adc311630e0a27 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f243ea00640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 0x7f242c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f243ea00640 1 -- 172.21.15.59:0/187038266 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24380cf850 con 0x7f2448103ff0 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f2446800640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 0x7f242c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f2433400640 1 -- 172.21.15.59:0/399690237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380de070 con 0x55efdb78cc30 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f2433400640 1 -- 172.21.15.59:0/399690237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243809c340 con 0x55efdb78cc30 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.540+0000 7f2433400640 1 -- 172.21.15.59:0/399690237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243809b780 con 0x55efdb78cc30 2024-09-05T12:05:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 -- 172.21.15.59:0/399690237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 msgr2=0x55efdb796fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f2446800640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 0x7f242c0a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440010040 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f24380d5a90 tx=0x7f24380d5b80 comp rx=0 tx=0).stop 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 -- 172.21.15.59:0/399690237 shutdown_connections 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb7974f0 0x55efdb7998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78cc30 0x55efdb796fb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 --2- 172.21.15.59:0/399690237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78c310 0x55efdb78c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.541+0000 7f244fc44740 1 -- 172.21.15.59:0/399690237 >> 172.21.15.59:0/399690237 conn(0x55efdb7888a0 msgr2=0x55efdb788c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.542+0000 7f244fc44740 1 -- 172.21.15.59:0/399690237 shutdown_connections 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.542+0000 7f244fc44740 1 -- 172.21.15.59:0/399690237 wait complete. 2024-09-05T12:05:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.542+0000 7f244fc44740 1 -- start start 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.542+0000 7f244fc44740 1 Processor -- start 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78c310 0x55efdb7943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb7974f0 0x55efdb794e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55efdb79a0c0 con 0x55efdb78c310 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55efdb6ad350 con 0x55efdb78cc30 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55efdb79a240 con 0x55efdb7974f0 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f244fc44740 10 client.0 _ll_register_callbacks cb 0x55efdb6d0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f2445400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f2445e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78c310 0x55efdb7943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f2445400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42686/0 (socket says 172.21.15.59:42686) 2024-09-05T12:05:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f2445400640 1 -- 172.21.15.59:0/2232906740 learned_addr learned my addr 172.21.15.59:0/2232906740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.543+0000 7f2446800640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb7974f0 0x55efdb794e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 -- 172.21.15.59:0/2232906740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb7974f0 msgr2=0x55efdb794e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb7974f0 0x55efdb794e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 -- 172.21.15.59:0/2232906740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78c310 msgr2=0x55efdb7943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78c310 0x55efdb7943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 -- 172.21.15.59:0/2232906740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efdb4cb960 con 0x55efdb78cc30 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2445400640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2434009780 tx=0x7f24340303d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5bbf6921784b365 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2427e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434037040 con 0x55efdb78cc30 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243809c340 con 0x55efdb78cc30 2024-09-05T12:05:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.544+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2434030550 con 0x55efdb78cc30 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f243404c050 con 0x55efdb78cc30 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 10 client.26603 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55efdb7a5ce0 con 0x55efdb78cc30 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243809c340 con 0x55efdb78cc30 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f243403d450 con 0x55efdb78cc30 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 10 client.26603 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 10 client.26603 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 20 client.26603 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.545+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb7a70f0 0x55efdb7a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.546+0000 7f244fc44740 4 client.26603 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.546+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efdb6ffbc0 con 0x55efdb7a70f0 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.546+0000 7f2445e00640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb7a70f0 0x55efdb7a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.546+0000 7f2445e00640 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb7a70f0 0x55efdb7a9540 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98985d98142cf78 server_cookie=48ad733444183a2d in_seq=0 out_seq=0 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.546+0000 7f2427e00640 10 client.26603 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f2427e00640 1 -- 172.21.15.59:0/2232906740 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efdb6ffbc0 con 0x55efdb7a70f0 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f2427e00640 10 client.26603 handle_command_reply: tid=0 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 client.26603 shutdown 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 msgr2=0x55efdb7948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2434009780 tx=0x7f24340303d0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 shutdown_connections 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb7974f0 0x55efdb794e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78cc30 0x55efdb7948e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb7a70f0 0x55efdb7a9540 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 --2- 172.21.15.59:0/2232906740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb78c310 0x55efdb7943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f2427e00640 0 client.26603 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:38.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.547+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 >> 172.21.15.59:0/2232906740 conn(0x55efdb7888a0 msgr2=0x55efdb787aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.548+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 shutdown_connections 2024-09-05T12:05:38.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.548+0000 7f244fc44740 1 -- 172.21.15.59:0/2232906740 wait complete. 2024-09-05T12:05:38.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.548+0000 7f244fc44740 20 client.26603 trim_cache size 0 max 16384 2024-09-05T12:05:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 Processor -- start 2024-09-05T12:05:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 -- start start 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb7aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb7af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24481af130 con 0x55efdb7974f0 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b19b0 con 0x55efdb78c310 2024-09-05T12:05:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.566+0000 7f244fc44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24481af7a0 con 0x55efdb78cc30 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42698/0 (socket says 172.21.15.59:42698) 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 -- 172.21.15.59:0/901779998 learned_addr learned my addr 172.21.15.59:0/901779998 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445400640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb7aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2446800640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb7af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 -- 172.21.15.59:0/901779998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 msgr2=0x55efdb7aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb7aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 -- 172.21.15.59:0/901779998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 msgr2=0x55efdb7af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb7af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 -- 172.21.15.59:0/901779998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efdb788340 con 0x55efdb78c310 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2446800640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb7af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.567+0000 7f2445e00640 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24380980a0 tx=0x7f243809ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4dee71398e22be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f2432a00640 1 -- 172.21.15.59:0/901779998 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380de040 con 0x55efdb78c310 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f2432a00640 1 -- 172.21.15.59:0/901779998 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243809c340 con 0x55efdb78c310 2024-09-05T12:05:38.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f2432a00640 1 -- 172.21.15.59:0/901779998 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243809b5a0 con 0x55efdb78c310 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 -- 172.21.15.59:0/901779998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 msgr2=0x55efdb796200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24380980a0 tx=0x7f243809ba10 comp rx=0 tx=0).stop 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 -- 172.21.15.59:0/901779998 shutdown_connections 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb7af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb7aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 --2- 172.21.15.59:0/901779998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb796200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.568+0000 7f244fc44740 1 -- 172.21.15.59:0/901779998 >> 172.21.15.59:0/901779998 conn(0x55efdb7888a0 msgr2=0x55efdb795af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.569+0000 7f244fc44740 1 -- 172.21.15.59:0/901779998 shutdown_connections 2024-09-05T12:05:38.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.569+0000 7f244fc44740 1 -- 172.21.15.59:0/901779998 wait complete. 2024-09-05T12:05:38.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.569+0000 7f244fc44740 1 -- start start 2024-09-05T12:05:38.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.569+0000 7f244fc44740 1 Processor -- start 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb79b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb79bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24481afe70 con 0x55efdb7974f0 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b12e0 con 0x55efdb78c310 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b0540 con 0x55efdb78cc30 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f244fc44740 10 client.0 _ll_register_callbacks cb 0x7f2438005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f2445e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42704/0 (socket says 172.21.15.59:42704) 2024-09-05T12:05:38.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.570+0000 7f2445e00640 1 -- 172.21.15.59:0/196297321 learned_addr learned my addr 172.21.15.59:0/196297321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445400640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb79b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2446800640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb79bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 -- 172.21.15.59:0/196297321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 msgr2=0x55efdb79b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb79b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 -- 172.21.15.59:0/196297321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 msgr2=0x55efdb79bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb79bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 -- 172.21.15.59:0/196297321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efdb810de0 con 0x55efdb78c310 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2445e00640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243809e040 tx=0x7f2438009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adaad2e59d4f262b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2427e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380b8070 con 0x55efdb78c310 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2438099ce0 con 0x55efdb78c310 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.571+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243809c340 con 0x55efdb78c310 2024-09-05T12:05:38.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f24380de070 con 0x55efdb78c310 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 10 client.26609 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55efdb8120d0 con 0x55efdb78c310 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243809c340 con 0x55efdb78c310 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f24380b4070 con 0x55efdb78c310 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 10 client.26609 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 10 client.26609 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 20 client.26609 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb812350 0x55efdb814740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 4 client.26609 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b8c9f4a", "release": true, "await": true}] 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b8c9f4a", "release": true, "await": true}) -- 0x55efdb6fa0d0 con 0x55efdb812350 2024-09-05T12:05:38.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.572+0000 7f2445400640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb812350 0x55efdb814740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.573+0000 7f2445400640 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb812350 0x55efdb814740 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea25f30f1a90c05 server_cookie=f6f3514664f06153 in_seq=0 out_seq=0 2024-09-05T12:05:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.573+0000 7f2427e00640 10 client.26609 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.574+0000 7f2427e00640 1 -- 172.21.15.59:0/196297321 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55efdb6fa0d0 con 0x55efdb812350 2024-09-05T12:05:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.574+0000 7f2427e00640 10 client.26609 handle_command_reply: tid=0 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.574+0000 7f244fc44740 1 client.26609 shutdown 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.574+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 msgr2=0x55efdb79b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.574+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243809e040 tx=0x7f2438009310 comp rx=0 tx=0).stop 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 shutdown_connections 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55efdb812350 0x55efdb814740 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55efdb7974f0 0x55efdb79bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efdb78cc30 0x55efdb79b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 --2- 172.21.15.59:0/196297321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55efdb78c310 0x55efdb79b1b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 >> 172.21.15.59:0/196297321 conn(0x55efdb7888a0 msgr2=0x55efdb787aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f2427e00640 0 client.26609 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 shutdown_connections 2024-09-05T12:05:38.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 1 -- 172.21.15.59:0/196297321 wait complete. 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.575+0000 7f244fc44740 20 client.26609 trim_cache size 0 max 16384 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 28, 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "b8c9f4a": { 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 28, 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:05:38.579 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 146.2, 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:38.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:38.581 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 msgr2=0x7f242c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 0x7f242c0a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2440005fd0 tx=0x7f2440010040 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 msgr2=0x7f244819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f243800e9f0 tx=0x7f243800edf0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 shutdown_connections 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f242c0a2c00 0x7f242c0a50c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2448109120 0x7f244819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24481049a0 0x7f244819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 --2- 172.21.15.59:0/187038266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2448103ff0 0x7f244819a630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 >> 172.21.15.59:0/187038266 conn(0x7f24480ffa90 msgr2=0x7f2448100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 shutdown_connections 2024-09-05T12:05:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:38.576+0000 7f244d000640 1 -- 172.21.15.59:0/187038266 wait complete. 2024-09-05T12:05:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:38 smithi189 ceph-mon[34772]: pgmap v633: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:39.337 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b8c9f4a', total seconds elapsed: 29.9 2024-09-05T12:05:39.337 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-09-05T12:05:39.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 -- 172.21.15.59:0/3929922891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364112a50 msgr2=0x7fd364114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 --2- 172.21.15.59:0/3929922891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364112a50 0x7fd364114e40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009ca0 tx=0x7fd35c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 -- 172.21.15.59:0/3929922891 shutdown_connections 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 --2- 172.21.15.59:0/3929922891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364112a50 0x7fd364114e40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 --2- 172.21.15.59:0/3929922891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364077040 0x7fd364112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 --2- 172.21.15.59:0/3929922891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364076720 0x7fd364076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 -- 172.21.15.59:0/3929922891 >> 172.21.15.59:0/3929922891 conn(0x7fd3640ffa50 msgr2=0x7fd364101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 -- 172.21.15.59:0/3929922891 shutdown_connections 2024-09-05T12:05:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.297+0000 7fd36b200640 1 -- 172.21.15.59:0/3929922891 wait complete. 2024-09-05T12:05:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.298+0000 7fd36b200640 1 Processor -- start 2024-09-05T12:05:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.298+0000 7fd36b200640 1 -- start start 2024-09-05T12:05:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364077040 0x7fd36419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364112a50 0x7fd3641a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd364118600 con 0x7fd364076720 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd364118480 con 0x7fd364077040 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd36b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd364118780 con 0x7fd364112a50 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53448/0 (socket says 172.21.15.59:53448) 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd363e00640 1 -- 172.21.15.59:0/356495805 learned_addr learned my addr 172.21.15.59:0/356495805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd35b400640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364077040 0x7fd36419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.299+0000 7fd369200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364112a50 0x7fd3641a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 -- 172.21.15.59:0/356495805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364112a50 msgr2=0x7fd3641a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364112a50 0x7fd3641a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 -- 172.21.15.59:0/356495805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364077040 msgr2=0x7fd36419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364077040 0x7fd36419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 -- 172.21.15.59:0/356495805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3641a3bb0 con 0x7fd364076720 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd363e00640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd35400e9f0 tx=0x7fd35400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35400cd80 con 0x7fd364076720 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd354010910 con 0x7fd364076720 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3641a3ea0 con 0x7fd364076720 2024-09-05T12:05:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354018990 con 0x7fd364076720 2024-09-05T12:05:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.300+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641ab730 con 0x7fd364076720 2024-09-05T12:05:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.302+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd364076b00 con 0x7fd364076720 2024-09-05T12:05:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.303+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd354010ac0 con 0x7fd364076720 2024-09-05T12:05:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.305+0000 7fd361600640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 0x7fd3480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.305+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd3540cf7d0 con 0x7fd364076720 2024-09-05T12:05:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.305+0000 7fd35b400640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 0x7fd3480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.306+0000 7fd35b400640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 0x7fd3480a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd3640792f0 tx=0x7fd34c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.309+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd35409c2c0 con 0x7fd364076720 2024-09-05T12:05:40.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:40 smithi089 ceph-mon[34499]: pgmap v634: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:40.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:40 smithi059 ceph-mon[30025]: pgmap v634: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:40.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.531+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36410dc10 con 0x7fd364076720 2024-09-05T12:05:40.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.532+0000 7fd361600640 1 -- 172.21.15.59:0/356495805 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fd35409bbc0 con 0x7fd364076720 2024-09-05T12:05:40.534 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:40.534 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:40.534 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:40.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 msgr2=0x7fd3480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:40.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 0x7fd3480a50c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd3640792f0 tx=0x7fd34c005f90 comp rx=0 tx=0).stop 2024-09-05T12:05:40.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 msgr2=0x7fd36419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd35400e9f0 tx=0x7fd35400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 shutdown_connections 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd3480a2c00 0x7fd3480a50c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364112a50 0x7fd3641a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd364077040 0x7fd36419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 --2- 172.21.15.59:0/356495805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd364076720 0x7fd36419eb80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 >> 172.21.15.59:0/356495805 conn(0x7fd3640ffa50 msgr2=0x7fd364101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 shutdown_connections 2024-09-05T12:05:40.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:40.535+0000 7fd36b200640 1 -- 172.21.15.59:0/356495805 wait complete. 2024-09-05T12:05:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:40 smithi189 ceph-mon[34772]: pgmap v634: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-05T12:05:41.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:41 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/356495805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:41.295 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-09-05T12:05:41.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-09-05T12:05:41.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:41 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/356495805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:41.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:41 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/356495805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:41.673 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:42.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 -- 172.21.15.59:0/1833020561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414076720 msgr2=0x7fe414076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 --2- 172.21.15.59:0/1833020561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414076720 0x7fe414076b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe404009900 tx=0x7fe40402a410 comp rx=0 tx=0).stop 2024-09-05T12:05:42.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 -- 172.21.15.59:0/1833020561 shutdown_connections 2024-09-05T12:05:42.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 --2- 172.21.15.59:0/1833020561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 0x7fe414114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 --2- 172.21.15.59:0/1833020561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414077040 0x7fe4141124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 --2- 172.21.15.59:0/1833020561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414076720 0x7fe414076b00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 -- 172.21.15.59:0/1833020561 >> 172.21.15.59:0/1833020561 conn(0x7fe4140ffa90 msgr2=0x7fe414101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 -- 172.21.15.59:0/1833020561 shutdown_connections 2024-09-05T12:05:42.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.243+0000 7fe419600640 1 -- 172.21.15.59:0/1833020561 wait complete. 2024-09-05T12:05:42.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.244+0000 7fe419600640 1 Processor -- start 2024-09-05T12:05:42.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.244+0000 7fe419600640 1 -- start start 2024-09-05T12:05:42.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414076720 0x7fe4141a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 0x7fe41419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141185a0 con 0x7fe414077040 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe414118420 con 0x7fe414112a30 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe419600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe414118720 con 0x7fe414076720 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34484/0 (socket says 172.21.15.59:34484) 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe40a000640 1 -- 172.21.15.59:0/109293231 learned_addr learned my addr 172.21.15.59:0/109293231 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe412a00640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414076720 0x7fe4141a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.245+0000 7fe413400640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 0x7fe41419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 -- 172.21.15.59:0/109293231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414076720 msgr2=0x7fe4141a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414076720 0x7fe4141a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 -- 172.21.15.59:0/109293231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 msgr2=0x7fe41419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 0x7fe41419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 -- 172.21.15.59:0/109293231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41419efe0 con 0x7fe414077040 2024-09-05T12:05:42.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40a000640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e9f0 tx=0x7fe3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40be00640 1 -- 172.21.15.59:0/109293231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00cd80 con 0x7fe414077040 2024-09-05T12:05:42.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40be00640 1 -- 172.21.15.59:0/109293231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc010910 con 0x7fe414077040 2024-09-05T12:05:42.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41419f2d0 con 0x7fe414077040 2024-09-05T12:05:42.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.246+0000 7fe40be00640 1 -- 172.21.15.59:0/109293231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc018990 con 0x7fe414077040 2024-09-05T12:05:42.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.247+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141ab5c0 con 0x7fe414077040 2024-09-05T12:05:42.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.249+0000 7fe41bfc7740 1 Processor -- start 2024-09-05T12:05:42.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.249+0000 7fe40be00640 1 -- 172.21.15.59:0/109293231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3fc010ac0 con 0x7fe414077040 2024-09-05T12:05:42.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 -- start start 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5572c6e8b260 con 0x5572c6f26e00 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5572c6e32500 con 0x5572c6f316c0 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5572c6e87220 con 0x5572c6f264e0 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe412a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe412a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34496/0 (socket says 172.21.15.59:34496) 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe413400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.250+0000 7fe412a00640 1 -- 172.21.15.59:0/1936236257 learned_addr learned my addr 172.21.15.59:0/1936236257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe40a000640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 -- 172.21.15.59:0/1936236257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 msgr2=0x5572c6f33ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f33ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 -- 172.21.15.59:0/1936236257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 msgr2=0x5572c6f31180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 -- 172.21.15.59:0/1936236257 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c6bfcba0 con 0x5572c6f264e0 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe412a00640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe40a000640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe413400640 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f268c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe41407aa10 tx=0x7fe40c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f03cec84a0f9df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.251+0000 7fe40be00640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 0x7fe3f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe40be00640 1 -- 172.21.15.59:0/109293231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe3fc0cf7e0 con 0x7fe414077040 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe403400640 1 -- 172.21.15.59:0/1936236257 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c004d20 con 0x5572c6f264e0 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe412a00640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 0x7fe3f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe403400640 1 -- 172.21.15.59:0/1936236257 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c004050 con 0x5572c6f264e0 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe403400640 1 -- 172.21.15.59:0/1936236257 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c005280 con 0x5572c6f264e0 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1936236257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 msgr2=0x5572c6f268c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.252+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f268c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe41407aa10 tx=0x7fe40c011810 comp rx=0 tx=0).stop 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1936236257 shutdown_connections 2024-09-05T12:05:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f31180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1936236257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f268c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1936236257 >> 172.21.15.59:0/1936236257 conn(0x5572c6f22a70 msgr2=0x5572c6f22e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1936236257 shutdown_connections 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe412a00640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 0x7fe3f80a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe4040058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1936236257 wait complete. 2024-09-05T12:05:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.253+0000 7fe41bfc7740 1 -- start start 2024-09-05T12:05:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 Processor -- start 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f2eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5572c6f34290 con 0x5572c6f26e00 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5572c6e45bd0 con 0x5572c6f316c0 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5572c6f34410 con 0x5572c6f264e0 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe41bfc7740 10 client.0 _ll_register_callbacks cb 0x5572c6b9a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe412a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f2eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe412a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:37594/0 (socket says 172.21.15.59:37594) 2024-09-05T12:05:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe412a00640 1 -- 172.21.15.59:0/1179772102 learned_addr learned my addr 172.21.15.59:0/1179772102 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.254+0000 7fe413400640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 -- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 msgr2=0x5572c6f2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f2f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 -- 172.21.15.59:0/1179772102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 msgr2=0x5572c6f2eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f2eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 -- 172.21.15.59:0/1179772102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c6f3d750 con 0x5572c6f264e0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe412a00640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe404004770 tx=0x7fe404009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3a6a67795928c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe3f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.255+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40404e070 con 0x5572c6f264e0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40c004050 con 0x5572c6f264e0 2024-09-05T12:05:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe4040037f0 con 0x5572c6f264e0 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe40405a050 con 0x5572c6f264e0 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe41bfc7740 10 client.26563 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5572c6f40130 con 0x5572c6f264e0 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c004050 con 0x5572c6f264e0 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.256+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe40404abe0 con 0x5572c6f264e0 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 10 client.26563 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 10 client.26563 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 20 client.26563 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6f41540 0x5572c6f43990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 4 client.26563 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:42.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c6e98990 con 0x5572c6f41540 2024-09-05T12:05:42.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.257+0000 7fe40a000640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6f41540 0x5572c6f43990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.258+0000 7fe40a000640 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6f41540 0x5572c6f43990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61da2dd2ee85a2f6 server_cookie=e6cbd3b7044596c6 in_seq=0 out_seq=0 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.258+0000 7fe3f7e00640 10 client.26563 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.258+0000 7fe3f7e00640 1 -- 172.21.15.59:0/1179772102 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572c6e98990 con 0x5572c6f41540 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.258+0000 7fe3f7e00640 10 client.26563 handle_command_reply: tid=0 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.258+0000 7fe41bfc7740 1 client.26563 shutdown 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 msgr2=0x5572c6f2e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe404004770 tx=0x7fe404009260 comp rx=0 tx=0).stop 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 shutdown_connections 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f316c0 0x5572c6f2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6f41540 0x5572c6f43990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f2eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/1179772102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f264e0 0x5572c6f2e610 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 >> 172.21.15.59:0/1179772102 conn(0x5572c6f22a70 msgr2=0x5572c6f287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe3f7e00640 0 client.26563 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 shutdown_connections 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 1 -- 172.21.15.59:0/1179772102 wait complete. 2024-09-05T12:05:42.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.259+0000 7fe41bfc7740 20 client.26563 trim_cache size 0 max 16384 2024-09-05T12:05:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 Processor -- start 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 -- start start 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f21bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b3540 con 0x5572c6f26e00 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b5dc0 con 0x5572c6f264e0 2024-09-05T12:05:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.277+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b3bb0 con 0x5572c6f316c0 2024-09-05T12:05:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34510/0 (socket says 172.21.15.59:34510) 2024-09-05T12:05:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 -- 172.21.15.59:0/3682482779 learned_addr learned my addr 172.21.15.59:0/3682482779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe412a00640 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f21bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe413400640 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 -- 172.21.15.59:0/3682482779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 msgr2=0x5572c6f497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 -- 172.21.15.59:0/3682482779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 msgr2=0x5572c6f21bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f21bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 -- 172.21.15.59:0/3682482779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c6f23b20 con 0x5572c6f26e00 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.278+0000 7fe40a000640 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0cce70 tx=0x7fe3fc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350fb6a5e10277b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe402a00640 1 -- 172.21.15.59:0/3682482779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc099f80 con 0x5572c6f26e00 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe402a00640 1 -- 172.21.15.59:0/3682482779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc0ccf80 con 0x5572c6f26e00 2024-09-05T12:05:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe402a00640 1 -- 172.21.15.59:0/3682482779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0cfe30 con 0x5572c6f26e00 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 -- 172.21.15.59:0/3682482779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 msgr2=0x5572c6f49260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0cce70 tx=0x7fe3fc018420 comp rx=0 tx=0).stop 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 -- 172.21.15.59:0/3682482779 shutdown_connections 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f49260 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/3682482779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f21bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.279+0000 7fe41bfc7740 1 -- 172.21.15.59:0/3682482779 >> 172.21.15.59:0/3682482779 conn(0x5572c6f22a70 msgr2=0x5572c6f2d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.280+0000 7fe41bfc7740 1 -- 172.21.15.59:0/3682482779 shutdown_connections 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.280+0000 7fe41bfc7740 1 -- 172.21.15.59:0/3682482779 wait complete. 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.280+0000 7fe41bfc7740 1 -- start start 2024-09-05T12:05:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.280+0000 7fe41bfc7740 1 Processor -- start 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b4280 con 0x5572c6f26e00 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b56f0 con 0x5572c6f264e0 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b4950 con 0x5572c6f316c0 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe41bfc7740 10 client.0 _ll_register_callbacks cb 0x7fe404009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe40a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34518/0 (socket says 172.21.15.59:34518) 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe40a000640 1 -- 172.21.15.59:0/413918235 learned_addr learned my addr 172.21.15.59:0/413918235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe412a00640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.281+0000 7fe413400640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 -- 172.21.15.59:0/413918235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 msgr2=0x5572c6f35f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f35f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 -- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 msgr2=0x5572c6f354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 -- 172.21.15.59:0/413918235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c6f1cfe0 con 0x5572c6f26e00 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe412a00640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe40a000640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00cfd0 tx=0x7fe3fc0a00c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad592fd5bf44d337 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe3f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0982f0 con 0x5572c6f26e00 2024-09-05T12:05:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe3fc0cfbd0 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc0ccf80 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe3fc0cfdd0 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe41bfc7740 10 client.16638 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.282+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5572c6fac450 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc0ccf80 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe3fc018cd0 con 0x5572c6f26e00 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 10 client.16638 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 10 client.16638 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 20 client.16638 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6facc20 0x5572c6faf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 4 client.16638 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x5572c6e92ce0 con 0x5572c6facc20 2024-09-05T12:05:42.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.283+0000 7fe412a00640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6facc20 0x5572c6faf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:42.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.284+0000 7fe412a00640 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6facc20 0x5572c6faf040 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2f6284031ed774 server_cookie=8f2caea4c04b0774 in_seq=0 out_seq=0 2024-09-05T12:05:42.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.284+0000 7fe3f7e00640 10 client.16638 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe3f7e00640 1 -- 172.21.15.59:0/413918235 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5572c6e92ce0 con 0x5572c6facc20 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe3f7e00640 10 client.16638 handle_command_reply: tid=0 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 client.16638 shutdown 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 msgr2=0x5572c6f359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00cfd0 tx=0x7fe3fc0a00c0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 shutdown_connections 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572c6f316c0 0x5572c6f35f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5572c6facc20 0x5572c6faf040 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5572c6f26e00 0x5572c6f359f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 --2- 172.21.15.59:0/413918235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c6f264e0 0x5572c6f354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe3f7e00640 0 client.16638 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 >> 172.21.15.59:0/413918235 conn(0x5572c6f22a70 msgr2=0x5572c6f21bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 shutdown_connections 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.292+0000 7fe41bfc7740 1 -- 172.21.15.59:0/413918235 wait complete. 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe41bfc7740 20 client.16638 trim_cache size 0 max 16384 2024-09-05T12:05:42.295 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 30, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "b88a007b": { 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 30, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:05:42.296 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 146.2, 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:05:42.297 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 msgr2=0x7fe3f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 0x7fe3f80a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe4040058d0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 msgr2=0x7fe4141a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00e9f0 tx=0x7fe3fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 shutdown_connections 2024-09-05T12:05:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3f80a2b80 0x7fe3f80a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe414112a30 0x7fe41419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe414077040 0x7fe4141a4c30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 --2- 172.21.15.59:0/109293231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe414076720 0x7fe4141a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.293+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 >> 172.21.15.59:0/109293231 conn(0x7fe4140ffa90 msgr2=0x7fe414101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.294+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 shutdown_connections 2024-09-05T12:05:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:42.294+0000 7fe419600640 1 -- 172.21.15.59:0/109293231 wait complete. 2024-09-05T12:05:42.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:42 smithi089 ceph-mon[34499]: pgmap v635: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:05:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:42 smithi059 ceph-mon[30025]: pgmap v635: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:05:42.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:42 smithi189 ceph-mon[34772]: pgmap v635: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:05:43.055 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b88a007b', quiesce took 3.7 seconds. Will release after: 22.45 2024-09-05T12:05:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:44 smithi089 ceph-mon[34499]: pgmap v636: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:05:44.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:05:44.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:44 smithi059 ceph-mon[30025]: pgmap v636: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:05:44.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:05:44.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:44 smithi189 ceph-mon[34772]: pgmap v636: 129 pgs: 129 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:05:44.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:05:45.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:45 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:05:45.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:45 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:05:45.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:45 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:05:45.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:45 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:05:45.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:45 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:05:45.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:45 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:05:45.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:45 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:05:45.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:45 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:05:45.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:45 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:05:46.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:46 smithi059 ceph-mon[30025]: pgmap v637: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:46.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:46 smithi189 ceph-mon[34772]: pgmap v637: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:46 smithi089 ceph-mon[34499]: pgmap v637: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:48 smithi059 ceph-mon[30025]: pgmap v638: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:48.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:48 smithi189 ceph-mon[34772]: pgmap v638: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:48 smithi089 ceph-mon[34499]: pgmap v638: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:50 smithi059 ceph-mon[30025]: pgmap v639: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:50 smithi189 ceph-mon[34772]: pgmap v639: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:50 smithi089 ceph-mon[34499]: pgmap v639: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:05:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:52 smithi059 ceph-mon[30025]: pgmap v640: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:52 smithi189 ceph-mon[34772]: pgmap v640: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:52 smithi089 ceph-mon[34499]: pgmap v640: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:05:54.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:54 smithi059 ceph-mon[30025]: pgmap v641: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:54 smithi189 ceph-mon[34772]: pgmap v641: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:55.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:05:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:54 smithi089 ceph-mon[34499]: pgmap v641: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:55.553 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 -- 172.21.15.59:0/3434671487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340076720 msgr2=0x7f6340076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 --2- 172.21.15.59:0/3434671487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340076720 0x7f6340076b00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6334009860 tx=0x7f633402a340 comp rx=0 tx=0).stop 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 -- 172.21.15.59:0/3434671487 shutdown_connections 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 --2- 172.21.15.59:0/3434671487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 0x7f6340114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 --2- 172.21.15.59:0/3434671487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340077040 0x7f63401124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 --2- 172.21.15.59:0/3434671487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340076720 0x7f6340076b00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 -- 172.21.15.59:0/3434671487 >> 172.21.15.59:0/3434671487 conn(0x7f63400ffa90 msgr2=0x7f6340101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 -- 172.21.15.59:0/3434671487 shutdown_connections 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.143+0000 7f6347800640 1 -- 172.21.15.59:0/3434671487 wait complete. 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.144+0000 7f6347800640 1 Processor -- start 2024-09-05T12:05:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.144+0000 7f6347800640 1 -- start start 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 0x7f63401a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 0x7f634019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63401185c0 con 0x7f6340077040 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6340118440 con 0x7f6340076720 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6347800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6340118740 con 0x7f6340112a30 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f633fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6344e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 0x7f63401a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f633fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33034/0 (socket says 172.21.15.59:33034) 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f633fe00640 1 -- 172.21.15.59:0/3176222212 learned_addr learned my addr 172.21.15.59:0/3176222212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.145+0000 7f6345800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 0x7f634019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 -- 172.21.15.59:0/3176222212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 msgr2=0x7f634019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 0x7f634019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 -- 172.21.15.59:0/3176222212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 msgr2=0x7f63401a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 0x7f63401a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 -- 172.21.15.59:0/3176222212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634019efe0 con 0x7f6340077040 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f6344e00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 0x7f63401a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633fe00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9f0 tx=0x7f632c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633d600640 1 -- 172.21.15.59:0/3176222212 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00cd80 con 0x7f6340077040 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634019f2d0 con 0x7f6340077040 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633d600640 1 -- 172.21.15.59:0/3176222212 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632c010910 con 0x7f6340077040 2024-09-05T12:05:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f633d600640 1 -- 172.21.15.59:0/3176222212 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c0188f0 con 0x7f6340077040 2024-09-05T12:05:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.146+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63401ab5c0 con 0x7f6340077040 2024-09-05T12:05:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 Processor -- start 2024-09-05T12:05:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f633d600640 1 -- 172.21.15.59:0/3176222212 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f632c010ac0 con 0x7f6340077040 2024-09-05T12:05:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 -- start start 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e4126cd70 0x560e4126f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e411c83e0 con 0x560e41261b90 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e4116fd80 con 0x560e4126cd70 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f634a1f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e411c43a0 con 0x560e412624b0 2024-09-05T12:05:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f6345800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f6345800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33048/0 (socket says 172.21.15.59:33048) 2024-09-05T12:05:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.150+0000 7f6345800640 1 -- 172.21.15.59:0/1321826269 learned_addr learned my addr 172.21.15.59:0/1321826269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6344e00640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f633fe00640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e4126cd70 0x560e4126f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 -- 172.21.15.59:0/1321826269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 msgr2=0x560e4126c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 -- 172.21.15.59:0/1321826269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e4126cd70 msgr2=0x560e4126f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e4126cd70 0x560e4126f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 -- 172.21.15.59:0/1321826269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4118eae0 con 0x560e41261b90 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6344e00640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f633d600640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 0x7f63280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.151+0000 7f6345800640 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6338009fc0 tx=0x7f63380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f517bcbbfe89557 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.152+0000 7f633d600640 1 -- 172.21.15.59:0/3176222212 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f632c0cf840 con 0x7f6340077040 2024-09-05T12:05:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.152+0000 7f6332000640 1 -- 172.21.15.59:0/1321826269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338003970 con 0x560e41261b90 2024-09-05T12:05:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.152+0000 7f6332000640 1 -- 172.21.15.59:0/1321826269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633800d770 con 0x560e41261b90 2024-09-05T12:05:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.152+0000 7f6332000640 1 -- 172.21.15.59:0/1321826269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004460 con 0x560e41261b90 2024-09-05T12:05:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.152+0000 7f6344e00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 0x7f63280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 -- 172.21.15.59:0/1321826269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 msgr2=0x560e41261f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f6344e00640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 0x7f63280a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6334009830 tx=0x7f63340056d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6338009fc0 tx=0x7f63380095c0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 -- 172.21.15.59:0/1321826269 shutdown_connections 2024-09-05T12:05:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e4126cd70 0x560e4126f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1321826269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41261f70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 -- 172.21.15.59:0/1321826269 >> 172.21.15.59:0/1321826269 conn(0x560e4125e120 msgr2=0x560e4125e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 -- 172.21.15.59:0/1321826269 shutdown_connections 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.153+0000 7f634a1f2740 1 -- 172.21.15.59:0/1321826269 wait complete. 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 -- start start 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 Processor -- start 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e4126a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e4126f9e0 con 0x560e4126cd70 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e411830f0 con 0x560e41261b90 2024-09-05T12:05:56.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e4126fb60 con 0x560e412624b0 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.154+0000 7f634a1f2740 10 client.0 _ll_register_callbacks cb 0x560e411749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6344e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e4126a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6344e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36220/0 (socket says 172.21.15.59:36220) 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6344e00640 1 -- 172.21.15.59:0/2925716732 learned_addr learned my addr 172.21.15.59:0/2925716732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f633fe00640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 -- 172.21.15.59:0/2925716732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 msgr2=0x560e4126a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 -- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 msgr2=0x560e41269ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6345800640 1 -- 172.21.15.59:0/2925716732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e40f393c0 con 0x560e4126cd70 2024-09-05T12:05:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.155+0000 7f6344e00640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6345800640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e4126a760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6338009f90 tx=0x7f633800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec912c1cab9134f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6326a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338003b80 con 0x560e4126cd70 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f633801c040 con 0x560e4126cd70 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633800d770 con 0x560e4126cd70 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6338033050 con 0x560e4126cd70 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f634a1f2740 10 client.16650 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560e4127b710 con 0x560e4126cd70 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.156+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633800d770 con 0x560e4126cd70 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f633800b040 con 0x560e4126cd70 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 10 client.16650 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 20 client.16650 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e4127cb70 0x560e4127ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 4 client.16650 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e411d5950 con 0x560e4127cb70 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.157+0000 7f6344e00640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e4127cb70 0x560e4127ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.158+0000 7f6344e00640 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e4127cb70 0x560e4127ef60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229ee99c537b01e7 server_cookie=bffa22727df15eb5 in_seq=0 out_seq=0 2024-09-05T12:05:56.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.158+0000 7f6326a00640 10 client.16650 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.158+0000 7f6326a00640 1 -- 172.21.15.59:0/2925716732 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e411d5950 con 0x560e4127cb70 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.158+0000 7f6326a00640 10 client.16650 handle_command_reply: tid=0 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 client.16650 shutdown 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 msgr2=0x560e4126a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e4126a760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6338009f90 tx=0x7f633800dbd0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 shutdown_connections 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e4127cb70 0x560e4127ef60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e4126a760 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f6326a00640 0 client.16650 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e4126a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 --2- 172.21.15.59:0/2925716732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41269ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 >> 172.21.15.59:0/2925716732 conn(0x560e4125e120 msgr2=0x560e41263e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 shutdown_connections 2024-09-05T12:05:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 1 -- 172.21.15.59:0/2925716732 wait complete. 2024-09-05T12:05:56.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.159+0000 7f634a1f2740 20 client.16650 trim_cache size 0 max 16384 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 Processor -- start 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 -- start start 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 0x560e41284560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 0x560e41284aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b34d0 con 0x560e41261b90 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b5d50 con 0x560e412624b0 2024-09-05T12:05:56.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.177+0000 7f634a1f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b3b40 con 0x560e4126cd70 2024-09-05T12:05:56.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33062/0 (socket says 172.21.15.59:33062) 2024-09-05T12:05:56.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 -- 172.21.15.59:0/3212768618 learned_addr learned my addr 172.21.15.59:0/3212768618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6345800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 0x560e41284aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f633fe00640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 0x560e41284560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 -- 172.21.15.59:0/3212768618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 msgr2=0x560e41284aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 0x560e41284aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 -- 172.21.15.59:0/3212768618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 msgr2=0x560e41284560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 0x560e41284560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6344e00640 1 -- 172.21.15.59:0/3212768618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e41268230 con 0x560e41261b90 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f6345800640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 0x560e41284aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.178+0000 7f633fe00640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 0x560e41284560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f6344e00640 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6334009780 tx=0x7f633403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed406b7a8bcea1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f6331600640 1 -- 172.21.15.59:0/3212768618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334056070 con 0x560e41261b90 2024-09-05T12:05:56.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f6331600640 1 -- 172.21.15.59:0/3212768618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6334043040 con 0x560e41261b90 2024-09-05T12:05:56.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f6331600640 1 -- 172.21.15.59:0/3212768618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334037040 con 0x560e41261b90 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f634a1f2740 1 -- 172.21.15.59:0/3212768618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 msgr2=0x560e41258e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.179+0000 7f634a1f2740 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6334009780 tx=0x7f633403d000 comp rx=0 tx=0).stop 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 -- 172.21.15.59:0/3212768618 shutdown_connections 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e4126cd70 0x560e41284aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e412624b0 0x560e41284560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 --2- 172.21.15.59:0/3212768618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e41261b90 0x560e41258e50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 -- 172.21.15.59:0/3212768618 >> 172.21.15.59:0/3212768618 conn(0x560e4125e120 msgr2=0x560e41268dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 -- 172.21.15.59:0/3212768618 shutdown_connections 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 -- 172.21.15.59:0/3212768618 wait complete. 2024-09-05T12:05:56.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 -- start start 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.180+0000 7f634a1f2740 1 Processor -- start 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41275570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e412708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b4210 con 0x560e4126cd70 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b5680 con 0x560e41261b90 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b48e0 con 0x560e412624b0 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f634a1f2740 10 client.0 _ll_register_callbacks cb 0x7f6338009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f6345800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f633fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e412708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f6345800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33078/0 (socket says 172.21.15.59:33078) 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f6345800640 1 -- 172.21.15.59:0/1020410492 learned_addr learned my addr 172.21.15.59:0/1020410492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.181+0000 7f6344e00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41275570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 -- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 msgr2=0x560e41275570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41275570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 -- 172.21.15.59:0/1020410492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 msgr2=0x560e41270e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 -- 172.21.15.59:0/1020410492 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e41271630 con 0x560e412624b0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f6345800640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f6344e00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41275570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f633fe00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e412708a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560e4125f4a0 tx=0x7f632c0980a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaeea26f061c7d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f6326a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c09b330 con 0x560e412624b0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f632c010430 con 0x560e412624b0 2024-09-05T12:05:56.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.182+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6334043040 con 0x560e412624b0 2024-09-05T12:05:56.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f632c09ffe0 con 0x560e412624b0 2024-09-05T12:05:56.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 10 client.26587 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:56.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560e412e75d0 con 0x560e412624b0 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6334043040 con 0x560e412624b0 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f632c0a1170 con 0x560e412624b0 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 10 client.26587 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 10 client.26587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 20 client.26587 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e412e7d00 0x560e412ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:56.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 4 client.26587 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.183+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e411cfe60 con 0x560e412e7d00 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f6344e00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e412e7d00 0x560e412ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f6344e00640 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e412e7d00 0x560e412ea120 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4591f472cc488e8e server_cookie=49e28f844af0613 in_seq=0 out_seq=0 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f6326a00640 10 client.26587 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f6326a00640 1 -- 172.21.15.59:0/1020410492 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e411cfe60 con 0x560e412e7d00 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f6326a00640 10 client.26587 handle_command_reply: tid=0 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.184+0000 7f634a1f2740 1 client.26587 shutdown 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 msgr2=0x560e412708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e412708a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560e4125f4a0 tx=0x7f632c0980a0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 shutdown_connections 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x560e412e7d00 0x560e412ea120 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e4126cd70 0x560e41270e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e412624b0 0x560e412708a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f6326a00640 0 client.26587 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 --2- 172.21.15.59:0/1020410492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e41261b90 0x560e41275570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 >> 172.21.15.59:0/1020410492 conn(0x560e4125e120 msgr2=0x560e41269c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 shutdown_connections 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 1 -- 172.21.15.59:0/1020410492 wait complete. 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.185+0000 7f634a1f2740 20 client.26587 trim_cache size 0 max 16384 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:05:56.190 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:05:56.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 msgr2=0x7f63280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 0x7f63280a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6334009830 tx=0x7f63340056d0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 msgr2=0x7f63401a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:56.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9f0 tx=0x7f632c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 shutdown_connections 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f63280a2b80 0x7f63280a5040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6340112a30 0x7f634019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6340077040 0x7f63401a4c30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 --2- 172.21.15.59:0/3176222212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6340076720 0x7f63401a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 >> 172.21.15.59:0/3176222212 conn(0x7f63400ffa90 msgr2=0x7f6340101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 shutdown_connections 2024-09-05T12:05:56.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:56.186+0000 7f6347800640 1 -- 172.21.15.59:0/3176222212 wait complete. 2024-09-05T12:05:56.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:05:56.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:05:56.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:56 smithi059 ceph-mon[30025]: pgmap v642: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:57.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:56 smithi189 ceph-mon[34772]: pgmap v642: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:56 smithi089 ceph-mon[34499]: pgmap v642: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-05T12:05:57.310 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 -- 172.21.15.59:0/3485756525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 msgr2=0x7f7440076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 --2- 172.21.15.59:0/3485756525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f7440076b00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7430009900 tx=0x7f743002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 -- 172.21.15.59:0/3485756525 shutdown_connections 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 --2- 172.21.15.59:0/3485756525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f7440114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 --2- 172.21.15.59:0/3485756525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 0x7f74401124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 --2- 172.21.15.59:0/3485756525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f7440076b00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 -- 172.21.15.59:0/3485756525 >> 172.21.15.59:0/3485756525 conn(0x7f74400ffa90 msgr2=0x7f7440101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 -- 172.21.15.59:0/3485756525 shutdown_connections 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.892+0000 7f7444c00640 1 -- 172.21.15.59:0/3485756525 wait complete. 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.893+0000 7f7444c00640 1 Processor -- start 2024-09-05T12:05:57.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 -- start start 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 0x7f74401a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f744019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74401185a0 con 0x7f7440076720 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7440118420 con 0x7f7440112a30 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f7444c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7440118720 con 0x7f7440077040 2024-09-05T12:05:57.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f743de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f743de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33088/0 (socket says 172.21.15.59:33088) 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f743de00640 1 -- 172.21.15.59:0/2977104504 learned_addr learned my addr 172.21.15.59:0/2977104504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f743d400640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 0x7f74401a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.894+0000 7f743e800640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f744019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 -- 172.21.15.59:0/2977104504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 msgr2=0x7f74401a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 0x7f74401a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 -- 172.21.15.59:0/2977104504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 msgr2=0x7f744019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f744019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 -- 172.21.15.59:0/2977104504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744019efe0 con 0x7f7440076720 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743e800640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f744019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f743de00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7430006010 tx=0x7f74300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743003f030 con 0x7f7440076720 2024-09-05T12:05:57.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7430003ce0 con 0x7f7440076720 2024-09-05T12:05:57.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744019f270 con 0x7f7440076720 2024-09-05T12:05:57.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.895+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430035070 con 0x7f7440076720 2024-09-05T12:05:57.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.896+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74401ab5c0 con 0x7f7440076720 2024-09-05T12:05:57.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.897+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440076b00 con 0x7f7440076720 2024-09-05T12:05:57.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.899+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7430042030 con 0x7f7440076720 2024-09-05T12:05:57.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.900+0000 7f7436a00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:57.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.900+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f74300efd60 con 0x7f7440076720 2024-09-05T12:05:57.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.900+0000 7f743d400640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:57.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.901+0000 7f743d400640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7440079370 tx=0x7f742c00b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:57.902+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74300bc2c0 con 0x7f7440076720 2024-09-05T12:05:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.127+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f744010dbc0 con 0x7f7440076720 2024-09-05T12:05:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.128+0000 7f7436a00640 1 -- 172.21.15.59:0/2977104504 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f74300c05c0 con 0x7f7440076720 2024-09-05T12:05:58.130 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:05:58.130 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:05:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:05:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 msgr2=0x7f74240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7440079370 tx=0x7f742c00b410 comp rx=0 tx=0).stop 2024-09-05T12:05:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 msgr2=0x7f74401a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7430006010 tx=0x7f74300042f0 comp rx=0 tx=0).stop 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 shutdown_connections 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7440112a30 0x7f744019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7440077040 0x7f74401a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 --2- 172.21.15.59:0/2977104504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7440076720 0x7f74401a46f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 >> 172.21.15.59:0/2977104504 conn(0x7f74400ffa90 msgr2=0x7f7440101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 shutdown_connections 2024-09-05T12:05:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:58.131+0000 7f7444c00640 1 -- 172.21.15.59:0/2977104504 wait complete. 2024-09-05T12:05:58.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:58 smithi059 ceph-mon[30025]: pgmap v643: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:05:58.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:05:58 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2977104504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:58.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:05:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:58 smithi189 ceph-mon[34772]: pgmap v643: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:05:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:05:58 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2977104504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:58 smithi089 ceph-mon[34499]: pgmap v643: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-05T12:05:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:05:58 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2977104504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:05:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 -- 172.21.15.59:0/2165888881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac109120 msgr2=0x7f91ac110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 --2- 172.21.15.59:0/2165888881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac109120 0x7f91ac110ca0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009ca0 tx=0x7f91a402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 -- 172.21.15.59:0/2165888881 shutdown_connections 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 --2- 172.21.15.59:0/2165888881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac109120 0x7f91ac110ca0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 --2- 172.21.15.59:0/2165888881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac1049a0 0x7f91ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 --2- 172.21.15.59:0/2165888881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac103ff0 0x7f91ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 -- 172.21.15.59:0/2165888881 >> 172.21.15.59:0/2165888881 conn(0x7f91ac0ffa90 msgr2=0x7f91ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 -- 172.21.15.59:0/2165888881 shutdown_connections 2024-09-05T12:05:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.918+0000 7f91b4000640 1 -- 172.21.15.59:0/2165888881 wait complete. 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.919+0000 7f91b4000640 1 Processor -- start 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.919+0000 7f91b4000640 1 -- start start 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac1049a0 0x7f91ac19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac109120 0x7f91ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1141a0 con 0x7f91ac103ff0 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac114020 con 0x7f91ac109120 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b4000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac114320 con 0x7f91ac1049a0 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33106/0 (socket says 172.21.15.59:33106) 2024-09-05T12:05:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b1600640 1 -- 172.21.15.59:0/2647875362 learned_addr learned my addr 172.21.15.59:0/2647875362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b2000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac109120 0x7f91ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.920+0000 7f91b0c00640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac1049a0 0x7f91ac19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 -- 172.21.15.59:0/2647875362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac1049a0 msgr2=0x7f91ac19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac1049a0 0x7f91ac19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 -- 172.21.15.59:0/2647875362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac109120 msgr2=0x7f91ac19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac109120 0x7f91ac19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 -- 172.21.15.59:0/2647875362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac19f570 con 0x7f91ac103ff0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b1600640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f91a000e9f0 tx=0x7f91a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91aa000640 1 -- 172.21.15.59:0/2647875362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a000cd80 con 0x7f91ac103ff0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91aa000640 1 -- 172.21.15.59:0/2647875362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a0005500 con 0x7f91ac103ff0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91aa000640 1 -- 172.21.15.59:0/2647875362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0010690 con 0x7f91ac103ff0 2024-09-05T12:05:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac19f860 con 0x7f91ac103ff0 2024-09-05T12:05:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.921+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac1a7140 con 0x7f91ac103ff0 2024-09-05T12:05:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.924+0000 7f91aa000640 1 -- 172.21.15.59:0/2647875362 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f91a00046c0 con 0x7f91ac103ff0 2024-09-05T12:05:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 Processor -- start 2024-09-05T12:05:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 -- start start 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd008050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd012910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556dbcf6e3e0 con 0x556dbd012e50 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556dbcf15d80 con 0x556dbd007c70 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b6a97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556dbcf6a3a0 con 0x556dbd008590 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33120/0 (socket says 172.21.15.59:33120) 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.925+0000 7f91b0c00640 1 -- 172.21.15.59:0/1238049732 learned_addr learned my addr 172.21.15.59:0/1238049732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd008050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b1600640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd012910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 -- 172.21.15.59:0/1238049732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 msgr2=0x556dbd012910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd012910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 -- 172.21.15.59:0/1238049732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 msgr2=0x556dbd015240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 -- 172.21.15.59:0/1238049732 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbcf34ae0 con 0x556dbd007c70 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b0c00640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91b2000640 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd008050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91a4005e30 tx=0x7f91a40093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e544804e5378bdda server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.926+0000 7f91aa000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 0x7f91940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.927+0000 7f919e600640 1 -- 172.21.15.59:0/1238049732 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4031db0 con 0x556dbd007c70 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.927+0000 7f919e600640 1 -- 172.21.15.59:0/1238049732 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4036030 con 0x556dbd007c70 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.927+0000 7f919e600640 1 -- 172.21.15.59:0/1238049732 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a40042a0 con 0x556dbd007c70 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.927+0000 7f91aa000640 1 -- 172.21.15.59:0/2647875362 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f91a00cf830 con 0x7f91ac103ff0 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.927+0000 7f91b1600640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 0x7f91940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- 172.21.15.59:0/1238049732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 msgr2=0x556dbd008050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd008050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91a4005e30 tx=0x7f91a40093e0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- 172.21.15.59:0/1238049732 shutdown_connections 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd012e50 0x556dbd015240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd012910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1238049732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd008050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- 172.21.15.59:0/1238049732 >> 172.21.15.59:0/1238049732 conn(0x556dbd004200 msgr2=0x556dbd0045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- 172.21.15.59:0/1238049732 shutdown_connections 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b1600640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 0x7f91940a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ccec0 tx=0x7f91a00d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- 172.21.15.59:0/1238049732 wait complete. 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.928+0000 7f91b6a97740 1 -- start start 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 Processor -- start 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd010830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556dbd015840 con 0x556dbd007c70 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556dbcf290f0 con 0x556dbd012e50 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556dbd0159c0 con 0x556dbd008590 2024-09-05T12:05:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b6a97740 10 client.0 _ll_register_callbacks cb 0x556dbcf10bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33122/0 (socket says 172.21.15.59:33122) 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b2000640 1 -- 172.21.15.59:0/2282474545 learned_addr learned my addr 172.21.15.59:0/2282474545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.929+0000 7f91b0c00640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd0102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b2000640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd010830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 -- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 msgr2=0x556dbd010830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd010830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 -- 172.21.15.59:0/2282474545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 msgr2=0x556dbd00fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 -- 172.21.15.59:0/2282474545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbccdf3c0 con 0x556dbd008590 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b2000640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd010830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b0c00640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd0102f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91980058c0 tx=0x7f9198012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98eed5bf6c6d3045 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f9192a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198004050 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a4036030 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.930+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f91980041f0 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f91980043f0 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 10 client.26605 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556dbd0219b0 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4036030 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9198003ca0 con 0x556dbd008590 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 10 client.26605 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 20 client.26605 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd022e70 0x556dbd0252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 4 client.26605 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.931+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dbcf7b950 con 0x556dbd022e70 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.932+0000 7f91b2000640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd022e70 0x556dbd0252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.932+0000 7f91b2000640 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd022e70 0x556dbd0252a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d006fec1b9ee85 server_cookie=c3a0c299500fe461 in_seq=0 out_seq=0 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.932+0000 7f9192a00640 10 client.26605 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f9192a00640 1 -- 172.21.15.59:0/2282474545 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556dbcf7b950 con 0x556dbd022e70 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f9192a00640 10 client.26605 handle_command_reply: tid=0 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 client.26605 shutdown 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 msgr2=0x556dbd0102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd0102f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91980058c0 tx=0x7f9198012b60 comp rx=0 tx=0).stop 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 shutdown_connections 2024-09-05T12:05:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd010830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd0102f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd022e70 0x556dbd0252a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 --2- 172.21.15.59:0/2282474545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f9192a00640 0 client.26605 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.933+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 >> 172.21.15.59:0/2282474545 conn(0x556dbd004200 msgr2=0x556dbd003380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.934+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 shutdown_connections 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.934+0000 7f91b6a97740 1 -- 172.21.15.59:0/2282474545 wait complete. 2024-09-05T12:05:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.934+0000 7f91b6a97740 20 client.26605 trim_cache size 0 max 16384 2024-09-05T12:05:59.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 Processor -- start 2024-09-05T12:05:59.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 -- start start 2024-09-05T12:05:59.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd00f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd02abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1aef60 con 0x556dbd007c70 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b17e0 con 0x556dbd012e50 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b6a97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1af5d0 con 0x556dbd008590 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33132/0 (socket says 172.21.15.59:33132) 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b2000640 1 -- 172.21.15.59:0/1904821679 learned_addr learned my addr 172.21.15.59:0/1904821679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.948+0000 7f91b0c00640 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd00f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd02abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 -- 172.21.15.59:0/1904821679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 msgr2=0x556dbd00f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd00f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 -- 172.21.15.59:0/1904821679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 msgr2=0x556dbd02abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd02abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 -- 172.21.15.59:0/1904821679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbd00ed60 con 0x556dbd007c70 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f91b2000640 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f91a4004af0 tx=0x7f91a4031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b967d7d83b7af974 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f919dc00640 1 -- 172.21.15.59:0/1904821679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4031bb0 con 0x556dbd007c70 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f919dc00640 1 -- 172.21.15.59:0/1904821679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4036040 con 0x556dbd007c70 2024-09-05T12:05:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.949+0000 7f919dc00640 1 -- 172.21.15.59:0/1904821679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004040 con 0x556dbd007c70 2024-09-05T12:05:59.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 -- 172.21.15.59:0/1904821679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 msgr2=0x556dbd00a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f91a4004af0 tx=0x7f91a4031970 comp rx=0 tx=0).stop 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 -- 172.21.15.59:0/1904821679 shutdown_connections 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd012e50 0x556dbd02abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd008590 0x556dbd00f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 --2- 172.21.15.59:0/1904821679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd007c70 0x556dbd00a6d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 -- 172.21.15.59:0/1904821679 >> 172.21.15.59:0/1904821679 conn(0x556dbd004200 msgr2=0x556dbd011e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 -- 172.21.15.59:0/1904821679 shutdown_connections 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.950+0000 7f91b6a97740 1 -- 172.21.15.59:0/1904821679 wait complete. 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 -- start start 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 Processor -- start 2024-09-05T12:05:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd016a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd012e50 0x556dbd0174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1afca0 con 0x556dbd008590 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b1110 con 0x556dbd007c70 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b0370 con 0x556dbd012e50 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.951+0000 7f91b6a97740 10 client.0 _ll_register_callbacks cb 0x7f9198009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33136/0 (socket says 172.21.15.59:33136) 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 -- 172.21.15.59:0/3617288596 learned_addr learned my addr 172.21.15.59:0/3617288596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b2000640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd016a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b2000640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd012e50 0x556dbd0174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 -- 172.21.15.59:0/3617288596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd012e50 msgr2=0x556dbd0174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd012e50 0x556dbd0174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 -- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 msgr2=0x556dbd016a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:05:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd016a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 -- 172.21.15.59:0/3617288596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbd0083c0 con 0x556dbd008590 2024-09-05T12:05:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f91b0c00640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9198012420 tx=0x7f9198010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bcc488f923a0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:05:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.952+0000 7f9192a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919802e070 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a4036040 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9198010dd0 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f919803a050 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f91b6a97740 10 client.16680 fetch_fsmap learned FSMap version 21 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556dbd08dd90 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.953+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4036040 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9198004590 con 0x556dbd008590 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 10 client.16680 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 20 client.16680 populate_metadata read hostname 'smithi059' 2024-09-05T12:05:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd08e540 0x556dbd090960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 4 client.16680 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dbcf75e60 con 0x556dbd08e540 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b2000640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd08e540 0x556dbd090960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f91b2000640 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd08e540 0x556dbd090960 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e7ed6fb18ff38 server_cookie=8ca69e71eb41c71 in_seq=0 out_seq=0 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.954+0000 7f9192a00640 10 client.16680 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f9192a00640 1 -- 172.21.15.59:0/3617288596 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dbcf75e60 con 0x556dbd08e540 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f9192a00640 10 client.16680 handle_command_reply: tid=0 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 client.16680 shutdown 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 msgr2=0x556dbd016f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9198012420 tx=0x7f9198010940 comp rx=0 tx=0).stop 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 shutdown_connections 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556dbd012e50 0x556dbd0174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556dbd08e540 0x556dbd090960 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556dbd008590 0x556dbd016f70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 --2- 172.21.15.59:0/3617288596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556dbd007c70 0x556dbd016a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 >> 172.21.15.59:0/3617288596 conn(0x556dbd004200 msgr2=0x556dbd00fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f9192a00640 0 client.16680 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 shutdown_connections 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 1 -- 172.21.15.59:0/3617288596 wait complete. 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.955+0000 7f91b6a97740 20 client.16680 trim_cache size 0 max 16384 2024-09-05T12:05:59.960 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 msgr2=0x7f91940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 0x7f91940a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ccec0 tx=0x7f91a00d5530 comp rx=0 tx=0).stop 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 msgr2=0x7f91ac19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f91a000e9f0 tx=0x7f91a000edf0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 shutdown_connections 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f91940a2c00 0x7f91940a50c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91ac109120 0x7f91ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91ac1049a0 0x7f91ac19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 --2- 172.21.15.59:0/2647875362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ac103ff0 0x7f91ac19a5c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:05:59.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 >> 172.21.15.59:0/2647875362 conn(0x7f91ac0ffa90 msgr2=0x7f91ac1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:05:59.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 shutdown_connections 2024-09-05T12:05:59.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:05:59.956+0000 7f91b4000640 1 -- 172.21.15.59:0/2647875362 wait complete. 2024-09-05T12:06:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:00 smithi189 ceph-mon[34772]: pgmap v644: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:06:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:00 smithi089 ceph-mon[34499]: pgmap v644: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:06:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:00 smithi059 ceph-mon[30025]: pgmap v644: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-05T12:06:01.754 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:06:01.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:06:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:02.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 -- 172.21.15.59:0/585987288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4103e30 msgr2=0x7f58f4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 --2- 172.21.15.59:0/585987288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4103e30 0x7f58f4104210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 -- 172.21.15.59:0/585987288 shutdown_connections 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 --2- 172.21.15.59:0/585987288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4108f60 0x7f58f4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 --2- 172.21.15.59:0/585987288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 --2- 172.21.15.59:0/585987288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4103e30 0x7f58f4104210 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 -- 172.21.15.59:0/585987288 >> 172.21.15.59:0/585987288 conn(0x7f58f40ff8d0 msgr2=0x7f58f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 -- 172.21.15.59:0/585987288 shutdown_connections 2024-09-05T12:06:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.679+0000 7f58f3200640 1 -- 172.21.15.59:0/585987288 wait complete. 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.680+0000 7f58f3200640 1 Processor -- start 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.680+0000 7f58f3200640 1 -- start start 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 0x7f58f419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4113fc0 con 0x7f58f4108f60 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4113e40 con 0x7f58f41047e0 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4114140 con 0x7f58f4103e30 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 0x7f58f419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 0x7f58f419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50144/0 (socket says 172.21.15.59:50144) 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f2800640 1 -- 172.21.15.59:0/4236897666 learned_addr learned my addr 172.21.15.59:0/4236897666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35262/0 (socket says 172.21.15.59:35262) 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.681+0000 7f58f1400640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 -- 172.21.15.59:0/4236897666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 msgr2=0x7f58f419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 -- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 msgr2=0x7f58f419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 0x7f58f419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 -- 172.21.15.59:0/4236897666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f419f430 con 0x7f58f4103e30 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1400640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f1e00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005940 tx=0x7f58e40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58eaa00640 1 -- 172.21.15.59:0/4236897666 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e403f030 con 0x7f58f4103e30 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f419f6c0 con 0x7f58f4103e30 2024-09-05T12:06:02.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58eaa00640 1 -- 172.21.15.59:0/4236897666 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e40044c0 con 0x7f58f4103e30 2024-09-05T12:06:02.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58eaa00640 1 -- 172.21.15.59:0/4236897666 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035070 con 0x7f58f4103e30 2024-09-05T12:06:02.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.682+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f41a7000 con 0x7f58f4103e30 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.685+0000 7f58fba90740 1 Processor -- start 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.685+0000 7f58fba90740 1 -- start start 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58eaa00640 1 -- 172.21.15.59:0/4236897666 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58e4042030 con 0x7f58f4103e30 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c3810950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3810e90 0x5641c3813280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641c376c290 con 0x5641c3810e90 2024-09-05T12:06:02.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5641c3713b40 con 0x5641c38065d0 2024-09-05T12:06:02.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58fba90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641c3768250 con 0x5641c3805cb0 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58f2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c3810950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58f2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35270/0 (socket says 172.21.15.59:35270) 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58f2800640 1 -- 172.21.15.59:0/2578562501 learned_addr learned my addr 172.21.15.59:0/2578562501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.686+0000 7f58f1400640 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3810e90 0x5641c3813280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 -- 172.21.15.59:0/2578562501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 msgr2=0x5641c3810950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c3810950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 -- 172.21.15.59:0/2578562501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3810e90 msgr2=0x5641c3813280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3810e90 0x5641c3813280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 -- 172.21.15.59:0/2578562501 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c34dd6a0 con 0x5641c3805cb0 2024-09-05T12:06:02.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58f2800640 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011470 tx=0x7f58ec011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3130c9992f4446fb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58df400640 1 -- 172.21.15.59:0/2578562501 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec004b20 con 0x5641c3805cb0 2024-09-05T12:06:02.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58df400640 1 -- 172.21.15.59:0/2578562501 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec004cc0 con 0x5641c3805cb0 2024-09-05T12:06:02.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.687+0000 7f58df400640 1 -- 172.21.15.59:0/2578562501 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec005420 con 0x5641c3805cb0 2024-09-05T12:06:02.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58eaa00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58eaa00640 1 -- 172.21.15.59:0/4236897666 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58e40ef8a0 con 0x7f58f4103e30 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58f1e00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 -- 172.21.15.59:0/2578562501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 msgr2=0x5641c3806090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011470 tx=0x7f58ec011930 comp rx=0 tx=0).stop 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 -- 172.21.15.59:0/2578562501 shutdown_connections 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3810e90 0x5641c3813280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c3810950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 --2- 172.21.15.59:0/2578562501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3805cb0 0x5641c3806090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 -- 172.21.15.59:0/2578562501 >> 172.21.15.59:0/2578562501 conn(0x5641c3802240 msgr2=0x5641c3802620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.688+0000 7f58fba90740 1 -- 172.21.15.59:0/2578562501 shutdown_connections 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.689+0000 7f58fba90740 1 -- 172.21.15.59:0/2578562501 wait complete. 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.689+0000 7f58f1e00640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 0x7f58d80a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58e402a910 tx=0x7f58e40f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.689+0000 7f58fba90740 1 -- start start 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.689+0000 7f58fba90740 1 Processor -- start 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c38106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c380deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641c3813b00 con 0x5641c38065d0 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5641c3726fa0 con 0x5641c3805cb0 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641c3813c80 con 0x5641c3810e90 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58fba90740 10 client.0 _ll_register_callbacks cb 0x5641c347aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50154/0 (socket says 172.21.15.59:50154) 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58f1400640 1 -- 172.21.15.59:0/2316916752 learned_addr learned my addr 172.21.15.59:0/2316916752 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58f2800640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c38106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.690+0000 7f58f2800640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c380deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 -- 172.21.15.59:0/2316916752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 msgr2=0x5641c380deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c380deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 -- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 msgr2=0x5641c38106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c38106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 -- 172.21.15.59:0/2316916752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c380e6e0 con 0x5641c38065d0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58f1400640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f58e00126b0 tx=0x7f58e0012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d2d16cf01e1834 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0004320 con 0x5641c38065d0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58e00119a0 con 0x5641c38065d0 2024-09-05T12:06:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.691+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ec004cc0 con 0x5641c38065d0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f58e00044c0 con 0x5641c38065d0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 10 client.16692 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5641c381f9b0 con 0x5641c38065d0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec004cc0 con 0x5641c38065d0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f58e000f440 con 0x5641c38065d0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 10 client.16692 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 20 client.16692 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c3820e70 0x5641c38232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 4 client.16692 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.692+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641c3779800 con 0x5641c3820e70 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.693+0000 7f58f2800640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c3820e70 0x5641c38232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.693+0000 7f58f2800640 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c3820e70 0x5641c38232a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb07b9cc0451e601 server_cookie=bddc3be59452d26d in_seq=0 out_seq=0 2024-09-05T12:06:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.693+0000 7f58d3e00640 10 client.16692 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58d3e00640 1 -- 172.21.15.59:0/2316916752 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641c3779800 con 0x5641c3820e70 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58d3e00640 10 client.16692 handle_command_reply: tid=0 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 client.16692 shutdown 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 msgr2=0x5641c380d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f58e00126b0 tx=0x7f58e0012b70 comp rx=0 tx=0).stop 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 shutdown_connections 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c380deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c3820e70 0x5641c38232a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c380d970 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 --2- 172.21.15.59:0/2316916752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c38106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 >> 172.21.15.59:0/2316916752 conn(0x5641c3802240 msgr2=0x5641c3807f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.694+0000 7f58d3e00640 0 client.16692 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.695+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 shutdown_connections 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.695+0000 7f58fba90740 1 -- 172.21.15.59:0/2316916752 wait complete. 2024-09-05T12:06:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.695+0000 7f58fba90740 20 client.16692 trim_cache size 0 max 16384 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.713+0000 7f58fba90740 1 Processor -- start 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.713+0000 7f58fba90740 1 -- start start 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3818ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41aeeb0 con 0x5641c3805cb0 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41b1730 con 0x5641c38065d0 2024-09-05T12:06:02.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58fba90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41af520 con 0x5641c3810e90 2024-09-05T12:06:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44784/0 (socket says 172.21.15.59:44784) 2024-09-05T12:06:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50162/0 (socket says 172.21.15.59:50162) 2024-09-05T12:06:02.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f1400640 1 -- 172.21.15.59:0/3042074627 learned_addr learned my addr 172.21.15.59:0/3042074627 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.714+0000 7f58f2800640 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3818ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 -- 172.21.15.59:0/3042074627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 msgr2=0x5641c3818ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3818ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 -- 172.21.15.59:0/3042074627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 msgr2=0x5641c3828bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 -- 172.21.15.59:0/3042074627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c381cd90 con 0x5641c38065d0 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f2800640 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:02.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58f1400640 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58e000f860 tx=0x7f58e00092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f83b4c041918598 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58dea00640 1 -- 172.21.15.59:0/3042074627 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0009680 con 0x5641c38065d0 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58dea00640 1 -- 172.21.15.59:0/3042074627 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0011e80 con 0x5641c38065d0 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.715+0000 7f58dea00640 1 -- 172.21.15.59:0/3042074627 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00052d0 con 0x5641c38065d0 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 -- 172.21.15.59:0/3042074627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 msgr2=0x5641c38290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58e000f860 tx=0x7f58e00092a0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 -- 172.21.15.59:0/3042074627 shutdown_connections 2024-09-05T12:06:02.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3818ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c38065d0 0x5641c38290f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 --2- 172.21.15.59:0/3042074627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c3805cb0 0x5641c3828bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 -- 172.21.15.59:0/3042074627 >> 172.21.15.59:0/3042074627 conn(0x5641c3802240 msgr2=0x5641c380ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 -- 172.21.15.59:0/3042074627 shutdown_connections 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.716+0000 7f58fba90740 1 -- 172.21.15.59:0/3042074627 wait complete. 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 -- start start 2024-09-05T12:06:02.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 Processor -- start 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c3816da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c38172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41afbf0 con 0x5641c38065d0 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41b1060 con 0x5641c3805cb0 2024-09-05T12:06:02.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58f41b02c0 con 0x5641c3810e90 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.717+0000 7f58fba90740 10 client.0 _ll_register_callbacks cb 0x7f58e0009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c38172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35306/0 (socket says 172.21.15.59:35306) 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 -- 172.21.15.59:0/2977342783 learned_addr learned my addr 172.21.15.59:0/2977342783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c3816da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 -- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 msgr2=0x5641c3816da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c3816da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 -- 172.21.15.59:0/2977342783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 msgr2=0x5641c38172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c38172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 -- 172.21.15.59:0/2977342783 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c388acf0 con 0x5641c3810e90 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58f2800640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011e30 tx=0x7f58ec00ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81774793765a83d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.718+0000 7f58d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:06:02.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00f950 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641c3773d10 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58ec028450 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f58ec00faf0 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58fba90740 10 client.26629 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5641c388bfd0 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5641c3773d10 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.719+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f58ec0052c0 con 0x5641c3810e90 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 10 client.26629 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 20 client.26629 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c388c3b0 0x5641c388e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 4 client.26629 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641c36effb0 con 0x5641c388c3b0 2024-09-05T12:06:02.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.720+0000 7f58f1400640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c388c3b0 0x5641c388e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:02.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.721+0000 7f58f1400640 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c388c3b0 0x5641c388e7d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4334710543bbc0 server_cookie=cbab9f83b23fc7ea in_seq=0 out_seq=0 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.721+0000 7f58d3e00640 10 client.26629 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.721+0000 7f58d3e00640 1 -- 172.21.15.59:0/2977342783 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641c36effb0 con 0x5641c388c3b0 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.721+0000 7f58d3e00640 10 client.26629 handle_command_reply: tid=0 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 client.26629 shutdown 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 msgr2=0x5641c3817820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011e30 tx=0x7f58ec00ffd0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 shutdown_connections 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641c3810e90 0x5641c3817820 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641c388c3b0 0x5641c388e7d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641c38065d0 0x5641c38172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 --2- 172.21.15.59:0/2977342783 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641c3805cb0 0x5641c3816da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 >> 172.21.15.59:0/2977342783 conn(0x5641c3802240 msgr2=0x5641c380f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58d3e00640 0 client.26629 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 shutdown_connections 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 1 -- 172.21.15.59:0/2977342783 wait complete. 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.722+0000 7f58fba90740 20 client.26629 trim_cache size 0 max 16384 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "585b23d2-4545-4c94-95bd-a8e1ef1ba5c5", 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:06:02.726 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 msgr2=0x7f58d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 0x7f58d80a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58e402a910 tx=0x7f58e40f5530 comp rx=0 tx=0).stop 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 msgr2=0x7f58f419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005940 tx=0x7f58e40040c0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 shutdown_connections 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58f4108f60 0x7f58f419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58f41047e0 0x7f58f419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 --2- 172.21.15.59:0/4236897666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58f4103e30 0x7f58f419a480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:02.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.723+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 >> 172.21.15.59:0/4236897666 conn(0x7f58f40ff8d0 msgr2=0x7f58f4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:02.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.724+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 shutdown_connections 2024-09-05T12:06:02.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:02.724+0000 7f58f3200640 1 -- 172.21.15.59:0/4236897666 wait complete. 2024-09-05T12:06:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:02 smithi059 ceph-mon[30025]: pgmap v645: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:06:02.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:02 smithi089 ceph-mon[34499]: pgmap v645: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:06:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:02 smithi189 ceph-mon[34772]: pgmap v645: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-05T12:06:03.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:06:03.933 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:03.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:03 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:06:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:03 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:03 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:06:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:03 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:03 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:06:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:03 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:03 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:06:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:03 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:04.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:03 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:06:04.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:03 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:04.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:03 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:06:04.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:03 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:04.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 -- 172.21.15.59:0/2245628444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060077040 msgr2=0x7fd0601124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 --2- 172.21.15.59:0/2245628444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060077040 0x7fd0601124f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009900 tx=0x7fd04c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 -- 172.21.15.59:0/2245628444 shutdown_connections 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 --2- 172.21.15.59:0/2245628444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 0x7fd060114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 --2- 172.21.15.59:0/2245628444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060077040 0x7fd0601124f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 --2- 172.21.15.59:0/2245628444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060076720 0x7fd060076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 -- 172.21.15.59:0/2245628444 >> 172.21.15.59:0/2245628444 conn(0x7fd0600ffa90 msgr2=0x7fd060101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 -- 172.21.15.59:0/2245628444 shutdown_connections 2024-09-05T12:06:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.454+0000 7fd067c00640 1 -- 172.21.15.59:0/2245628444 wait complete. 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 Processor -- start 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 -- start start 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060077040 0x7fd0601a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 0x7fd06019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601185c0 con 0x7fd060076720 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd060118440 con 0x7fd060112a30 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.455+0000 7fd067c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd060118740 con 0x7fd060077040 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50194/0 (socket says 172.21.15.59:50194) 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 -- 172.21.15.59:0/2507111424 learned_addr learned my addr 172.21.15.59:0/2507111424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd05fe00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060077040 0x7fd0601a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 0x7fd06019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 -- 172.21.15.59:0/2507111424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060077040 msgr2=0x7fd0601a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060077040 0x7fd0601a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 -- 172.21.15.59:0/2507111424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 msgr2=0x7fd06019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 0x7fd06019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 -- 172.21.15.59:0/2507111424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06019f100 con 0x7fd060076720 2024-09-05T12:06:04.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.456+0000 7fd065200640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9f0 tx=0x7fd05400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.457+0000 7fd05d600640 1 -- 172.21.15.59:0/2507111424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400ccc0 con 0x7fd060076720 2024-09-05T12:06:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.457+0000 7fd05d600640 1 -- 172.21.15.59:0/2507111424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05400ce60 con 0x7fd060076720 2024-09-05T12:06:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.457+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06019f3f0 con 0x7fd060076720 2024-09-05T12:06:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.457+0000 7fd05d600640 1 -- 172.21.15.59:0/2507111424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054010690 con 0x7fd060076720 2024-09-05T12:06:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.457+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601ab6e0 con 0x7fd060076720 2024-09-05T12:06:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.461+0000 7fd06a7ff740 1 Processor -- start 2024-09-05T12:06:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.461+0000 7fd06a7ff740 1 -- start start 2024-09-05T12:06:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd05d600640 1 -- 172.21.15.59:0/2507111424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0540046c0 con 0x7fd060076720 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d52160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac88cab3e0 con 0x55ac88d4fd70 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac88c52d80 con 0x55ac88d454b0 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac88ca73a0 con 0x55ac88d44b90 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd065c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d52160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd065c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35340/0 (socket says 172.21.15.59:35340) 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd065c00640 1 -- 172.21.15.59:0/1002615646 learned_addr learned my addr 172.21.15.59:0/1002615646 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:04.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.462+0000 7fd065200640 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 -- 172.21.15.59:0/1002615646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 msgr2=0x55ac88d4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 -- 172.21.15.59:0/1002615646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 msgr2=0x55ac88d52160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d52160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 -- 172.21.15.59:0/1002615646 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac88c71ae0 con 0x55ac88d44b90 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd065c00640 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd058009f40 tx=0x7fd0580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ac1b9e3de3dc5e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.463+0000 7fd052000640 1 -- 172.21.15.59:0/1002615646 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05801e070 con 0x55ac88d44b90 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd052000640 1 -- 172.21.15.59:0/1002615646 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058003c30 con 0x55ac88d44b90 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd052000640 1 -- 172.21.15.59:0/1002615646 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800f920 con 0x55ac88d44b90 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd05d600640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd05d600640 1 -- 172.21.15.59:0/2507111424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd0540cf870 con 0x7fd060076720 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd065200640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 -- 172.21.15.59:0/1002615646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 msgr2=0x55ac88d44f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd058009f40 tx=0x7fd0580118d0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 -- 172.21.15.59:0/1002615646 shutdown_connections 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d52160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/1002615646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d44f70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.464+0000 7fd06a7ff740 1 -- 172.21.15.59:0/1002615646 >> 172.21.15.59:0/1002615646 conn(0x55ac88d41120 msgr2=0x55ac88d41500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.465+0000 7fd06a7ff740 1 -- 172.21.15.59:0/1002615646 shutdown_connections 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.465+0000 7fd065200640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 0x7fd0480a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd05400d010 tx=0x7fd0540d5550 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.465+0000 7fd06a7ff740 1 -- 172.21.15.59:0/1002615646 wait complete. 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.465+0000 7fd06a7ff740 1 -- start start 2024-09-05T12:06:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.465+0000 7fd06a7ff740 1 Processor -- start 2024-09-05T12:06:04.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d454b0 0x55ac88d4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d4d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac88d529e0 con 0x55ac88d454b0 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ac88c660f0 con 0x55ac88d44b90 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac88d52b60 con 0x55ac88d4fd70 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd06a7ff740 10 client.0 _ll_register_callbacks cb 0x55ac88c579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44842/0 (socket says 172.21.15.59:44842) 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 -- 172.21.15.59:0/4127362287 learned_addr learned my addr 172.21.15.59:0/4127362287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd05fe00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d454b0 0x55ac88d4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d4d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 -- 172.21.15.59:0/4127362287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 msgr2=0x55ac88d4d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d4d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 -- 172.21.15.59:0/4127362287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d454b0 msgr2=0x55ac88d4d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d454b0 0x55ac88d4d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.466+0000 7fd065c00640 1 -- 172.21.15.59:0/4127362287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac88a1c3c0 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd065c00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fd0 tx=0x7fd05800f480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba66d9d6833687e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd046a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05801e070 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd058004460 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd058003c30 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd05800f6c0 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd06a7ff740 10 client.26705 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.467+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac88d5e6c0 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058003c30 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd058024070 con 0x55ac88d44b90 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 10 client.26705 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 20 client.26705 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88d5fac0 0x55ac88d61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 4 client.26705 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:04.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.468+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac88cb8950 con 0x55ac88d5fac0 2024-09-05T12:06:04.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.469+0000 7fd05fe00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88d5fac0 0x55ac88d61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.469+0000 7fd05fe00640 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88d5fac0 0x55ac88d61eb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9637559bbd0697e server_cookie=88226abd76942a76 in_seq=0 out_seq=0 2024-09-05T12:06:04.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.469+0000 7fd046a00640 10 client.26705 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd046a00640 1 -- 172.21.15.59:0/4127362287 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac88cb8950 con 0x55ac88d5fac0 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd046a00640 10 client.26705 handle_command_reply: tid=0 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 client.26705 shutdown 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 msgr2=0x55ac88d4cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fd0 tx=0x7fd05800f480 comp rx=0 tx=0).stop 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 shutdown_connections 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d4d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88d5fac0 0x55ac88d61eb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d454b0 0x55ac88d4d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/4127362287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d44b90 0x55ac88d4cc80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd046a00640 0 client.26705 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:04.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.470+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 >> 172.21.15.59:0/4127362287 conn(0x55ac88d41120 msgr2=0x55ac88d46e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.471+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 shutdown_connections 2024-09-05T12:06:04.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.471+0000 7fd06a7ff740 1 -- 172.21.15.59:0/4127362287 wait complete. 2024-09-05T12:06:04.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.471+0000 7fd06a7ff740 20 client.26705 trim_cache size 0 max 16384 2024-09-05T12:06:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 Processor -- start 2024-09-05T12:06:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 -- start start 2024-09-05T12:06:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b3490 con 0x55ac88d44b90 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b5d10 con 0x55ac88d454b0 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b3b00 con 0x55ac88d4fd70 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.489+0000 7fd065c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50214/0 (socket says 172.21.15.59:50214) 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 -- 172.21.15.59:0/3520631134 learned_addr learned my addr 172.21.15.59:0/3520631134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd05fe00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 -- 172.21.15.59:0/3520631134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 msgr2=0x55ac88d675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 -- 172.21.15.59:0/3520631134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 msgr2=0x55ac88d4bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 -- 172.21.15.59:0/3520631134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac88d409b0 con 0x55ac88d44b90 2024-09-05T12:06:04.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd05fe00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:04.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.490+0000 7fd065c00640 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0580099a0 tx=0x7fd058005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcdbec5fa19e01a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd051600640 1 -- 172.21.15.59:0/3520631134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05801e040 con 0x55ac88d44b90 2024-09-05T12:06:04.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd051600640 1 -- 172.21.15.59:0/3520631134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05802c070 con 0x55ac88d44b90 2024-09-05T12:06:04.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd051600640 1 -- 172.21.15.59:0/3520631134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058004730 con 0x55ac88d44b90 2024-09-05T12:06:04.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 -- 172.21.15.59:0/3520631134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 msgr2=0x55ac88d475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0580099a0 tx=0x7fd058005d30 comp rx=0 tx=0).stop 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 -- 172.21.15.59:0/3520631134 shutdown_connections 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d4fd70 0x55ac88d675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d4bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/3520631134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d44b90 0x55ac88d475f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 -- 172.21.15.59:0/3520631134 >> 172.21.15.59:0/3520631134 conn(0x55ac88d41120 msgr2=0x55ac88d4d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.491+0000 7fd06a7ff740 1 -- 172.21.15.59:0/3520631134 shutdown_connections 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.492+0000 7fd06a7ff740 1 -- 172.21.15.59:0/3520631134 wait complete. 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.492+0000 7fd06a7ff740 1 -- start start 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.492+0000 7fd06a7ff740 1 Processor -- start 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.492+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d58430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d53ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b41d0 con 0x55ac88d4fd70 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b5640 con 0x55ac88d454b0 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601b48a0 con 0x55ac88d44b90 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd06a7ff740 10 client.0 _ll_register_callbacks cb 0x7fd0540d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd065c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d58430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd065c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d53ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd065c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d58430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35372/0 (socket says 172.21.15.59:35372) 2024-09-05T12:06:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd065c00640 1 -- 172.21.15.59:0/2772600296 learned_addr learned my addr 172.21.15.59:0/2772600296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 -- 172.21.15.59:0/2772600296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 msgr2=0x55ac88d58430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d58430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 -- 172.21.15.59:0/2772600296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 msgr2=0x55ac88d53ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d53ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.493+0000 7fd05fe00640 1 -- 172.21.15.59:0/2772600296 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac88d544d0 con 0x55ac88d454b0 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd05fe00640 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d53760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005940 tx=0x7fd04c0317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce6cfec871e5a3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd046a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c035040 con 0x55ac88d454b0 2024-09-05T12:06:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd04c004240 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05802c070 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd04c058050 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd06a7ff740 10 client.26711 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.494+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ac88dca550 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05802c070 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd04c0485a0 con 0x55ac88d454b0 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 10 client.26711 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 20 client.26711 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88dcad50 0x55ac88dcd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 4 client.26711 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:06:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac88cb2e60 con 0x55ac88dcad50 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.495+0000 7fd065c00640 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88dcad50 0x55ac88dcd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd065c00640 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88dcad50 0x55ac88dcd170 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cbe6799be37ec6 server_cookie=5bba52b15c2e207f in_seq=0 out_seq=0 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd046a00640 10 client.26711 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd046a00640 1 -- 172.21.15.59:0/2772600296 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ac88cb2e60 con 0x55ac88dcad50 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd046a00640 10 client.26711 handle_command_reply: tid=0 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 client.26711 shutdown 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 msgr2=0x55ac88d53760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d53760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005940 tx=0x7fd04c0317d0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 shutdown_connections 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ac88dcad50 0x55ac88dcd170 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac88d4fd70 0x55ac88d53ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ac88d454b0 0x55ac88d53760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 --2- 172.21.15.59:0/2772600296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac88d44b90 0x55ac88d58430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 >> 172.21.15.59:0/2772600296 conn(0x55ac88d41120 msgr2=0x55ac88d4cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.496+0000 7fd046a00640 0 client.26711 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.497+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 shutdown_connections 2024-09-05T12:06:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.497+0000 7fd06a7ff740 1 -- 172.21.15.59:0/2772600296 wait complete. 2024-09-05T12:06:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.497+0000 7fd06a7ff740 20 client.26711 trim_cache size 0 max 16384 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "585b23d2-4545-4c94-95bd-a8e1ef1ba5c5": { 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "585b23d2-4545-4c94-95bd-a8e1ef1ba5c5", 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:04.502 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 msgr2=0x7fd0480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 0x7fd0480a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd05400d010 tx=0x7fd0540d5550 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 msgr2=0x7fd0601a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9f0 tx=0x7fd05400edf0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 shutdown_connections 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0480a2b80 0x7fd0480a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd060112a30 0x7fd06019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd060077040 0x7fd0601a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 --2- 172.21.15.59:0/2507111424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd060076720 0x7fd0601a4810 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.498+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 >> 172.21.15.59:0/2507111424 conn(0x7fd0600ffa90 msgr2=0x7fd060101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.499+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 shutdown_connections 2024-09-05T12:06:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:04.499+0000 7fd067c00640 1 -- 172.21.15.59:0/2507111424 wait complete. 2024-09-05T12:06:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:04 smithi189 ceph-mon[34772]: pgmap v646: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:06:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:04 smithi089 ceph-mon[34499]: pgmap v646: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:06:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:04 smithi059 ceph-mon[30025]: pgmap v646: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:06:05.341 INFO:tasks.cephfs.filesystem:scrub status for tag:585b23d2-4545-4c94-95bd-a8e1ef1ba5c5 - {'path': '/', 'tag': '585b23d2-4545-4c94-95bd-a8e1ef1ba5c5', 'options': 'recursive,force'} 2024-09-05T12:06:05.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:06:05.883 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:05.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:05 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:06:06.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:05 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:06:06.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:05 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 -- 172.21.15.59:0/1249754564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130076720 msgr2=0x7f3130076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 --2- 172.21.15.59:0/1249754564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130076720 0x7f3130076b00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3124009900 tx=0x7f312402a400 comp rx=0 tx=0).stop 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 -- 172.21.15.59:0/1249754564 shutdown_connections 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 --2- 172.21.15.59:0/1249754564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f3130114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 --2- 172.21.15.59:0/1249754564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130077040 0x7f3130112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 --2- 172.21.15.59:0/1249754564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130076720 0x7f3130076b00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 -- 172.21.15.59:0/1249754564 >> 172.21.15.59:0/1249754564 conn(0x7f31300ff910 msgr2=0x7f3130101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:06.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 -- 172.21.15.59:0/1249754564 shutdown_connections 2024-09-05T12:06:06.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.428+0000 7f3136200640 1 -- 172.21.15.59:0/1249754564 wait complete. 2024-09-05T12:06:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.429+0000 7f3136200640 1 Processor -- start 2024-09-05T12:06:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.429+0000 7f3136200640 1 -- start start 2024-09-05T12:06:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f313019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3130118400 con 0x7f3130077040 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3130118280 con 0x7f3130076720 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3136200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3130118580 con 0x7f3130112870 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f312fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3134e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3134e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44858/0 (socket says 172.21.15.59:44858) 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f312fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50240/0 (socket says 172.21.15.59:50240) 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3134e00640 1 -- 172.21.15.59:0/992784584 learned_addr learned my addr 172.21.15.59:0/992784584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.430+0000 7f3135800640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f313019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 -- 172.21.15.59:0/992784584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 msgr2=0x7f313019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f313019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 -- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 msgr2=0x7f31301a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:06.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 -- 172.21.15.59:0/992784584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313019efa0 con 0x7f3130076720 2024-09-05T12:06:06.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3135800640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f313019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:06:06.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f312fe00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:06.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3134e00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312403f030 con 0x7f3130076720 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124003ce0 con 0x7f3130076720 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124035070 con 0x7f3130076720 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.431+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313019f230 con 0x7f3130076720 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.432+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301ab410 con 0x7f3130076720 2024-09-05T12:06:06.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.433+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3130077e60 con 0x7f3130076720 2024-09-05T12:06:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.435+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3124042030 con 0x7f3130076720 2024-09-05T12:06:06.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.436+0000 7f312d600640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 0x7f31180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:06.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.436+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f31240efd80 con 0x7f3130076720 2024-09-05T12:06:06.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.436+0000 7f312fe00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 0x7f31180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:06.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.437+0000 7f312fe00640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 0x7f31180a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f313019ff80 tx=0x7f311c00b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:06.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.438+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31240bc930 con 0x7f3130076720 2024-09-05T12:06:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.662+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3130076b00 con 0x7f3130076720 2024-09-05T12:06:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.663+0000 7f312d600640 1 -- 172.21.15.59:0/992784584 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f31240bc230 con 0x7f3130076720 2024-09-05T12:06:06.665 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:06:06.665 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:06:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:06:06.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 msgr2=0x7f31180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:06.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 0x7f31180a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f313019ff80 tx=0x7f311c00b390 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 msgr2=0x7f31301a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240042f0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 shutdown_connections 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f31180a2c00 0x7f31180a50c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3130112870 0x7f313019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3130077040 0x7f31301a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 --2- 172.21.15.59:0/992784584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3130076720 0x7f31301a4660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 >> 172.21.15.59:0/992784584 conn(0x7f31300ff910 msgr2=0x7f3130101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 shutdown_connections 2024-09-05T12:06:06.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:06.666+0000 7f3136200640 1 -- 172.21.15.59:0/992784584 wait complete. 2024-09-05T12:06:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:06 smithi059 ceph-mon[30025]: pgmap v647: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:06:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:06 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/992784584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:06 smithi189 ceph-mon[34772]: pgmap v647: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:06:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:06 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/992784584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:06 smithi089 ceph-mon[34499]: pgmap v647: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-05T12:06:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:06 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/992784584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:07.470 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id b88a007b --release --await' 2024-09-05T12:06:07.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id b88a007b --release --await 2024-09-05T12:06:07.850 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 -- 172.21.15.59:0/1690289616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 msgr2=0x7fe7ec0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/1690289616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec0748f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a480 comp rx=0 tx=0).stop 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 -- 172.21.15.59:0/1690289616 shutdown_connections 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/1690289616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/1690289616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/1690289616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec0748f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 -- 172.21.15.59:0/1690289616 >> 172.21.15.59:0/1690289616 conn(0x7fe7ec0fd740 msgr2=0x7fe7ec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 -- 172.21.15.59:0/1690289616 shutdown_connections 2024-09-05T12:06:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.406+0000 7fe7ebe00640 1 -- 172.21.15.59:0/1690289616 wait complete. 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 Processor -- start 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 -- start start 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec113fb0 con 0x7fe7ec074510 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec113e30 con 0x7fe7ec074e30 2024-09-05T12:06:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.407+0000 7fe7ebe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec114130 con 0x7fe7ec10e280 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35408/0 (socket says 172.21.15.59:35408) 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7eb400640 1 -- 172.21.15.59:0/215072844 learned_addr learned my addr 172.21.15.59:0/215072844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 -- 172.21.15.59:0/215072844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 msgr2=0x7fe7ec19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 -- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 msgr2=0x7fe7ec19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7e2000640 1 -- 172.21.15.59:0/215072844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19f4d0 con 0x7fe7ec074e30 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.408+0000 7fe7eb400640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7e2000640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec19aa60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e9f0 tx=0x7fe7d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7e3e00640 1 -- 172.21.15.59:0/215072844 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400ccc0 con 0x7fe7ec074e30 2024-09-05T12:06:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19f7c0 con 0x7fe7ec074e30 2024-09-05T12:06:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7e3e00640 1 -- 172.21.15.59:0/215072844 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4010910 con 0x7fe7ec074e30 2024-09-05T12:06:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7e3e00640 1 -- 172.21.15.59:0/215072844 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40189c0 con 0x7fe7ec074e30 2024-09-05T12:06:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.409+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a70a0 con 0x7fe7ec074e30 2024-09-05T12:06:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.413+0000 7fe7f409b740 1 Processor -- start 2024-09-05T12:06:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.413+0000 7fe7f409b740 1 -- start start 2024-09-05T12:06:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7e3e00640 1 -- 172.21.15.59:0/215072844 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7d4010ac0 con 0x7fe7ec074e30 2024-09-05T12:06:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd007a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd007caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fcffd5650 con 0x561fd006fe40 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fcff7cf10 con 0x561fd006f520 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7f409b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561fcffd1610 con 0x561fd007a700 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44878/0 (socket says 172.21.15.59:44878) 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7eb400640 1 -- 172.21.15.59:0/1579921100 learned_addr learned my addr 172.21.15.59:0/1579921100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7e2000640 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd007caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.414+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd007a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 -- 172.21.15.59:0/1579921100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 msgr2=0x561fd007caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd007caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 -- 172.21.15.59:0/1579921100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 msgr2=0x561fd007a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd007a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 -- 172.21.15.59:0/1579921100 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fcfce4580 con 0x561fd006f520 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7eb400640 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4011810 tx=0x7fe7e4011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9b6a094a324491 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7db400640 1 -- 172.21.15.59:0/1579921100 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400fb40 con 0x561fd006f520 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7db400640 1 -- 172.21.15.59:0/1579921100 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e400fce0 con 0x561fd006f520 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7e3e00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 0x7fe7d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.415+0000 7fe7db400640 1 -- 172.21.15.59:0/1579921100 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4021600 con 0x561fd006f520 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.416+0000 7fe7e3e00640 1 -- 172.21.15.59:0/215072844 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe7d40cf7e0 con 0x7fe7ec074e30 2024-09-05T12:06:08.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.416+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 0x7fe7d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.416+0000 7fe7f409b740 1 -- 172.21.15.59:0/1579921100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 msgr2=0x561fd006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.416+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4011810 tx=0x7fe7e4011cd0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 -- 172.21.15.59:0/1579921100 shutdown_connections 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd007caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd007a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1579921100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd006f900 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 -- 172.21.15.59:0/1579921100 >> 172.21.15.59:0/1579921100 conn(0x561fd006bab0 msgr2=0x561fd006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 -- 172.21.15.59:0/1579921100 shutdown_connections 2024-09-05T12:06:08.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 -- 172.21.15.59:0/1579921100 wait complete. 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 0x7fe7d00a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009230 tx=0x7fe7dc0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.417+0000 7fe7f409b740 1 -- start start 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 Processor -- start 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006f520 0x561fd0077650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd007a700 0x561fd00780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561fd007d2d0 con 0x561fd006fe40 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561fcff90350 con 0x561fd007a700 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561fd007d450 con 0x561fd006f520 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7f409b740 10 client.0 _ll_register_callbacks cb 0x561fcffb3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.418+0000 7fe7e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50282/0 (socket says 172.21.15.59:50282) 2024-09-05T12:06:08.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 -- 172.21.15.59:0/2164198406 learned_addr learned my addr 172.21.15.59:0/2164198406 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:08.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006f520 0x561fd0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7eb400640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd007a700 0x561fd00780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 -- 172.21.15.59:0/2164198406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006f520 msgr2=0x561fd0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006f520 0x561fd0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 -- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd007a700 msgr2=0x561fd00780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd007a700 0x561fd00780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 -- 172.21.15.59:0/2164198406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fcfdae960 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7e2000640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409b980 tx=0x7fe7d4010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d613f28016e7bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.419+0000 7fe7cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d409b5e0 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e400fce0 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe7d40cfbd0 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe7d409bc80 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7f409b740 10 client.16734 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fd0088fd0 con 0x561fd006fe40 2024-09-05T12:06:08.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e400fce0 con 0x561fd006fe40 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.420+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe7d40d1ce0 con 0x561fd006fe40 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 10 client.16734 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 10 client.16734 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 20 client.16734 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd008a3e0 0x561fd008c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 4 client.16734 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fcffe2bc0 con 0x561fd008a3e0 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.421+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd008a3e0 0x561fd008c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.422+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd008a3e0 0x561fd008c810 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca3ed8029ebc7ad server_cookie=78f66c6890b69407 in_seq=0 out_seq=0 2024-09-05T12:06:08.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.422+0000 7fe7cfe00640 10 client.16734 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:08.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.422+0000 7fe7cfe00640 1 -- 172.21.15.59:0/2164198406 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fcffe2bc0 con 0x561fd008a3e0 2024-09-05T12:06:08.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.422+0000 7fe7cfe00640 10 client.16734 handle_command_reply: tid=0 2024-09-05T12:06:08.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 client.16734 shutdown 2024-09-05T12:06:08.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 msgr2=0x561fd0077b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe7d409b980 tx=0x7fe7d4010ee0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 shutdown_connections 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd007a700 0x561fd00780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd008a3e0 0x561fd008c810 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd0077b90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 --2- 172.21.15.59:0/2164198406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006f520 0x561fd0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 >> 172.21.15.59:0/2164198406 conn(0x561fd006bab0 msgr2=0x561fd006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7cfe00640 0 client.16734 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 shutdown_connections 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 1 -- 172.21.15.59:0/2164198406 wait complete. 2024-09-05T12:06:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.423+0000 7fe7f409b740 20 client.16734 trim_cache size 0 max 16384 2024-09-05T12:06:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.441+0000 7fe7f409b740 1 Processor -- start 2024-09-05T12:06:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.441+0000 7fe7f409b740 1 -- start start 2024-09-05T12:06:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd00920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1aabd0 con 0x561fd006fe40 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ad450 con 0x561fd006f520 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7f409b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ab240 con 0x561fd007a700 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50286/0 (socket says 172.21.15.59:50286) 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7e2000640 1 -- 172.21.15.59:0/3324194714 learned_addr learned my addr 172.21.15.59:0/3324194714 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.442+0000 7fe7eb400640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd00920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 -- 172.21.15.59:0/3324194714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 msgr2=0x561fd00920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd00920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 -- 172.21.15.59:0/3324194714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 msgr2=0x561fd00794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 -- 172.21.15.59:0/3324194714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd006cb60 con 0x561fd006f520 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7e2000640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd0071fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc005940 tx=0x7fe7dc03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd59213b57c81ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7daa00640 1 -- 172.21.15.59:0/3324194714 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035040 con 0x561fd006f520 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7daa00640 1 -- 172.21.15.59:0/3324194714 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc0543f0 con 0x561fd006f520 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.443+0000 7fe7daa00640 1 -- 172.21.15.59:0/3324194714 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc03d690 con 0x561fd006f520 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 -- 172.21.15.59:0/3324194714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 msgr2=0x561fd0071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd0071fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc005940 tx=0x7fe7dc03f000 comp rx=0 tx=0).stop 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 -- 172.21.15.59:0/3324194714 shutdown_connections 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd007a700 0x561fd00920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd006fe40 0x561fd00794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 --2- 172.21.15.59:0/3324194714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd0071fd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 -- 172.21.15.59:0/3324194714 >> 172.21.15.59:0/3324194714 conn(0x561fd006bab0 msgr2=0x561fd0078da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 -- 172.21.15.59:0/3324194714 shutdown_connections 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.444+0000 7fe7f409b740 1 -- 172.21.15.59:0/3324194714 wait complete. 2024-09-05T12:06:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 -- start start 2024-09-05T12:06:08.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 Processor -- start 2024-09-05T12:06:08.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006fe40 0x561fd007eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd007a700 0x561fd00f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ab910 con 0x561fd007a700 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1acd80 con 0x561fd006f520 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1abfe0 con 0x561fd006fe40 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.445+0000 7fe7f409b740 10 client.0 _ll_register_callbacks cb 0x7fe7d40d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44916/0 (socket says 172.21.15.59:44916) 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 -- 172.21.15.59:0/1335217449 learned_addr learned my addr 172.21.15.59:0/1335217449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7e2000640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006fe40 0x561fd007eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eb400640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd007a700 0x561fd00f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 -- 172.21.15.59:0/1335217449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006fe40 msgr2=0x561fd007eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006fe40 0x561fd007eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 -- 172.21.15.59:0/1335217449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd007a700 msgr2=0x561fd00f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:08.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd007a700 0x561fd00f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.446+0000 7fe7eaa00640 1 -- 172.21.15.59:0/1335217449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd006b360 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7eaa00640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc031710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf64f003fa08664 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc061020 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc0543f0 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe7dc003c50 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0545a0 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7f409b740 10 client.26735 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.447+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fd006c740 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc0543f0 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe7dc03d9f0 con 0x561fd006f520 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 10 client.26735 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 10 client.26735 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:06:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 20 client.26735 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd00f5b60 0x561fd00f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 4 client.26735 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b88a007b", "release": true, "await": true}] 2024-09-05T12:06:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b88a007b", "release": true, "await": true}) -- 0x561fcffdd0d0 con 0x561fd00f5b60 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.448+0000 7fe7e2000640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd00f5b60 0x561fd00f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.449+0000 7fe7e2000640 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd00f5b60 0x561fd00f7f80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10d46dd191ce965 server_cookie=559d2fa82ed45ea0 in_seq=0 out_seq=0 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.449+0000 7fe7cfe00640 10 client.26735 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.450+0000 7fe7cfe00640 1 -- 172.21.15.59:0/1335217449 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561fcffdd0d0 con 0x561fd00f5b60 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.450+0000 7fe7cfe00640 10 client.26735 handle_command_reply: tid=0 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.450+0000 7fe7f409b740 1 client.26735 shutdown 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 msgr2=0x561fd007e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc031710 comp rx=0 tx=0).stop 2024-09-05T12:06:08.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 shutdown_connections 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561fd00f5b60 0x561fd00f7f80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fd007a700 0x561fd00f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd006fe40 0x561fd007eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 --2- 172.21.15.59:0/1335217449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561fd006f520 0x561fd007e860 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7cfe00640 0 client.26735 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 >> 172.21.15.59:0/1335217449 conn(0x561fd006bab0 msgr2=0x561fd0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 shutdown_connections 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 1 -- 172.21.15.59:0/1335217449 wait complete. 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.451+0000 7fe7f409b740 20 client.26735 trim_cache size 0 max 16384 2024-09-05T12:06:08.454 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 32, 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "b88a007b": { 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 32, 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:06:08.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 146.2, 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:08.456 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 msgr2=0x7fe7d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 0x7fe7d00a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009230 tx=0x7fe7dc0058d0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 msgr2=0x7fe7ec19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec19aa60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e9f0 tx=0x7fe7d400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 shutdown_connections 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe7d00a2c00 0x7fe7d00a50c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7ec10e280 0x7fe7ec19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7ec074e30 0x7fe7ec19aa60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 --2- 172.21.15.59:0/215072844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec074510 0x7fe7ec19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 >> 172.21.15.59:0/215072844 conn(0x7fe7ec0fd740 msgr2=0x7fe7ec0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 shutdown_connections 2024-09-05T12:06:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:08.452+0000 7fe7ebe00640 1 -- 172.21.15.59:0/215072844 wait complete. 2024-09-05T12:06:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:08 smithi189 ceph-mon[34772]: pgmap v648: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:09.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:08 smithi059 ceph-mon[30025]: pgmap v648: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:08 smithi089 ceph-mon[34499]: pgmap v648: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:09.205 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b88a007b', total seconds elapsed: 29.9 2024-09-05T12:06:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:10 smithi189 ceph-mon[34772]: pgmap v649: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:11.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:10 smithi089 ceph-mon[34499]: pgmap v649: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:10 smithi059 ceph-mon[30025]: pgmap v649: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-05T12:06:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:12 smithi189 ceph-mon[34772]: pgmap v650: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:06:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:12 smithi089 ceph-mon[34499]: pgmap v650: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:06:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:12 smithi059 ceph-mon[30025]: pgmap v650: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-05T12:06:15.011 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:14 smithi059 ceph-mon[30025]: pgmap v651: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:06:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:14 smithi189 ceph-mon[34772]: pgmap v651: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:06:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:14 smithi089 ceph-mon[34499]: pgmap v651: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:06:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:16 smithi189 ceph-mon[34772]: pgmap v652: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:16 smithi089 ceph-mon[34499]: pgmap v652: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:17.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:16 smithi059 ceph-mon[30025]: pgmap v652: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:19.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:18 smithi189 ceph-mon[34772]: pgmap v653: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:18 smithi089 ceph-mon[34499]: pgmap v653: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:19.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:18 smithi059 ceph-mon[30025]: pgmap v653: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:21.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:20 smithi189 ceph-mon[34772]: pgmap v654: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:20 smithi089 ceph-mon[34499]: pgmap v654: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:21.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:20 smithi059 ceph-mon[30025]: pgmap v654: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:06:23.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:22 smithi189 ceph-mon[34772]: pgmap v655: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:06:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:22 smithi089 ceph-mon[34499]: pgmap v655: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:06:23.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:22 smithi059 ceph-mon[30025]: pgmap v655: 129 pgs: 129 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:06:25.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:24 smithi059 ceph-mon[30025]: pgmap v656: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:06:25.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:24 smithi189 ceph-mon[34772]: pgmap v656: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:06:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:24 smithi089 ceph-mon[34499]: pgmap v656: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:06:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:26 smithi189 ceph-mon[34772]: pgmap v657: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:26 smithi089 ceph-mon[34499]: pgmap v657: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:26 smithi059 ceph-mon[30025]: pgmap v657: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:28 smithi189 ceph-mon[34772]: pgmap v658: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:28 smithi089 ceph-mon[34499]: pgmap v658: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:29.217 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:28 smithi059 ceph-mon[30025]: pgmap v658: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:30 smithi189 ceph-mon[34772]: pgmap v659: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:30 smithi089 ceph-mon[34499]: pgmap v659: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:30 smithi059 ceph-mon[30025]: pgmap v659: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:32 smithi189 ceph-mon[34772]: pgmap v660: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:32 smithi089 ceph-mon[34499]: pgmap v660: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:33.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:32 smithi059 ceph-mon[30025]: pgmap v660: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:34 smithi189 ceph-mon[34772]: pgmap v661: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:34 smithi089 ceph-mon[34499]: pgmap v661: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:35.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:34 smithi059 ceph-mon[30025]: pgmap v661: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:35.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:06:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 -- 172.21.15.59:0/3076298674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d41049c0 msgr2=0x7f04d4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3076298674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d41049c0 0x7f04d4108a10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009900 tx=0x7f04bc02a410 comp rx=0 tx=0).stop 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 -- 172.21.15.59:0/3076298674 shutdown_connections 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3076298674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 0x7f04d4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3076298674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d41049c0 0x7f04d4108a10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3076298674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d4104010 0x7f04d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 -- 172.21.15.59:0/3076298674 >> 172.21.15.59:0/3076298674 conn(0x7f04d40ffa50 msgr2=0x7f04d4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 -- 172.21.15.59:0/3076298674 shutdown_connections 2024-09-05T12:06:36.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.328+0000 7f04d9a00640 1 -- 172.21.15.59:0/3076298674 wait complete. 2024-09-05T12:06:36.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.329+0000 7f04d9a00640 1 Processor -- start 2024-09-05T12:06:36.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.329+0000 7f04d9a00640 1 -- start start 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 0x7f04d419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 0x7f04d419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41141b0 con 0x7f04d4104010 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4114030 con 0x7f04d4109140 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d9a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4114330 con 0x7f04d41049c0 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60740/0 (socket says 172.21.15.59:60740) 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 -- 172.21.15.59:0/3106735030 learned_addr learned my addr 172.21.15.59:0/3106735030 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:36.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 0x7f04d419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d3400640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 0x7f04d419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 -- 172.21.15.59:0/3106735030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 msgr2=0x7f04d419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 0x7f04d419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 -- 172.21.15.59:0/3106735030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 msgr2=0x7f04d419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 0x7f04d419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04d2a00640 1 -- 172.21.15.59:0/3106735030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d419f6c0 con 0x7f04d4104010 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.330+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 0x7f04d419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04d2a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9f0 tx=0x7f04c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04cb400640 1 -- 172.21.15.59:0/3106735030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c400cd80 con 0x7f04d4104010 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04cb400640 1 -- 172.21.15.59:0/3106735030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c4005500 con 0x7f04d4104010 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d419f9b0 con 0x7f04d4104010 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04cb400640 1 -- 172.21.15.59:0/3106735030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4010690 con 0x7f04d4104010 2024-09-05T12:06:36.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.331+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a7290 con 0x7f04d4104010 2024-09-05T12:06:36.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.334+0000 7f04dc4a2740 1 Processor -- start 2024-09-05T12:06:36.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.334+0000 7f04cb400640 1 -- 172.21.15.59:0/3106735030 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04c40046c0 con 0x7f04d4104010 2024-09-05T12:06:36.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 -- start start 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c8ae04a3e0 con 0x55c8ae0eed70 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8adff1d80 con 0x55c8ae0e44b0 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8ae0463a0 con 0x55c8ae0e3b90 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04d3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ee830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50532/0 (socket says 172.21.15.59:50532) 2024-09-05T12:06:36.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.335+0000 7f04d2a00640 1 -- 172.21.15.59:0/4245757254 learned_addr learned my addr 172.21.15.59:0/4245757254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 -- 172.21.15.59:0/4245757254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 msgr2=0x55c8ae0ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 -- 172.21.15.59:0/4245757254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 msgr2=0x55c8ae0f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 -- 172.21.15.59:0/4245757254 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ae010ae0 con 0x55c8ae0e3b90 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04cb400640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 0x7f04b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d3400640 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e3f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009f00 tx=0x7f04cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a424f32a5b37156 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04cb400640 1 -- 172.21.15.59:0/3106735030 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f04c40cf940 con 0x7f04d4104010 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.336+0000 7f04d2a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 0x7f04b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04c3e00640 1 -- 172.21.15.59:0/4245757254 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004d20 con 0x55c8ae0e3b90 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04c3e00640 1 -- 172.21.15.59:0/4245757254 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc004050 con 0x55c8ae0e3b90 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04c3e00640 1 -- 172.21.15.59:0/4245757254 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc005280 con 0x55c8ae0e3b90 2024-09-05T12:06:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04dc4a2740 1 -- 172.21.15.59:0/4245757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 msgr2=0x55c8ae0e3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04d2a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 0x7f04b80a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9c0 tx=0x7f04c40d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.337+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e3f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009f00 tx=0x7f04cc011810 comp rx=0 tx=0).stop 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 -- 172.21.15.59:0/4245757254 shutdown_connections 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/4245757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e3f70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 -- 172.21.15.59:0/4245757254 >> 172.21.15.59:0/4245757254 conn(0x55c8ae0e0120 msgr2=0x55c8ae0e0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 -- 172.21.15.59:0/4245757254 shutdown_connections 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 -- 172.21.15.59:0/4245757254 wait complete. 2024-09-05T12:06:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.338+0000 7f04dc4a2740 1 -- start start 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 Processor -- start 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c8ae0f19e0 con 0x55c8ae0eed70 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8ae0050f0 con 0x55c8ae0e3b90 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8ae0f1b60 con 0x55c8ae0e44b0 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04dc4a2740 10 client.0 _ll_register_callbacks cb 0x55c8adff69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50546/0 (socket says 172.21.15.59:50546) 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.339+0000 7f04d3400640 1 -- 172.21.15.59:0/2591770094 learned_addr learned my addr 172.21.15.59:0/2591770094 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 -- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 msgr2=0x55c8ae0ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 -- 172.21.15.59:0/2591770094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 msgr2=0x55c8ae0ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 -- 172.21.15.59:0/2591770094 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8addbb3c0 con 0x55c8ae0e44b0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04cbe00640 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04bc02a930 tx=0x7f04bc005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ed76b53263435c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.340+0000 7f04c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc035040 con 0x55c8ae0e44b0 2024-09-05T12:06:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f04bc0048c0 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc004050 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f04bc005d50 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04dc4a2740 10 client.26683 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8ae0fd6f0 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.341+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc004050 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f04bc031b00 con 0x55c8ae0e44b0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 10 client.26683 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 20 client.26683 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae0febb0 0x55c8ae100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 4 client.26683 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8ae057950 con 0x55c8ae0febb0 2024-09-05T12:06:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.342+0000 7f04d3400640 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae0febb0 0x55c8ae100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.343+0000 7f04d3400640 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae0febb0 0x55c8ae100fa0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c162e42f015d5971 server_cookie=31d6339c04547583 in_seq=0 out_seq=0 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.343+0000 7f04c0c00640 10 client.26683 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04c0c00640 1 -- 172.21.15.59:0/2591770094 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8ae057950 con 0x55c8ae0febb0 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04c0c00640 10 client.26683 handle_command_reply: tid=0 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 client.26683 shutdown 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 msgr2=0x55c8ae0ec200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f04bc02a930 tx=0x7f04bc005270 comp rx=0 tx=0).stop 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 shutdown_connections 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae0febb0 0x55c8ae100fa0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae0ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec200 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04c0c00640 0 client.26683 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:36.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/2591770094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 >> 172.21.15.59:0/2591770094 conn(0x55c8ae0e0120 msgr2=0x55c8ae0e5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 shutdown_connections 2024-09-05T12:06:36.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.344+0000 7f04dc4a2740 1 -- 172.21.15.59:0/2591770094 wait complete. 2024-09-05T12:06:36.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.345+0000 7f04dc4a2740 20 client.26683 trim_cache size 0 max 16384 2024-09-05T12:06:36.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 Processor -- start 2024-09-05T12:06:36.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 -- start start 2024-09-05T12:06:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41af1c0 con 0x55c8ae0eed70 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b1a40 con 0x55c8ae0e44b0 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.363+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41af830 con 0x55c8ae0e3b90 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50560/0 (socket says 172.21.15.59:50560) 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 -- 172.21.15.59:0/3565221781 learned_addr learned my addr 172.21.15.59:0/3565221781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04d3400640 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04d3400640 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 -- 172.21.15.59:0/3565221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 msgr2=0x55c8ae0e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 -- 172.21.15.59:0/3565221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 msgr2=0x55c8ae106870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae106870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 -- 172.21.15.59:0/3565221781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ae0e0a60 con 0x55c8ae0e44b0 2024-09-05T12:06:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.364+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04bc031ed0 tx=0x7f04bc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec93486dc0babd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04c3400640 1 -- 172.21.15.59:0/3565221781 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc035070 con 0x55c8ae0e44b0 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04c3400640 1 -- 172.21.15.59:0/3565221781 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc040070 con 0x55c8ae0e44b0 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04c3400640 1 -- 172.21.15.59:0/3565221781 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc003e30 con 0x55c8ae0e44b0 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3565221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 msgr2=0x55c8ae0ec590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04bc031ed0 tx=0x7f04bc02a410 comp rx=0 tx=0).stop 2024-09-05T12:06:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3565221781 shutdown_connections 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0eed70 0x55c8ae106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0ec590 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3565221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3565221781 >> 172.21.15.59:0/3565221781 conn(0x55c8ae0e0120 msgr2=0x55c8ae0df1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3565221781 shutdown_connections 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.365+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3565221781 wait complete. 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.366+0000 7f04dc4a2740 1 -- start start 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.366+0000 7f04dc4a2740 1 Processor -- start 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0eed70 0x55c8ae168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41aff00 con 0x55c8ae0e44b0 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b1370 con 0x55c8ae0e3b90 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b05d0 con 0x55c8ae0eed70 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04dc4a2740 10 client.0 _ll_register_callbacks cb 0x7f04bc0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50576/0 (socket says 172.21.15.59:50576) 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 -- 172.21.15.59:0/3063948212 learned_addr learned my addr 172.21.15.59:0/3063948212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0eed70 0x55c8ae168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 -- 172.21.15.59:0/3063948212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0eed70 msgr2=0x55c8ae168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0eed70 0x55c8ae168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 -- 172.21.15.59:0/3063948212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0e44b0 msgr2=0x55c8ae0f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.367+0000 7f04d3400640 1 -- 172.21.15.59:0/3063948212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ae168d80 con 0x55c8ae0e3b90 2024-09-05T12:06:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04d3400640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000f80 tx=0x7f04cc013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99c99b049ec8a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004610 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f04cc028450 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc040070 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f04cc028650 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04dc4a2740 10 client.26753 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.368+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8ae16a090 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc040070 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f04cc037040 con 0x55c8ae0e3b90 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 10 client.26753 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 20 client.26753 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae16a5d0 0x55c8ae16c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 4 client.26753 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8ae051e60 con 0x55c8ae16a5d0 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.369+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae16a5d0 0x55c8ae16c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04cbe00640 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae16a5d0 0x55c8ae16c9c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe4cbf1ec828b91 server_cookie=d422664e6c4f2847 in_seq=0 out_seq=0 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04c0c00640 10 client.26753 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04c0c00640 1 -- 172.21.15.59:0/3063948212 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c8ae051e60 con 0x55c8ae16a5d0 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04c0c00640 10 client.26753 handle_command_reply: tid=0 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04dc4a2740 1 client.26753 shutdown 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 msgr2=0x55c8ae0f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.370+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000f80 tx=0x7f04cc013040 comp rx=0 tx=0).stop 2024-09-05T12:06:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 shutdown_connections 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8ae0eed70 0x55c8ae168840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8ae16a5d0 0x55c8ae16c9c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8ae0e44b0 0x55c8ae0f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 --2- 172.21.15.59:0/3063948212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8ae0e3b90 0x55c8ae0f32a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 >> 172.21.15.59:0/3063948212 conn(0x55c8ae0e0120 msgr2=0x55c8ae0ed520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04c0c00640 0 client.26753 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 shutdown_connections 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 1 -- 172.21.15.59:0/3063948212 wait complete. 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04dc4a2740 20 client.26753 trim_cache size 0 max 16384 2024-09-05T12:06:36.376 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:36.377 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:06:36.377 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:06:36.377 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 msgr2=0x7f04b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 0x7f04b80a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9c0 tx=0x7f04c40d5530 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 msgr2=0x7f04d419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.371+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9f0 tx=0x7f04c400edf0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 shutdown_connections 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f04b80a2b80 0x7f04b80a5040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04d4109140 0x7f04d419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d41049c0 0x7f04d419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 --2- 172.21.15.59:0/3106735030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04d4104010 0x7f04d419a710 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:36.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 >> 172.21.15.59:0/3106735030 conn(0x7f04d40ffa50 msgr2=0x7f04d4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:36.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 shutdown_connections 2024-09-05T12:06:36.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:36.372+0000 7f04d9a00640 1 -- 172.21.15.59:0/3106735030 wait complete. 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:251/600: open fd = 3 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:252/600: open fd = 3 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:253/600: open fd = 3 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.676 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:254/600: open fd = 3 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:255/600: open fd = 3 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:256/600: open fd = 3 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.677 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:257/600: open fd = 3 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:258/600: open fd = 3 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:259/600: open fd = 3 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:260/600: open fd = 3 2024-09-05T12:06:36.678 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:261/600: open fd = 3 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:262/600: open fd = 3 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:263/600: open fd = 3 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.679 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:264/600: open fd = 3 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:265/600: open fd = 3 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:266/600: open fd = 3 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:267/600: open fd = 3 2024-09-05T12:06:36.680 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:268/600: open fd = 3 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:269/600: open fd = 3 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:270/600: open fd = 3 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.681 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:271/600: open fd = 3 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:272/600: open fd = 3 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:273/600: open fd = 3 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.682 INFO:tasks.workunit.client.0.smithi059.stdout:274/600: open fd = 3 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:275/600: open fd = 3 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:276/600: open fd = 3 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.683 INFO:tasks.workunit.client.0.smithi059.stdout:277/600: open fd = 3 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:278/600: open fd = 3 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:279/600: open fd = 3 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:280/600: open fd = 3 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.684 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:281/600: open fd = 3 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:282/600: open fd = 3 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:283/600: open fd = 3 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:284/600: open fd = 3 2024-09-05T12:06:36.685 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:285/600: open fd = 3 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:286/600: open fd = 3 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:287/600: open fd = 3 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.686 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:288/600: open fd = 3 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:289/600: open fd = 3 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:290/600: open fd = 3 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:291/600: open fd = 3 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.687 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:292/600: open fd = 3 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:293/600: open fd = 3 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:294/600: open fd = 3 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.688 INFO:tasks.workunit.client.0.smithi059.stdout:295/600: open fd = 3 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:296/600: open fd = 3 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:297/600: open fd = 3 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:298/600: open fd = 3 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.689 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:299/600: open fd = 3 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:300/600: open fd = 3 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:301/600: open fd = 3 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.690 INFO:tasks.workunit.client.0.smithi059.stdout:302/600: open fd = 3 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:303/600: open fd = 3 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:304/600: open fd = 3 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:305/600: open fd = 3 2024-09-05T12:06:36.691 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:306/600: open fd = 3 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:307/600: open fd = 3 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:308/600: open fd = 3 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.692 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:309/600: open fd = 3 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:310/600: open fd = 3 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:311/600: open fd = 3 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:06:36.693 INFO:tasks.workunit.client.0.smithi059.stdout:312/600: open fd = 3 2024-09-05T12:06:36.694 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:36.694 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:06:37.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:36 smithi189 ceph-mon[34772]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:37.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:36 smithi059 ceph-mon[30025]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:36 smithi089 ceph-mon[34499]: pgmap v662: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:37.223 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:06:37.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:06:37.604 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 -- 172.21.15.59:0/3693065805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54076720 msgr2=0x7f8f54076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 --2- 172.21.15.59:0/3693065805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54076720 0x7f8f54076b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009900 tx=0x7f8f4402a410 comp rx=0 tx=0).stop 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 -- 172.21.15.59:0/3693065805 shutdown_connections 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 --2- 172.21.15.59:0/3693065805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54112a30 0x7f8f54114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 --2- 172.21.15.59:0/3693065805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54077040 0x7f8f541124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 --2- 172.21.15.59:0/3693065805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54076720 0x7f8f54076b00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 -- 172.21.15.59:0/3693065805 >> 172.21.15.59:0/3693065805 conn(0x7f8f540ffa90 msgr2=0x7f8f54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:38.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 -- 172.21.15.59:0/3693065805 shutdown_connections 2024-09-05T12:06:38.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.200+0000 7f8f59000640 1 -- 172.21.15.59:0/3693065805 wait complete. 2024-09-05T12:06:38.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.201+0000 7f8f59000640 1 Processor -- start 2024-09-05T12:06:38.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.201+0000 7f8f59000640 1 -- start start 2024-09-05T12:06:38.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54076720 0x7f8f541a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:38.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:38.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54112a30 0x7f8f5419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541185a0 con 0x7f8f54077040 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54118420 con 0x7f8f54112a30 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f59000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54118720 con 0x7f8f54076720 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54076720 0x7f8f541a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60804/0 (socket says 172.21.15.59:60804) 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 -- 172.21.15.59:0/826350844 learned_addr learned my addr 172.21.15.59:0/826350844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f52800640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54112a30 0x7f8f5419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 -- 172.21.15.59:0/826350844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54076720 msgr2=0x7f8f541a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54076720 0x7f8f541a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 -- 172.21.15.59:0/826350844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54112a30 msgr2=0x7f8f5419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:38.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54112a30 0x7f8f5419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.202+0000 7f8f49400640 1 -- 172.21.15.59:0/826350844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5419efb0 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f49400640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9f0 tx=0x7f8f3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00cd80 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c010910 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5419f2a0 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c018990 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.203+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541ab590 con 0x7f8f54077040 2024-09-05T12:06:38.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.204+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54076b00 con 0x7f8f54077040 2024-09-05T12:06:38.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.206+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f3c010ac0 con 0x7f8f54077040 2024-09-05T12:06:38.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.208+0000 7f8f4b400640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 0x7f8f380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:38.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.208+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8f3c0cf720 con 0x7f8f54077040 2024-09-05T12:06:38.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.208+0000 7f8f51e00640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 0x7f8f380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:38.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.209+0000 7f8f51e00640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 0x7f8f380a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f5419f9c0 tx=0x7f8f440058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:38.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.212+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f3c09bbc0 con 0x7f8f54077040 2024-09-05T12:06:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.434+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f541ab8b0 con 0x7f8f54077040 2024-09-05T12:06:38.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.435+0000 7f8f4b400640 1 -- 172.21.15.59:0/826350844 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f8f3c09fec0 con 0x7f8f54077040 2024-09-05T12:06:38.437 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:06:38.437 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:06:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:06:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 msgr2=0x7f8f380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 0x7f8f380a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f5419f9c0 tx=0x7f8f440058d0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 msgr2=0x7f8f541a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9f0 tx=0x7f8f3c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 shutdown_connections 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8f380a2c00 0x7f8f380a50c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f54112a30 0x7f8f5419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f54077040 0x7f8f541a4c00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 --2- 172.21.15.59:0/826350844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f54076720 0x7f8f541a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 >> 172.21.15.59:0/826350844 conn(0x7f8f540ffa90 msgr2=0x7f8f54101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 shutdown_connections 2024-09-05T12:06:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:38.438+0000 7f8f59000640 1 -- 172.21.15.59:0/826350844 wait complete. 2024-09-05T12:06:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:38 smithi189 ceph-mon[34772]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:38 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/826350844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:38 smithi089 ceph-mon[34499]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:38 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/826350844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:38 smithi059 ceph-mon[30025]: pgmap v663: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:38 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/826350844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:06:39.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:06:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 -- 172.21.15.59:0/3892152262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 msgr2=0x7f71f8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 --2- 172.21.15.59:0/3892152262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f8110ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009ca0 tx=0x7f71f002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 -- 172.21.15.59:0/3892152262 shutdown_connections 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 --2- 172.21.15.59:0/3892152262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f8110ca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 --2- 172.21.15.59:0/3892152262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81049a0 0x7f71f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 --2- 172.21.15.59:0/3892152262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8103ff0 0x7f71f81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 -- 172.21.15.59:0/3892152262 >> 172.21.15.59:0/3892152262 conn(0x7f71f80ffa90 msgr2=0x7f71f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 -- 172.21.15.59:0/3892152262 shutdown_connections 2024-09-05T12:06:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.244+0000 7f71ffc00640 1 -- 172.21.15.59:0/3892152262 wait complete. 2024-09-05T12:06:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 Processor -- start 2024-09-05T12:06:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 -- start start 2024-09-05T12:06:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 0x7f71f819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8114150 con 0x7f71f8109120 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8113fd0 con 0x7f71f81049a0 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.245+0000 7f71ffc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81142d0 con 0x7f71f8103ff0 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 0x7f71f819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50606/0 (socket says 172.21.15.59:50606) 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71efe00640 1 -- 172.21.15.59:0/1489970155 learned_addr learned my addr 172.21.15.59:0/1489970155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fdc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 -- 172.21.15.59:0/1489970155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 msgr2=0x7f71f819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 -- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 msgr2=0x7f71f819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71fd200640 1 -- 172.21.15.59:0/1489970155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f819f560 con 0x7f71f8103ff0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.246+0000 7f71efe00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71fd200640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 0x7f71f819a5b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71f6000640 1 -- 172.21.15.59:0/1489970155 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800ccc0 con 0x7f71f8103ff0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71f6000640 1 -- 172.21.15.59:0/1489970155 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e8010910 con 0x7f71f8103ff0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f819f850 con 0x7f71f8103ff0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71f6000640 1 -- 172.21.15.59:0/1489970155 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8018960 con 0x7f71f8103ff0 2024-09-05T12:06:40.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.247+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a7130 con 0x7f71f8103ff0 2024-09-05T12:06:40.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 Processor -- start 2024-09-05T12:06:40.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f71f6000640 1 -- 172.21.15.59:0/1489970155 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71e8010ac0 con 0x7f71f8103ff0 2024-09-05T12:06:40.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 -- start start 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4aa2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4aa5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571d49fe3e0 con 0x5571d4a97c70 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.250+0000 7f72026ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5571d49a5d80 con 0x5571d4aa2e50 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f72026ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571d49fa3a0 con 0x5571d4a98590 2024-09-05T12:06:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60830/0 (socket says 172.21.15.59:60830) 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 -- 172.21.15.59:0/3878789513 learned_addr learned my addr 172.21.15.59:0/3878789513 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71efe00640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4aa5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fd200640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4aa2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 -- 172.21.15.59:0/3878789513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 msgr2=0x5571d4aa2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4aa2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 -- 172.21.15.59:0/3878789513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 msgr2=0x5571d4aa5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4aa5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71fdc00640 1 -- 172.21.15.59:0/3878789513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d49c4ae0 con 0x5571d4a97c70 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.251+0000 7f71efe00640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4aa5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71fdc00640 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009ca0 tx=0x7f71f0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa65105e18d45b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71f6000640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 0x7f71dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71ee000640 1 -- 172.21.15.59:0/3878789513 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0031db0 con 0x5571d4a97c70 2024-09-05T12:06:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71f6000640 1 -- 172.21.15.59:0/1489970155 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71e80cf840 con 0x7f71f8103ff0 2024-09-05T12:06:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71ee000640 1 -- 172.21.15.59:0/3878789513 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0036040 con 0x5571d4a97c70 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71ee000640 1 -- 172.21.15.59:0/3878789513 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004330 con 0x5571d4a97c70 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.252+0000 7f71fd200640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 0x7f71dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 -- 172.21.15.59:0/3878789513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 msgr2=0x5571d4a98050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009ca0 tx=0x7f71f0005f10 comp rx=0 tx=0).stop 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f71fd200640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 0x7f71dc0a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f71e80986b0 tx=0x7f71e80d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 -- 172.21.15.59:0/3878789513 shutdown_connections 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4aa5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4aa2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 --2- 172.21.15.59:0/3878789513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a98050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 -- 172.21.15.59:0/3878789513 >> 172.21.15.59:0/3878789513 conn(0x5571d4a94200 msgr2=0x5571d4a945e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 -- 172.21.15.59:0/3878789513 shutdown_connections 2024-09-05T12:06:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.253+0000 7f72026ad740 1 -- 172.21.15.59:0/3878789513 wait complete. 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 -- start start 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 Processor -- start 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 0x5571d4a9fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aa0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 0x5571d4aa0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571d4aa5840 con 0x5571d4aa2e50 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5571d49b90f0 con 0x5571d4a98590 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571d4aa59c0 con 0x5571d4a97c70 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.254+0000 7f72026ad740 10 client.0 _ll_register_callbacks cb 0x5571d49a0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 0x5571d4a9fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aa0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 0x5571d4a9fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:59724/0 (socket says 172.21.15.59:59724) 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71fdc00640 1 -- 172.21.15.59:0/757620786 learned_addr learned my addr 172.21.15.59:0/757620786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71fdc00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 0x5571d4aa0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 -- 172.21.15.59:0/757620786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 msgr2=0x5571d4a9fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 0x5571d4a9fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 -- 172.21.15.59:0/757620786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 msgr2=0x5571d4aa0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 0x5571d4aa0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71efe00640 1 -- 172.21.15.59:0/757620786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d476f3c0 con 0x5571d4a98590 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.255+0000 7f71fdc00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 0x5571d4aa0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71efe00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aa0340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5571d4a95580 tx=0x7f71e000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8126d64934e39c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00120d0 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f0036040 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71e0005020 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f71e0012270 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f72026ad740 10 client.26771 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571d4ab19f0 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.256+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0036040 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f71e001c470 con 0x5571d4a98590 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 10 client.26771 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 10 client.26771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 20 client.26771 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4ab2f00 0x5571d4ab5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 4 client.26771 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d4a0b950 con 0x5571d4ab2f00 2024-09-05T12:06:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.257+0000 7f71fdc00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4ab2f00 0x5571d4ab5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.258+0000 7f71fdc00640 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4ab2f00 0x5571d4ab5320 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b15318d6cba06f server_cookie=83e38b270ffd811a in_seq=0 out_seq=0 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.258+0000 7f71e6a00640 10 client.26771 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.258+0000 7f71e6a00640 1 -- 172.21.15.59:0/757620786 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571d4a0b950 con 0x5571d4ab2f00 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.258+0000 7f71e6a00640 10 client.26771 handle_command_reply: tid=0 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 client.26771 shutdown 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 msgr2=0x5571d4aa0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aa0340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5571d4a95580 tx=0x7f71e000f790 comp rx=0 tx=0).stop 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 shutdown_connections 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4ab2f00 0x5571d4ab5320 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4aa2e50 0x5571d4aa0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aa0340 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f71e6a00640 0 client.26771 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 --2- 172.21.15.59:0/757620786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a97c70 0x5571d4a9fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 >> 172.21.15.59:0/757620786 conn(0x5571d4a94200 msgr2=0x5571d4a93380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 shutdown_connections 2024-09-05T12:06:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 1 -- 172.21.15.59:0/757620786 wait complete. 2024-09-05T12:06:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.259+0000 7f72026ad740 20 client.26771 trim_cache size 0 max 16384 2024-09-05T12:06:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 Processor -- start 2024-09-05T12:06:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 -- start start 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4ababb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4abb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81aefc0 con 0x5571d4a97c70 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81b1840 con 0x5571d4aa2e50 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.273+0000 7f72026ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81af630 con 0x5571d4a98590 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60852/0 (socket says 172.21.15.59:60852) 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 -- 172.21.15.59:0/755010160 learned_addr learned my addr 172.21.15.59:0/755010160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71efe00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4ababb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4abb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 -- 172.21.15.59:0/755010160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 msgr2=0x5571d4ababb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4ababb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 -- 172.21.15.59:0/755010160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 msgr2=0x5571d4abb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4abb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 -- 172.21.15.59:0/755010160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d4a983c0 con 0x5571d4a97c70 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71efe00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4ababb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.274+0000 7f71fdc00640 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f71f0004af0 tx=0x7f71f002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4243f6adeee16138 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f71ed600640 1 -- 172.21.15.59:0/755010160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0031570 con 0x5571d4a97c70 2024-09-05T12:06:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f71ed600640 1 -- 172.21.15.59:0/755010160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0036070 con 0x5571d4a97c70 2024-09-05T12:06:40.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f71ed600640 1 -- 172.21.15.59:0/755010160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004310 con 0x5571d4a97c70 2024-09-05T12:06:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 -- 172.21.15.59:0/755010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 msgr2=0x5571d4a93270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f71f0004af0 tx=0x7f71f002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 -- 172.21.15.59:0/755010160 shutdown_connections 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4aa2e50 0x5571d4abb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4a98590 0x5571d4ababb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 --2- 172.21.15.59:0/755010160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4a93270 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 -- 172.21.15.59:0/755010160 >> 172.21.15.59:0/755010160 conn(0x5571d4a94200 msgr2=0x5571d4a92b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 -- 172.21.15.59:0/755010160 shutdown_connections 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.275+0000 7f72026ad740 1 -- 172.21.15.59:0/755010160 wait complete. 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 -- start start 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 Processor -- start 2024-09-05T12:06:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4aa2e50 0x5571d4aa7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81afd00 con 0x5571d4a97c70 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81b1170 con 0x5571d4a98590 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.276+0000 7f72026ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81b03d0 con 0x5571d4aa2e50 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f72026ad740 10 client.0 _ll_register_callbacks cb 0x7f71e0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60862/0 (socket says 172.21.15.59:60862) 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 -- 172.21.15.59:0/1350443115 learned_addr learned my addr 172.21.15.59:0/1350443115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71efe00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4aa2e50 0x5571d4aa7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 -- 172.21.15.59:0/1350443115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4aa2e50 msgr2=0x5571d4aa7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4aa2e50 0x5571d4aa7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 -- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 msgr2=0x5571d4aab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71fdc00640 1 -- 172.21.15.59:0/1350443115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d4aa7970 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.277+0000 7f71efe00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f71fdc00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71f0031aa0 tx=0x7f71f0031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d1634e3e593fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f71e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004b40 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71f0004ce0 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f0036070 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f71f0003970 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f72026ad740 10 client.16782 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.278+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571d4b1e630 con 0x5571d4a97c70 2024-09-05T12:06:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0036070 con 0x5571d4a97c70 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f71f0050040 con 0x5571d4a97c70 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 10 client.16782 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 20 client.16782 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4b1e8c0 0x5571d4b20cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 4 client.16782 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571d4a05e60 con 0x5571d4b1e8c0 2024-09-05T12:06:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.279+0000 7f71efe00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4b1e8c0 0x5571d4b20cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f71efe00640 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4b1e8c0 0x5571d4b20cb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee754180ee79c68 server_cookie=3c6aeecfaf1e3966 in_seq=0 out_seq=0 2024-09-05T12:06:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f71e6a00640 10 client.16782 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f71e6a00640 1 -- 172.21.15.59:0/1350443115 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571d4a05e60 con 0x5571d4b1e8c0 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f71e6a00640 10 client.16782 handle_command_reply: tid=0 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 client.16782 shutdown 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 msgr2=0x5571d4aaad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f71f0031aa0 tx=0x7f71f0031450 comp rx=0 tx=0).stop 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 shutdown_connections 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571d4aa2e50 0x5571d4aa7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571d4a98590 0x5571d4aab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571d4b1e8c0 0x5571d4b20cb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 --2- 172.21.15.59:0/1350443115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571d4a97c70 0x5571d4aaad70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f71e6a00640 0 client.16782 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 >> 172.21.15.59:0/1350443115 conn(0x5571d4a94200 msgr2=0x5571d4aa05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.280+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 shutdown_connections 2024-09-05T12:06:40.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f72026ad740 1 -- 172.21.15.59:0/1350443115 wait complete. 2024-09-05T12:06:40.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f72026ad740 20 client.16782 trim_cache size 0 max 16384 2024-09-05T12:06:40.287 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 msgr2=0x7f71dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 0x7f71dc0a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f71e80986b0 tx=0x7f71e80d5530 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 msgr2=0x7f71f819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 0x7f71f819a5b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 shutdown_connections 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71dc0a2b80 0x7f71dc0a5040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71f8109120 0x7f71f819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f81049a0 0x7f71f819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 --2- 172.21.15.59:0/1489970155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8103ff0 0x7f71f819a5b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:40.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 >> 172.21.15.59:0/1489970155 conn(0x7f71f80ffa90 msgr2=0x7f71f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:40.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.281+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 shutdown_connections 2024-09-05T12:06:40.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:40.282+0000 7f71ffc00640 1 -- 172.21.15.59:0/1489970155 wait complete. 2024-09-05T12:06:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:40 smithi059 ceph-mon[30025]: pgmap v664: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:06:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:40 smithi189 ceph-mon[34772]: pgmap v664: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:06:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:40 smithi089 ceph-mon[34499]: pgmap v664: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:06:42.149 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:06:42.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:06:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:42 smithi059 ceph-mon[30025]: pgmap v665: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:42 smithi189 ceph-mon[34772]: pgmap v665: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 -- 172.21.15.59:0/1961874116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c68104010 msgr2=0x7f4c681043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 --2- 172.21.15.59:0/1961874116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c68104010 0x7f4c681043f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009900 tx=0x7f4c5c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 -- 172.21.15.59:0/1961874116 shutdown_connections 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 --2- 172.21.15.59:0/1961874116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68109140 0x7f4c68110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 --2- 172.21.15.59:0/1961874116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c681049c0 0x7f4c68108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 --2- 172.21.15.59:0/1961874116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c68104010 0x7f4c681043f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 -- 172.21.15.59:0/1961874116 >> 172.21.15.59:0/1961874116 conn(0x7f4c680ffa50 msgr2=0x7f4c68101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 -- 172.21.15.59:0/1961874116 shutdown_connections 2024-09-05T12:06:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.088+0000 7f4c6f600640 1 -- 172.21.15.59:0/1961874116 wait complete. 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.089+0000 7f4c6f600640 1 Processor -- start 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.089+0000 7f4c6f600640 1 -- start start 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68104010 0x7f4c6819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68114140 con 0x7f4c681049c0 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68113fc0 con 0x7f4c68104010 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6f600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681142c0 con 0x7f4c68109140 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:49154/0 (socket says 172.21.15.59:49154) 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34028/0 (socket says 172.21.15.59:34028) 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6d600640 1 -- 172.21.15.59:0/32352601 learned_addr learned my addr 172.21.15.59:0/32352601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68104010 0x7f4c6819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 -- 172.21.15.59:0/32352601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 msgr2=0x7f4c6819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 -- 172.21.15.59:0/32352601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68104010 msgr2=0x7f4c6819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68104010 0x7f4c6819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c67e00640 1 -- 172.21.15.59:0/32352601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6819f690 con 0x7f4c681049c0 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.090+0000 7f4c6d600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c67e00640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005fd0 tx=0x7f4c5400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c65600640 1 -- 172.21.15.59:0/32352601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54009c30 con 0x7f4c681049c0 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c65600640 1 -- 172.21.15.59:0/32352601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54009dd0 con 0x7f4c681049c0 2024-09-05T12:06:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c65600640 1 -- 172.21.15.59:0/32352601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54010460 con 0x7f4c681049c0 2024-09-05T12:06:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6819f980 con 0x7f4c681049c0 2024-09-05T12:06:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.091+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a7260 con 0x7f4c681049c0 2024-09-05T12:06:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.095+0000 7f4c720c0740 1 Processor -- start 2024-09-05T12:06:43.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.095+0000 7f4c720c0740 1 -- start start 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc34a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc34d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5619dc2a6290 con 0x5619dc33fac0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5619dc24db40 con 0x5619dc3403e0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c720c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5619dc2a2250 con 0x5619dc34aca0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c65600640 1 -- 172.21.15.59:0/32352601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c54003ef0 con 0x7f4c681049c0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c6d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc34a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c6d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34040/0 (socket says 172.21.15.59:34040) 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c6d600640 1 -- 172.21.15.59:0/3422937525 learned_addr learned my addr 172.21.15.59:0/3422937525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.096+0000 7f4c67e00640 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc34d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3422937525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 msgr2=0x5619dc34d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc34d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3422937525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 msgr2=0x5619dc33fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3422937525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dc0176a0 con 0x5619dc3403e0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6d600640 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc34a760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005940 tx=0x7f4c5c005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3def2116450ed6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.097+0000 7f4c59c00640 1 -- 172.21.15.59:0/3422937525 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c040070 con 0x5619dc3403e0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c65600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 0x7f4c500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c59c00640 1 -- 172.21.15.59:0/3422937525 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c031830 con 0x5619dc3403e0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c59c00640 1 -- 172.21.15.59:0/3422937525 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c035040 con 0x5619dc3403e0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c65600640 1 -- 172.21.15.59:0/32352601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4c540cef40 con 0x7f4c681049c0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c6d600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 0x7f4c500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c720c0740 1 -- 172.21.15.59:0/3422937525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 msgr2=0x5619dc34a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.098+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc34a760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005940 tx=0x7f4c5c005e00 comp rx=0 tx=0).stop 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c6d600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 0x7f4c500a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c60011880 tx=0x7f4c60009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 -- 172.21.15.59:0/3422937525 shutdown_connections 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc34d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc34a760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3422937525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc33fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 -- 172.21.15.59:0/3422937525 >> 172.21.15.59:0/3422937525 conn(0x5619dc33c050 msgr2=0x5619dc33c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 -- 172.21.15.59:0/3422937525 shutdown_connections 2024-09-05T12:06:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 -- 172.21.15.59:0/3422937525 wait complete. 2024-09-05T12:06:43.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 -- start start 2024-09-05T12:06:43.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.099+0000 7f4c720c0740 1 Processor -- start 2024-09-05T12:06:43.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc3403e0 0x5619dc3478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc347e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5619dc34d8b0 con 0x5619dc34aca0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5619dc260fa0 con 0x5619dc33fac0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5619dc34da30 con 0x5619dc3403e0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c720c0740 10 client.0 _ll_register_callbacks cb 0x5619dbfb4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c6d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc347e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:54974/0 (socket says 172.21.15.59:54974) 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c6cc00640 1 -- 172.21.15.59:0/733171210 learned_addr learned my addr 172.21.15.59:0/733171210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.100+0000 7f4c67e00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc3403e0 0x5619dc3478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 -- 172.21.15.59:0/733171210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc3403e0 msgr2=0x5619dc3478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc3403e0 0x5619dc3478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 -- 172.21.15.59:0/733171210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 msgr2=0x5619dc347e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc347e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 -- 172.21.15.59:0/733171210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dc348640 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c006010 tx=0x7f4c5c0319e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1c7da78a8abf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c040030 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c031830 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.101+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4c5c004030 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c035070 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 10 client.26789 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619dc3597f0 con 0x5619dc33fac0 2024-09-05T12:06:43.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c031830 con 0x5619dc33fac0 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4c5c0035f0 con 0x5619dc33fac0 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 10 client.26789 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 20 client.26789 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc35ab40 0x5619dc35cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 4 client.26789 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.102+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619dc2b3800 con 0x5619dc35ab40 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.103+0000 7f4c67e00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc35ab40 0x5619dc35cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.103+0000 7f4c67e00640 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc35ab40 0x5619dc35cf30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1069b03dd3bcfe69 server_cookie=5426c0bb27d4c3f8 in_seq=0 out_seq=0 2024-09-05T12:06:43.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.103+0000 7f4c4e000640 10 client.26789 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:43.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c4e000640 1 -- 172.21.15.59:0/733171210 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619dc2b3800 con 0x5619dc35ab40 2024-09-05T12:06:43.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c4e000640 10 client.26789 handle_command_reply: tid=0 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 client.26789 shutdown 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 msgr2=0x5619dc34a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c006010 tx=0x7f4c5c0319e0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 shutdown_connections 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc35ab40 0x5619dc35cf30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc347e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc3403e0 0x5619dc3478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 --2- 172.21.15.59:0/733171210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc33fac0 0x5619dc34a600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c4e000640 0 client.26789 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 >> 172.21.15.59:0/733171210 conn(0x5619dc33c050 msgr2=0x5619dc33b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 shutdown_connections 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.104+0000 7f4c720c0740 1 -- 172.21.15.59:0/733171210 wait complete. 2024-09-05T12:06:43.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.105+0000 7f4c720c0740 20 client.26789 trim_cache size 0 max 16384 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.122+0000 7f4c720c0740 1 Processor -- start 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 -- start start 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc362a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc344b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681af170 con 0x5619dc33fac0 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b19f0 con 0x5619dc3403e0 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c720c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681af7e0 con 0x5619dc34aca0 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34048/0 (socket says 172.21.15.59:34048) 2024-09-05T12:06:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3913875785 learned_addr learned my addr 172.21.15.59:0/3913875785 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c6d600640 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc344b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.123+0000 7f4c67e00640 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc362a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3913875785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 msgr2=0x5619dc344b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc344b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3913875785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 msgr2=0x5619dc362a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc362a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 -- 172.21.15.59:0/3913875785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dc345e40 con 0x5619dc33fac0 2024-09-05T12:06:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b4e8b07b60fd24 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c59200640 1 -- 172.21.15.59:0/3913875785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c040040 con 0x5619dc33fac0 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c59200640 1 -- 172.21.15.59:0/3913875785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c031830 con 0x5619dc33fac0 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.124+0000 7f4c59200640 1 -- 172.21.15.59:0/3913875785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c004e00 con 0x5619dc33fac0 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 -- 172.21.15.59:0/3913875785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 msgr2=0x5619dc362500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 -- 172.21.15.59:0/3913875785 shutdown_connections 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc34aca0 0x5619dc344b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc362a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 --2- 172.21.15.59:0/3913875785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc33fac0 0x5619dc362500 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 -- 172.21.15.59:0/3913875785 >> 172.21.15.59:0/3913875785 conn(0x5619dc33c050 msgr2=0x5619dc349930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 -- 172.21.15.59:0/3913875785 shutdown_connections 2024-09-05T12:06:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.125+0000 7f4c720c0740 1 -- 172.21.15.59:0/3913875785 wait complete. 2024-09-05T12:06:43.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 -- start start 2024-09-05T12:06:43.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 Processor -- start 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc3515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc3c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681afeb0 con 0x5619dc34aca0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b1320 con 0x5619dc3403e0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b0580 con 0x5619dc33fac0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c720c0740 10 client.0 _ll_register_callbacks cb 0x5619dc348810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.126+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:49206/0 (socket says 172.21.15.59:49206) 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 -- 172.21.15.59:0/573434699 learned_addr learned my addr 172.21.15.59:0/573434699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6d600640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc3c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c67e00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc3515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 -- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 msgr2=0x5619dc3515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc3515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 -- 172.21.15.59:0/573434699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 msgr2=0x5619dc3c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc3c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 -- 172.21.15.59:0/573434699 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dc3c4a20 con 0x5619dc33fac0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c67e00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc3515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c6cc00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009800 tx=0x7f4c5c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e4385350cb6d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.127+0000 7f4c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c06a060 con 0x5619dc33fac0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c031830 con 0x5619dc33fac0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4c5c0048d0 con 0x5619dc33fac0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c071050 con 0x5619dc33fac0 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c720c0740 10 client.26731 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619dc3c5cc0 con 0x5619dc33fac0 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.128+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c031830 con 0x5619dc33fac0 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4c5c03c440 con 0x5619dc33fac0 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 10 client.26731 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 20 client.26731 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc3c6140 0x5619dc3c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 4 client.26731 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5619dc2add10 con 0x5619dc3c6140 2024-09-05T12:06:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.129+0000 7f4c67e00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc3c6140 0x5619dc3c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.130+0000 7f4c67e00640 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc3c6140 0x5619dc3c8530 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87e4af314df6163 server_cookie=b47740d3b0a7ff96 in_seq=0 out_seq=0 2024-09-05T12:06:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.130+0000 7f4c4e000640 10 client.26731 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c4e000640 1 -- 172.21.15.59:0/573434699 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5619dc2add10 con 0x5619dc3c6140 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c4e000640 10 client.26731 handle_command_reply: tid=0 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 client.26731 shutdown 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 msgr2=0x5619dc3510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009800 tx=0x7f4c5c003a90 comp rx=0 tx=0).stop 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 shutdown_connections 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5619dc3c6140 0x5619dc3c8530 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5619dc34aca0 0x5619dc3c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5619dc3403e0 0x5619dc3515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 --2- 172.21.15.59:0/573434699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dc33fac0 0x5619dc3510b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 >> 172.21.15.59:0/573434699 conn(0x5619dc33c050 msgr2=0x5619dc349110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c4e000640 0 client.26731 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 shutdown_connections 2024-09-05T12:06:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 1 -- 172.21.15.59:0/573434699 wait complete. 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.131+0000 7f4c720c0740 20 client.26731 trim_cache size 0 max 16384 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "5d3e92f7-e796-4876-b365-1087046e5832", 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:06:43.136 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 msgr2=0x7f4c500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 0x7f4c500a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c60011880 tx=0x7f4c60009500 comp rx=0 tx=0).stop 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 msgr2=0x7f4c6819ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005fd0 tx=0x7f4c5400dde0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 shutdown_connections 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4c500a2c00 0x7f4c500a50c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c68109140 0x7f4c6819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c681049c0 0x7f4c6819ac20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 --2- 172.21.15.59:0/32352601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c68104010 0x7f4c6819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:43.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 >> 172.21.15.59:0/32352601 conn(0x7f4c680ffa50 msgr2=0x7f4c68100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:43.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 shutdown_connections 2024-09-05T12:06:43.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:43.132+0000 7f4c6f600640 1 -- 172.21.15.59:0/32352601 wait complete. 2024-09-05T12:06:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:42 smithi089 ceph-mon[34499]: pgmap v665: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:06:43.876 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:06:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 -- 172.21.15.59:0/1004922414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c101df0 msgr2=0x7ff88c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 --2- 172.21.15.59:0/1004922414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c101df0 0x7ff88c1021d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009900 tx=0x7ff87c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 -- 172.21.15.59:0/1004922414 shutdown_connections 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 --2- 172.21.15.59:0/1004922414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c106f20 0x7ff88c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 --2- 172.21.15.59:0/1004922414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c1027a0 0x7ff88c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 --2- 172.21.15.59:0/1004922414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c101df0 0x7ff88c1021d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 -- 172.21.15.59:0/1004922414 >> 172.21.15.59:0/1004922414 conn(0x7ff88c0fd8b0 msgr2=0x7ff88c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 -- 172.21.15.59:0/1004922414 shutdown_connections 2024-09-05T12:06:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.822+0000 7ff891600640 1 -- 172.21.15.59:0/1004922414 wait complete. 2024-09-05T12:06:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.824+0000 7ff891600640 1 Processor -- start 2024-09-05T12:06:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.824+0000 7ff891600640 1 -- start start 2024-09-05T12:06:44.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.824+0000 7ff891600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 0x7ff88c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff891600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff891600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c106f20 0x7ff88c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff891600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c111fd0 con 0x7ff88c101df0 2024-09-05T12:06:44.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff891600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c111e50 con 0x7ff88c106f20 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff891600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c112150 con 0x7ff88c1027a0 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 0x7ff88c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:49226/0 (socket says 172.21.15.59:49226) 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 -- 172.21.15.59:0/4080798605 learned_addr learned my addr 172.21.15.59:0/4080798605 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 0x7ff88c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34066/0 (socket says 172.21.15.59:34066) 2024-09-05T12:06:44.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff88b400640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c106f20 0x7ff88c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 -- 172.21.15.59:0/4080798605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c106f20 msgr2=0x7ff88c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c106f20 0x7ff88c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 -- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 msgr2=0x7ff88c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 0x7ff88c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.825+0000 7ff883e00640 1 -- 172.21.15.59:0/4080798605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c19f580 con 0x7ff88c1027a0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff883e00640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e9f0 tx=0x7ff87400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff883400640 1 -- 172.21.15.59:0/4080798605 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87400cd80 con 0x7ff88c1027a0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff883400640 1 -- 172.21.15.59:0/4080798605 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874010910 con 0x7ff88c1027a0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c19f870 con 0x7ff88c1027a0 2024-09-05T12:06:44.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff883400640 1 -- 172.21.15.59:0/4080798605 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874018990 con 0x7ff88c1027a0 2024-09-05T12:06:44.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.826+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c1a7150 con 0x7ff88c1027a0 2024-09-05T12:06:44.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.829+0000 7ff8940a5740 1 Processor -- start 2024-09-05T12:06:44.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.829+0000 7ff8940a5740 1 -- start start 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe6473e0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe5eed80 con 0x55bdbe6e0b90 2024-09-05T12:06:44.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff8940a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe6433a0 con 0x55bdbe6ebd70 2024-09-05T12:06:44.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:55024/0 (socket says 172.21.15.59:55024) 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 -- 172.21.15.59:0/2944069119 learned_addr learned my addr 172.21.15.59:0/2944069119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 -- 172.21.15.59:0/2944069119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 msgr2=0x55bdbe6ee160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6ee160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 -- 172.21.15.59:0/2944069119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 msgr2=0x55bdbe6eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.830+0000 7ff88b400640 1 -- 172.21.15.59:0/2944069119 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbe60dae0 con 0x55bdbe6e0b90 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.831+0000 7ff88b400640 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff884009f50 tx=0x7ff8840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=780976470a4448e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.831+0000 7ff883400640 1 -- 172.21.15.59:0/4080798605 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff874010ac0 con 0x7ff88c1027a0 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.831+0000 7ff87b400640 1 -- 172.21.15.59:0/2944069119 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884004b20 con 0x55bdbe6e0b90 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.831+0000 7ff87b400640 1 -- 172.21.15.59:0/2944069119 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884004cc0 con 0x55bdbe6e0b90 2024-09-05T12:06:44.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.831+0000 7ff87b400640 1 -- 172.21.15.59:0/2944069119 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884005430 con 0x55bdbe6e0b90 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- 172.21.15.59:0/2944069119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 msgr2=0x55bdbe6e0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff884009f50 tx=0x7ff8840118d0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- 172.21.15.59:0/2944069119 shutdown_connections 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2944069119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e0f70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- 172.21.15.59:0/2944069119 >> 172.21.15.59:0/2944069119 conn(0x55bdbe6dd120 msgr2=0x55bdbe6dd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- 172.21.15.59:0/2944069119 shutdown_connections 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- 172.21.15.59:0/2944069119 wait complete. 2024-09-05T12:06:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.832+0000 7ff8940a5740 1 -- start start 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff883400640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 0x7ff8700a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 Processor -- start 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff883400640 1 -- 172.21.15.59:0/4080798605 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff8740cf720 con 0x7ff88c1027a0 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff88b400640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 0x7ff8700a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6e9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe6ee9e0 con 0x55bdbe6e0b90 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe6020f0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdbe6eeb60 con 0x55bdbe6ebd70 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.833+0000 7ff8940a5740 10 client.0 _ll_register_callbacks cb 0x55bdbe5f39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34074/0 (socket says 172.21.15.59:34074) 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88aa00640 1 -- 172.21.15.59:0/1964483853 learned_addr learned my addr 172.21.15.59:0/1964483853 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:55036/0 (socket says 172.21.15.59:55036) 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88b400640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 0x7ff8700a4fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff884009d60 tx=0x7ff884005bd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88b400640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 -- 172.21.15.59:0/1964483853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 msgr2=0x55bdbe6e9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6e9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 -- 172.21.15.59:0/1964483853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 msgr2=0x55bdbe6e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff883e00640 1 -- 172.21.15.59:0/1964483853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbe3b83c0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.834+0000 7ff88aa00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:06:44.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff883e00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8740ccd80 tx=0x7ff874011ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43dd1447bf7f57f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff86fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8740a4780 con 0x55bdbe6e14b0 2024-09-05T12:06:44.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff884004cc0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff874018b30 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff874018d30 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff8940a5740 10 client.26807 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bdbe6fa710 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.835+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884004cc0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff874097910 con 0x55bdbe6e14b0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 10 client.26807 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 20 client.26807 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe6fbb20 0x55bdbe6fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 4 client.26807 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdbe654950 con 0x55bdbe6fbb20 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.836+0000 7ff88aa00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe6fbb20 0x55bdbe6fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.837+0000 7ff88aa00640 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe6fbb20 0x55bdbe6fdf10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1b89c2417728e4 server_cookie=4d66c88178518fed in_seq=0 out_seq=0 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.837+0000 7ff86fe00640 10 client.26807 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.837+0000 7ff86fe00640 1 -- 172.21.15.59:0/1964483853 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdbe654950 con 0x55bdbe6fbb20 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.837+0000 7ff86fe00640 10 client.26807 handle_command_reply: tid=0 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 client.26807 shutdown 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 msgr2=0x55bdbe6e91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8740ccd80 tx=0x7ff874011ee0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 shutdown_connections 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6e9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6e91f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe6fbb20 0x55bdbe6fdf10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 --2- 172.21.15.59:0/1964483853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 >> 172.21.15.59:0/1964483853 conn(0x55bdbe6dd120 msgr2=0x55bdbe6e2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff86fe00640 0 client.26807 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:44.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 shutdown_connections 2024-09-05T12:06:44.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 1 -- 172.21.15.59:0/1964483853 wait complete. 2024-09-05T12:06:44.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.838+0000 7ff8940a5740 20 client.26807 trim_cache size 0 max 16384 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.856+0000 7ff8940a5740 1 Processor -- start 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.856+0000 7ff8940a5740 1 -- start start 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.856+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe703720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6ebd70 0x55bdbe703c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff8940a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1af090 con 0x55bdbe6ebd70 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff8940a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b1910 con 0x55bdbe6e14b0 2024-09-05T12:06:44.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff8940a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1af700 con 0x55bdbe6e0b90 2024-09-05T12:06:44.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe703720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:49238/0 (socket says 172.21.15.59:49238) 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff88aa00640 1 -- 172.21.15.59:0/2966419554 learned_addr learned my addr 172.21.15.59:0/2966419554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff88b400640 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6ebd70 0x55bdbe703c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 -- 172.21.15.59:0/2966419554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 msgr2=0x55bdbe6e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 -- 172.21.15.59:0/2966419554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6ebd70 msgr2=0x55bdbe703c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6ebd70 0x55bdbe703c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.857+0000 7ff883e00640 1 -- 172.21.15.59:0/2966419554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbe6e7c60 con 0x55bdbe6e14b0 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.858+0000 7ff883e00640 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe703720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff874098540 tx=0x7ff87409a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d488026909f2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.858+0000 7ff87aa00640 1 -- 172.21.15.59:0/2966419554 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8740a4780 con 0x55bdbe6e14b0 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.858+0000 7ff87aa00640 1 -- 172.21.15.59:0/2966419554 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874097910 con 0x55bdbe6e14b0 2024-09-05T12:06:44.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.858+0000 7ff87aa00640 1 -- 172.21.15.59:0/2966419554 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87409ae20 con 0x55bdbe6e14b0 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- 172.21.15.59:0/2966419554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 msgr2=0x55bdbe703720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe703720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff874098540 tx=0x7ff87409a0d0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- 172.21.15.59:0/2966419554 shutdown_connections 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6ebd70 0x55bdbe703c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e14b0 0x55bdbe703720 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 --2- 172.21.15.59:0/2966419554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6e2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- 172.21.15.59:0/2966419554 >> 172.21.15.59:0/2966419554 conn(0x55bdbe6dd120 msgr2=0x55bdbe6e25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- 172.21.15.59:0/2966419554 shutdown_connections 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- 172.21.15.59:0/2966419554 wait complete. 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 -- start start 2024-09-05T12:06:44.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.859+0000 7ff8940a5740 1 Processor -- start 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1afdd0 con 0x55bdbe6e14b0 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b1240 con 0x55bdbe6e0b90 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b04a0 con 0x55bdbe6ebd70 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff8940a5740 10 client.0 _ll_register_callbacks cb 0x7ff87409b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff883e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff88b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff88aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:55050/0 (socket says 172.21.15.59:55050) 2024-09-05T12:06:44.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.860+0000 7ff88aa00640 1 -- 172.21.15.59:0/411710032 learned_addr learned my addr 172.21.15.59:0/411710032 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 -- 172.21.15.59:0/411710032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 msgr2=0x55bdbe6efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 -- 172.21.15.59:0/411710032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 msgr2=0x55bdbe6ef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6ef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 -- 172.21.15.59:0/411710032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbe6f00e0 con 0x55bdbe6e0b90 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff88aa00640 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff87c031550 tx=0x7ff87c004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36001d0eaae7ce07 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff86fe00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c035040 con 0x55bdbe6e0b90 2024-09-05T12:06:44.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff874097910 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.861+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff87c005450 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff87c057050 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 10 client.26813 fetch_fsmap learned FSMap version 21 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bdbe766510 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874097910 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff87c031880 con 0x55bdbe6e0b90 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 10 client.26813 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 20 client.26813 populate_metadata read hostname 'smithi059' 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe766c40 0x55bdbe769060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 4 client.26813 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdbe64ee60 con 0x55bdbe766c40 2024-09-05T12:06:44.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.862+0000 7ff883e00640 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe766c40 0x55bdbe769060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff883e00640 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe766c40 0x55bdbe769060 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35eb6cdc0ec2c633 server_cookie=b54f952422554fa in_seq=0 out_seq=0 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff86fe00640 10 client.26813 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff86fe00640 1 -- 172.21.15.59:0/411710032 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bdbe64ee60 con 0x55bdbe766c40 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff86fe00640 10 client.26813 handle_command_reply: tid=0 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 client.26813 shutdown 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 msgr2=0x55bdbe6f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff87c031550 tx=0x7ff87c004340 comp rx=0 tx=0).stop 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 shutdown_connections 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdbe6ebd70 0x55bdbe6efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bdbe766c40 0x55bdbe769060 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bdbe6e14b0 0x55bdbe6ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 --2- 172.21.15.59:0/411710032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bdbe6e0b90 0x55bdbe6f42b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 >> 172.21.15.59:0/411710032 conn(0x55bdbe6dd120 msgr2=0x55bdbe6ead30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.863+0000 7ff86fe00640 0 client.26813 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 shutdown_connections 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff8940a5740 1 -- 172.21.15.59:0/411710032 wait complete. 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff8940a5740 20 client.26813 trim_cache size 0 max 16384 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "5d3e92f7-e796-4876-b365-1087046e5832": { 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:06:44.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "5d3e92f7-e796-4876-b365-1087046e5832", 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 msgr2=0x7ff8700a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 0x7ff8700a4fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff884009d60 tx=0x7ff884005bd0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 msgr2=0x7ff88c19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.864+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff87400e9f0 tx=0x7ff87400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 shutdown_connections 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8700a2b10 0x7ff8700a4fd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff88c106f20 0x7ff88c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c1027a0 0x7ff88c19ab10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 --2- 172.21.15.59:0/4080798605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff88c101df0 0x7ff88c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 >> 172.21.15.59:0/4080798605 conn(0x7ff88c0fd8b0 msgr2=0x7ff88c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 shutdown_connections 2024-09-05T12:06:44.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:06:44.865+0000 7ff891600640 1 -- 172.21.15.59:0/4080798605 wait complete. 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: pgmap v666: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:44.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:44 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:06:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: pgmap v666: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:06:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:06:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:45.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:44 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:06:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: pgmap v666: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:06:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:06:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:45.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:06:45.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:06:45.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:44 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:06:45.932 INFO:tasks.cephfs.filesystem:scrub status for tag:5d3e92f7-e796-4876-b365-1087046e5832 - {'path': '/', 'tag': '5d3e92f7-e796-4876-b365-1087046e5832', 'options': 'recursive,force'} 2024-09-05T12:06:46.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:45 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:06:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:45 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:06:46.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:45 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:06:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:46 smithi189 ceph-mon[34772]: pgmap v667: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:46 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:46 smithi089 ceph-mon[34499]: pgmap v667: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:46 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:46 smithi059 ceph-mon[30025]: pgmap v667: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:06:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:46 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: pgmap v668: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:48.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:06:48.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:06:48.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: pgmap v668: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:06:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: pgmap v668: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:06:48.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:06:50.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:50 smithi189 ceph-mon[34772]: pgmap v669: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:50.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:50 smithi089 ceph-mon[34499]: pgmap v669: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:50.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:50 smithi059 ceph-mon[30025]: pgmap v669: 129 pgs: 129 active+clean; 4.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:06:52.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:52 smithi189 ceph-mon[34772]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:52.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:52 smithi089 ceph-mon[34499]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:52.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:52 smithi059 ceph-mon[30025]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:06:54.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:54 smithi189 ceph-mon[34772]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:54.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:54 smithi089 ceph-mon[34499]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:54.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:54 smithi059 ceph-mon[30025]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:06:56.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:56 smithi189 ceph-mon[34772]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:06:56.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:56 smithi089 ceph-mon[34499]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:06:56.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:56 smithi059 ceph-mon[30025]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:06:58.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:06:58 smithi189 ceph-mon[34772]: pgmap v673: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:06:58.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:06:58 smithi089 ceph-mon[34499]: pgmap v673: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:06:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:06:58 smithi059 ceph-mon[30025]: pgmap v673: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-05T12:07:00.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:00 smithi189 ceph-mon[34772]: pgmap v674: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:00 smithi089 ceph-mon[34499]: pgmap v674: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:00 smithi059 ceph-mon[30025]: pgmap v674: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:02.760 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:02 smithi089 ceph-mon[34499]: pgmap v675: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:02.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:02 smithi189 ceph-mon[34772]: pgmap v675: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:02 smithi059 ceph-mon[30025]: pgmap v675: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:04.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:04 smithi189 ceph-mon[34772]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-05T12:07:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:04 smithi089 ceph-mon[34499]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-05T12:07:04.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:04 smithi059 ceph-mon[30025]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-09-05T12:07:06.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:06 smithi189 ceph-mon[34772]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:06 smithi089 ceph-mon[34499]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:06.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:06 smithi059 ceph-mon[30025]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:08.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:08 smithi189 ceph-mon[34772]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:08 smithi089 ceph-mon[34499]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:08 smithi059 ceph-mon[30025]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:10.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:10 smithi189 ceph-mon[34772]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:10 smithi089 ceph-mon[34499]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:10.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:10 smithi059 ceph-mon[30025]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:12.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:12 smithi189 ceph-mon[34772]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:12.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:12 smithi089 ceph-mon[34499]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:12.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:12 smithi059 ceph-mon[30025]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:14.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:14 smithi189 ceph-mon[34772]: pgmap v681: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:14.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:14 smithi089 ceph-mon[34499]: pgmap v681: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:14.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:14 smithi059 ceph-mon[30025]: pgmap v681: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:15.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:07:16.324 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:16.715 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:16 smithi059 ceph-mon[30025]: pgmap v682: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:16.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:16 smithi189 ceph-mon[34772]: pgmap v682: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:16.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:16 smithi089 ceph-mon[34499]: pgmap v682: 129 pgs: 129 active+clean; 4.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 -- 172.21.15.59:0/3224932615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59241049c0 msgr2=0x7f5924108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 --2- 172.21.15.59:0/3224932615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59241049c0 0x7f5924108a10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5910009900 tx=0x7f591002a410 comp rx=0 tx=0).stop 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 -- 172.21.15.59:0/3224932615 shutdown_connections 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 --2- 172.21.15.59:0/3224932615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924109140 0x7f5924110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 --2- 172.21.15.59:0/3224932615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59241049c0 0x7f5924108a10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 --2- 172.21.15.59:0/3224932615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5924104010 0x7f59241043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 -- 172.21.15.59:0/3224932615 >> 172.21.15.59:0/3224932615 conn(0x7f59240ffa50 msgr2=0x7f5924101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 -- 172.21.15.59:0/3224932615 shutdown_connections 2024-09-05T12:07:17.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.069+0000 7f5929800640 1 -- 172.21.15.59:0/3224932615 wait complete. 2024-09-05T12:07:17.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.070+0000 7f5929800640 1 Processor -- start 2024-09-05T12:07:17.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.070+0000 7f5929800640 1 -- start start 2024-09-05T12:07:17.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59241049c0 0x7f59241a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5924109140 0x7f592419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59241141b0 con 0x7f5924109140 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5924114030 con 0x7f59241049c0 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5929800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5924114330 con 0x7f5924104010 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5922a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5922a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:60298/0 (socket says 172.21.15.59:60298) 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5922a00640 1 -- 172.21.15.59:0/2320815893 learned_addr learned my addr 172.21.15.59:0/2320815893 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5923400640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5924109140 0x7f592419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.071+0000 7f5922000640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59241049c0 0x7f59241a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 -- 172.21.15.59:0/2320815893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59241049c0 msgr2=0x7f59241a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59241049c0 0x7f59241a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 -- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5924109140 msgr2=0x7f592419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5924109140 0x7f592419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 -- 172.21.15.59:0/2320815893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592419acf0 con 0x7f5924104010 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5922a00640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9f0 tx=0x7f591400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f591b400640 1 -- 172.21.15.59:0/2320815893 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400cd80 con 0x7f5924104010 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f591b400640 1 -- 172.21.15.59:0/2320815893 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5914010910 con 0x7f5924104010 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592419afe0 con 0x7f5924104010 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.072+0000 7f591b400640 1 -- 172.21.15.59:0/2320815893 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914018990 con 0x7f5924104010 2024-09-05T12:07:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.073+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241a72d0 con 0x7f5924104010 2024-09-05T12:07:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.076+0000 7f591b400640 1 -- 172.21.15.59:0/2320815893 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5914010ac0 con 0x7f5924104010 2024-09-05T12:07:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f592c1fd740 1 Processor -- start 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f592c1fd740 1 -- start start 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f591b400640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 0x7f59080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.077+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c83160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f592c1fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8bdc3e0 con 0x55b7a8c764b0 2024-09-05T12:07:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f592c1fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8b83d80 con 0x55b7a8c80d70 2024-09-05T12:07:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f592c1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8bd83a0 con 0x55b7a8c75b90 2024-09-05T12:07:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f591b400640 1 -- 172.21.15.59:0/2320815893 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f59140cf7c0 con 0x7f5924104010 2024-09-05T12:07:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5923400640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 0x7f59080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5922000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5922000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33242/0 (socket says 172.21.15.59:33242) 2024-09-05T12:07:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5922000640 1 -- 172.21.15.59:0/3566787458 learned_addr learned my addr 172.21.15.59:0/3566787458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5923400640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c83160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.078+0000 7f5922a00640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 -- 172.21.15.59:0/3566787458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 msgr2=0x55b7a8c75f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c75f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 -- 172.21.15.59:0/3566787458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 msgr2=0x55b7a8c83160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c83160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 -- 172.21.15.59:0/3566787458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a8ba2ae0 con 0x55b7a8c764b0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922a00640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c75f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5923400640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 0x7f59080a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f591c009fd0 tx=0x7f591c009d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f5922000640 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5910009900 tx=0x7f5910005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f8b8ce7679cd9a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.079+0000 7f590fe00640 1 -- 172.21.15.59:0/3566787458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910035040 con 0x55b7a8c764b0 2024-09-05T12:07:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f590fe00640 1 -- 172.21.15.59:0/3566787458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910004400 con 0x55b7a8c764b0 2024-09-05T12:07:17.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f590fe00640 1 -- 172.21.15.59:0/3566787458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100035f0 con 0x55b7a8c764b0 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 -- 172.21.15.59:0/3566787458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 msgr2=0x55b7a8c80830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5910009900 tx=0x7f5910005de0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 -- 172.21.15.59:0/3566787458 shutdown_connections 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c83160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c80830 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3566787458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c75f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.080+0000 7f592c1fd740 1 -- 172.21.15.59:0/3566787458 >> 172.21.15.59:0/3566787458 conn(0x55b7a8c72120 msgr2=0x55b7a8c72500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.081+0000 7f592c1fd740 1 -- 172.21.15.59:0/3566787458 shutdown_connections 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.081+0000 7f592c1fd740 1 -- 172.21.15.59:0/3566787458 wait complete. 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.081+0000 7f592c1fd740 1 -- start start 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.081+0000 7f592c1fd740 1 Processor -- start 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8c83980 con 0x55b7a8c75b90 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8b970f0 con 0x55b7a8c80d70 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7a8c83b00 con 0x55b7a8c764b0 2024-09-05T12:07:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f592c1fd740 10 client.0 _ll_register_callbacks cb 0x55b7a8b889e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f5922a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f5923400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f5923400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36760/0 (socket says 172.21.15.59:36760) 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f5923400640 1 -- 172.21.15.59:0/3662883172 learned_addr learned my addr 172.21.15.59:0/3662883172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.082+0000 7f5922000640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 -- 172.21.15.59:0/3662883172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 msgr2=0x55b7a8c7e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 -- 172.21.15.59:0/3662883172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 msgr2=0x55b7a8c7dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 -- 172.21.15.59:0/3662883172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a894d3c0 con 0x55b7a8c80d70 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5922a00640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f5923400640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f591c00fef0 tx=0x7f591c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eea5b14dc5a746f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.083+0000 7f590cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c013330 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5910004400 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f591c01c040 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f591c003970 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f592c1fd740 10 client.26825 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b7a8c8f650 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.084+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910004400 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f591c020050 con 0x55b7a8c80d70 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 10 client.26825 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 20 client.26825 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:17.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8c90a60 0x55b7a8c92f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 4 client.26825 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:17.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a8be9950 con 0x55b7a8c90a60 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.085+0000 7f5922a00640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8c90a60 0x55b7a8c92f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.086+0000 7f5922a00640 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8c90a60 0x55b7a8c92f00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379daa2541a51bb0 server_cookie=6834a09c0d67f43a in_seq=0 out_seq=0 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.086+0000 7f590cc00640 10 client.26825 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.086+0000 7f590cc00640 1 -- 172.21.15.59:0/3662883172 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7a8be9950 con 0x55b7a8c90a60 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.086+0000 7f590cc00640 10 client.26825 handle_command_reply: tid=0 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.086+0000 7f592c1fd740 1 client.26825 shutdown 2024-09-05T12:07:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 msgr2=0x55b7a8c7e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f591c00fef0 tx=0x7f591c01d000 comp rx=0 tx=0).stop 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 shutdown_connections 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7e6b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8c90a60 0x55b7a8c92f00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3662883172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f590cc00640 0 client.26825 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 >> 172.21.15.59:0/3662883172 conn(0x55b7a8c72120 msgr2=0x55b7a8c77e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 shutdown_connections 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 1 -- 172.21.15.59:0/3662883172 wait complete. 2024-09-05T12:07:17.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.087+0000 7f592c1fd740 20 client.26825 trim_cache size 0 max 16384 2024-09-05T12:07:17.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.105+0000 7f592c1fd740 1 Processor -- start 2024-09-05T12:07:17.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 -- start start 2024-09-05T12:07:17.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 0x55b7a8c98d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59241af140 con 0x55b7a8c80d70 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59241b19c0 con 0x55b7a8c75b90 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f592c1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59241af7b0 con 0x55b7a8c764b0 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5923400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 0x55b7a8c98d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5922a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5923400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 0x55b7a8c98d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33260/0 (socket says 172.21.15.59:33260) 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5923400640 1 -- 172.21.15.59:0/112931988 learned_addr learned my addr 172.21.15.59:0/112931988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5922a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36762/0 (socket says 172.21.15.59:36762) 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.106+0000 7f5922000640 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 -- 172.21.15.59:0/112931988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 msgr2=0x55b7a8c987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 -- 172.21.15.59:0/112931988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 msgr2=0x55b7a8c98d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 0x55b7a8c98d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 -- 172.21.15.59:0/112931988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a8c93580 con 0x55b7a8c75b90 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f5922a00640 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f59140ccd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829c1f6aa536e5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.107+0000 7f590f400640 1 -- 172.21.15.59:0/112931988 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b8020 con 0x55b7a8c75b90 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.108+0000 7f590f400640 1 -- 172.21.15.59:0/112931988 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59140b4050 con 0x55b7a8c75b90 2024-09-05T12:07:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.108+0000 7f590f400640 1 -- 172.21.15.59:0/112931988 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b2070 con 0x55b7a8c75b90 2024-09-05T12:07:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- 172.21.15.59:0/112931988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 msgr2=0x55b7a8c785f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9c0 tx=0x7f59140ccd20 comp rx=0 tx=0).stop 2024-09-05T12:07:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- 172.21.15.59:0/112931988 shutdown_connections 2024-09-05T12:07:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c80d70 0x55b7a8c98d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c764b0 0x55b7a8c987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 --2- 172.21.15.59:0/112931988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c75b90 0x55b7a8c785f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- 172.21.15.59:0/112931988 >> 172.21.15.59:0/112931988 conn(0x55b7a8c72120 msgr2=0x55b7a8c7fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- 172.21.15.59:0/112931988 shutdown_connections 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- 172.21.15.59:0/112931988 wait complete. 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.109+0000 7f592c1fd740 1 -- start start 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 Processor -- start 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59241afe80 con 0x55b7a8c764b0 2024-09-05T12:07:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59241b12f0 con 0x55b7a8c80d70 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59241b0550 con 0x55b7a8c75b90 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f592c1fd740 10 client.0 _ll_register_callbacks cb 0x7f591c00fb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.110+0000 7f5922000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5922000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33274/0 (socket says 172.21.15.59:33274) 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5922000640 1 -- 172.21.15.59:0/3056428232 learned_addr learned my addr 172.21.15.59:0/3056428232 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5922a00640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 -- 172.21.15.59:0/3056428232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 msgr2=0x55b7a8c7abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 -- 172.21.15.59:0/3056428232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 msgr2=0x55b7a8c7b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 -- 172.21.15.59:0/3056428232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a8cfa7a0 con 0x55b7a8c80d70 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5922000640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f5923400640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7b640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b7a8c6ce90 tx=0x7f591c013680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6f04bd5b882e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.111+0000 7f590cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c009100 con 0x55b7a8c80d70 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59140b4050 con 0x55b7a8c80d70 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f591c02c560 con 0x55b7a8c80d70 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f591c02c760 con 0x55b7a8c80d70 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f592c1fd740 10 client.26831 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b7a8cfb9f0 con 0x55b7a8c80d70 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.112+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59140b4050 con 0x55b7a8c80d70 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f591c005030 con 0x55b7a8c80d70 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 10 client.26831 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 10 client.26831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 20 client.26831 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8cfc1c0 0x55b7a8cfe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 4 client.26831 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7a8be3e60 con 0x55b7a8cfc1c0 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.113+0000 7f5922a00640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8cfc1c0 0x55b7a8cfe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f5922a00640 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8cfc1c0 0x55b7a8cfe5e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde95f5891438ed8 server_cookie=9124c6efc6771c in_seq=0 out_seq=0 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f590cc00640 10 client.26831 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f590cc00640 1 -- 172.21.15.59:0/3056428232 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7a8be3e60 con 0x55b7a8cfc1c0 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f590cc00640 10 client.26831 handle_command_reply: tid=0 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 client.26831 shutdown 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 msgr2=0x55b7a8c7b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7b640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b7a8c6ce90 tx=0x7f591c013680 comp rx=0 tx=0).stop 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 shutdown_connections 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7a8c80d70 0x55b7a8c7b640 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b7a8cfc1c0 0x55b7a8cfe5e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b7a8c764b0 0x55b7a8c7b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 --2- 172.21.15.59:0/3056428232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a8c75b90 0x55b7a8c7abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 >> 172.21.15.59:0/3056428232 conn(0x55b7a8c72120 msgr2=0x55b7a8c7f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f590cc00640 0 client.26831 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.114+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 shutdown_connections 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f592c1fd740 1 -- 172.21.15.59:0/3056428232 wait complete. 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f592c1fd740 20 client.26831 trim_cache size 0 max 16384 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:17.120 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:07:17.121 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:07:17.121 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 msgr2=0x7f59080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 0x7f59080a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f591c009fd0 tx=0x7f591c009d80 comp rx=0 tx=0).stop 2024-09-05T12:07:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 msgr2=0x7f59241a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f591400e9f0 tx=0x7f591400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 shutdown_connections 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f59080a2b80 0x7f59080a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5924109140 0x7f592419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59241049c0 0x7f59241a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 --2- 172.21.15.59:0/2320815893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5924104010 0x7f59241a0400 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.115+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 >> 172.21.15.59:0/2320815893 conn(0x7f59240ffa50 msgr2=0x7f5924100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.116+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 shutdown_connections 2024-09-05T12:07:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:17.116+0000 7f5929800640 1 -- 172.21.15.59:0/2320815893 wait complete. 2024-09-05T12:07:18.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:07:18.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:07:18.610 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:18.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:18 smithi189 ceph-mon[34772]: pgmap v683: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:18.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:18 smithi059 ceph-mon[30025]: pgmap v683: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:18.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:18 smithi089 ceph-mon[34499]: pgmap v683: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.169+0000 7f96c5c00640 1 -- 172.21.15.59:0/1356149724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c01047e0 msgr2=0x7f96c0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.169+0000 7f96c5c00640 1 --2- 172.21.15.59:0/1356149724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c01047e0 0x7f96c0108830 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009900 tx=0x7f96a802a400 comp rx=0 tx=0).stop 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 -- 172.21.15.59:0/1356149724 shutdown_connections 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 --2- 172.21.15.59:0/1356149724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0108f60 0x7f96c0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 --2- 172.21.15.59:0/1356149724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c01047e0 0x7f96c0108830 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 --2- 172.21.15.59:0/1356149724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c0103e30 0x7f96c0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 -- 172.21.15.59:0/1356149724 >> 172.21.15.59:0/1356149724 conn(0x7f96c00ff8d0 msgr2=0x7f96c0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 -- 172.21.15.59:0/1356149724 shutdown_connections 2024-09-05T12:07:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.170+0000 7f96c5c00640 1 -- 172.21.15.59:0/1356149724 wait complete. 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 Processor -- start 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 -- start start 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 0x7f96c019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c01047e0 0x7f96c019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 0x7f96c019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0114010 con 0x7f96c0108f60 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113e90 con 0x7f96c0103e30 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0114190 con 0x7f96c01047e0 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.171+0000 7f96c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 0x7f96c019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 0x7f96c019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 0x7f96c019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33296/0 (socket says 172.21.15.59:33296) 2024-09-05T12:07:19.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96c5200640 1 -- 172.21.15.59:0/3369940598 learned_addr learned my addr 172.21.15.59:0/3369940598 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96b7e00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c01047e0 0x7f96c019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 -- 172.21.15.59:0/3369940598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c01047e0 msgr2=0x7f96c019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c01047e0 0x7f96c019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 -- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 msgr2=0x7f96c019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 0x7f96c019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 -- 172.21.15.59:0/3369940598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c019f440 con 0x7f96c0103e30 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.172+0000 7f96bfe00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 0x7f96c019a490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96b000d720 tx=0x7f96b000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:19.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.173+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0017070 con 0x7f96c0103e30 2024-09-05T12:07:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.173+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c019f730 con 0x7f96c0103e30 2024-09-05T12:07:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.173+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b000de30 con 0x7f96c0103e30 2024-09-05T12:07:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.173+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00047a0 con 0x7f96c0103e30 2024-09-05T12:07:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.173+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a7010 con 0x7f96c0103e30 2024-09-05T12:07:19.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.174+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c0104210 con 0x7f96c0103e30 2024-09-05T12:07:19.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.176+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96b00099d0 con 0x7f96c0103e30 2024-09-05T12:07:19.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.177+0000 7f96bd600640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 0x7f96a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:19.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.177+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f96b00cf4d0 con 0x7f96c0103e30 2024-09-05T12:07:19.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.177+0000 7f96b7e00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 0x7f96a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:19.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.178+0000 7f96b7e00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 0x7f96a40a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000c00 tx=0x7f96a8005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:19.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.179+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96b0020020 con 0x7f96c0103e30 2024-09-05T12:07:19.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.403+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96c010c140 con 0x7f96c0103e30 2024-09-05T12:07:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.404+0000 7f96bd600640 1 -- 172.21.15.59:0/3369940598 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f96b009bfc0 con 0x7f96c0103e30 2024-09-05T12:07:19.406 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:07:19.406 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:07:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.406+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 msgr2=0x7f96a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.406+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 0x7f96a40a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000c00 tx=0x7f96a8005b50 comp rx=0 tx=0).stop 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.406+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 msgr2=0x7f96c019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.406+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 0x7f96c019a490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96b000d720 tx=0x7f96b000dbe0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 shutdown_connections 2024-09-05T12:07:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f96a40a2b80 0x7f96a40a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96c0108f60 0x7f96c019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c01047e0 0x7f96c019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 --2- 172.21.15.59:0/3369940598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c0103e30 0x7f96c019a490 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 >> 172.21.15.59:0/3369940598 conn(0x7f96c00ff8d0 msgr2=0x7f96c0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 shutdown_connections 2024-09-05T12:07:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:19.407+0000 7f96c5c00640 1 -- 172.21.15.59:0/3369940598 wait complete. 2024-09-05T12:07:19.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:19 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3369940598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:19.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:19 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3369940598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:19.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:19 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3369940598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:20.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:07:20.538 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:20.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:20 smithi059 ceph-mon[30025]: pgmap v684: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:07:20.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:20 smithi189 ceph-mon[34772]: pgmap v684: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:07:20.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:20 smithi089 ceph-mon[34499]: pgmap v684: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:07:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 -- 172.21.15.59:0/2432236509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69301079c0 msgr2=0x7f6930107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 --2- 172.21.15.59:0/2432236509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69301079c0 0x7f6930107da0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6924009930 tx=0x7f692402a420 comp rx=0 tx=0).stop 2024-09-05T12:07:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 -- 172.21.15.59:0/2432236509 shutdown_connections 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 --2- 172.21.15.59:0/2432236509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6930102360 0x7f693010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 --2- 172.21.15.59:0/2432236509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 --2- 172.21.15.59:0/2432236509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69301079c0 0x7f6930107da0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 -- 172.21.15.59:0/2432236509 >> 172.21.15.59:0/2432236509 conn(0x7f69300fd740 msgr2=0x7f69300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 -- 172.21.15.59:0/2432236509 shutdown_connections 2024-09-05T12:07:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.117+0000 7f6936000640 1 -- 172.21.15.59:0/2432236509 wait complete. 2024-09-05T12:07:21.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.118+0000 7f6936000640 1 Processor -- start 2024-09-05T12:07:21.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 -- start start 2024-09-05T12:07:21.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 0x7f69301987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69301079c0 0x7f693019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6930111e10 con 0x7f6930102360 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6930111c90 con 0x7f69301019c0 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6936000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6930111f90 con 0x7f69301079c0 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 0x7f69301987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 0x7f69301987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38990/0 (socket says 172.21.15.59:38990) 2024-09-05T12:07:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40364/0 (socket says 172.21.15.59:40364) 2024-09-05T12:07:21.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.119+0000 7f692fe00640 1 -- 172.21.15.59:0/374107405 learned_addr learned my addr 172.21.15.59:0/374107405 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6935600640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69301079c0 0x7f693019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 -- 172.21.15.59:0/374107405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69301079c0 msgr2=0x7f693019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69301079c0 0x7f693019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 -- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 msgr2=0x7f69301987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 0x7f69301987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 -- 172.21.15.59:0/374107405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693019d240 con 0x7f69301019c0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f6934c00640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f692402a940 tx=0x7f6924003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.120+0000 7f692d600640 1 -- 172.21.15.59:0/374107405 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692403f070 con 0x7f69301019c0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.121+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693019d4d0 con 0x7f69301019c0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.121+0000 7f692d600640 1 -- 172.21.15.59:0/374107405 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6924031c90 con 0x7f69301019c0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.121+0000 7f692d600640 1 -- 172.21.15.59:0/374107405 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924035030 con 0x7f69301019c0 2024-09-05T12:07:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.121+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69301a4e10 con 0x7f69301019c0 2024-09-05T12:07:21.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 Processor -- start 2024-09-05T12:07:21.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 -- start start 2024-09-05T12:07:21.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f692d600640 1 -- 172.21.15.59:0/374107405 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6924042070 con 0x7f69301019c0 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b169de33e0 con 0x55b169e88020 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b169d8ad80 con 0x55b169e7d760 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.124+0000 7f6939fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b169ddf3a0 con 0x55b169e7ce40 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40378/0 (socket says 172.21.15.59:40378) 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 -- 172.21.15.59:0/860704048 learned_addr learned my addr 172.21.15.59:0/860704048 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38994/0 (socket says 172.21.15.59:38994) 2024-09-05T12:07:21.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6935600640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 -- 172.21.15.59:0/860704048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 msgr2=0x55b169e7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e7d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 -- 172.21.15.59:0/860704048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 msgr2=0x55b169e8a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 -- 172.21.15.59:0/860704048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b169da9ae0 con 0x55b169e7d760 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6935600640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f692fe00640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.125+0000 7f6934c00640 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69240f3eb0 tx=0x7f69240f6400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fea42253991c8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f692d600640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 0x7f69180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f692d600640 1 -- 172.21.15.59:0/374107405 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69240efe00 con 0x7f69301019c0 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f6921c00640 1 -- 172.21.15.59:0/860704048 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924105070 con 0x55b169e7d760 2024-09-05T12:07:21.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f6921c00640 1 -- 172.21.15.59:0/860704048 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69240bc8f0 con 0x55b169e7d760 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f6921c00640 1 -- 172.21.15.59:0/860704048 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240bbfc0 con 0x55b169e7d760 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.126+0000 7f6935600640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 0x7f69180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 -- 172.21.15.59:0/860704048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 msgr2=0x55b169e87ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69240f3eb0 tx=0x7f69240f6400 comp rx=0 tx=0).stop 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 -- 172.21.15.59:0/860704048 shutdown_connections 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169e8a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7d760 0x55b169e87ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 --2- 172.21.15.59:0/860704048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 -- 172.21.15.59:0/860704048 >> 172.21.15.59:0/860704048 conn(0x55b169e793d0 msgr2=0x55b169e797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 -- 172.21.15.59:0/860704048 shutdown_connections 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6935600640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 0x7f69180a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6928005fd0 tx=0x7f6928009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.127+0000 7f6939fce740 1 -- 172.21.15.59:0/860704048 wait complete. 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 -- start start 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 Processor -- start 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e84f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e85480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 0x55b169e859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b169e8a9b0 con 0x55b169e7d760 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b169d9e0f0 con 0x55b169e88020 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b169e8ab30 con 0x55b169e7ce40 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.128+0000 7f6939fce740 10 client.0 _ll_register_callbacks cb 0x55b169d85bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6935600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 0x55b169e859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e84f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6935600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 0x55b169e859c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40390/0 (socket says 172.21.15.59:40390) 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6935600640 1 -- 172.21.15.59:0/3609362001 learned_addr learned my addr 172.21.15.59:0/3609362001 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f692fe00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e85480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 -- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 msgr2=0x55b169e859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 0x55b169e859c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 -- 172.21.15.59:0/3609362001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 msgr2=0x55b169e85480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e85480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 -- 172.21.15.59:0/3609362001 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b169b543c0 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6934c00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e84f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69240f3eb0 tx=0x7f6924005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2c78e92aba7e89a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.129+0000 7f6916000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924049070 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69240ed5a0 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69240bc8f0 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69240bbb50 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6939fce740 10 client.26782 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.130+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b169e8a9b0 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69240bc8f0 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f69240b8860 con 0x55b169e7ce40 2024-09-05T12:07:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 10 client.26782 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 20 client.26782 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169e9bf20 0x55b169e9e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 4 client.26782 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b169df0950 con 0x55b169e9bf20 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.131+0000 7f692fe00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169e9bf20 0x55b169e9e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.132+0000 7f692fe00640 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169e9bf20 0x55b169e9e310 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f605bdad0dbc2980 server_cookie=6a6c8c87cf4bab13 in_seq=0 out_seq=0 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.132+0000 7f6916000640 10 client.26782 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6916000640 1 -- 172.21.15.59:0/3609362001 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b169df0950 con 0x55b169e9bf20 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6916000640 10 client.26782 handle_command_reply: tid=0 2024-09-05T12:07:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 client.26782 shutdown 2024-09-05T12:07:21.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 msgr2=0x55b169e84f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e84f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69240f3eb0 tx=0x7f6924005d50 comp rx=0 tx=0).stop 2024-09-05T12:07:21.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 shutdown_connections 2024-09-05T12:07:21.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e88020 0x55b169e859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169e9bf20 0x55b169e9e310 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e85480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 --2- 172.21.15.59:0/3609362001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7ce40 0x55b169e84f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6916000640 0 client.26782 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 >> 172.21.15.59:0/3609362001 conn(0x55b169e793d0 msgr2=0x55b169e7f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.133+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 shutdown_connections 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.134+0000 7f6939fce740 1 -- 172.21.15.59:0/3609362001 wait complete. 2024-09-05T12:07:21.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.134+0000 7f6939fce740 20 client.26782 trim_cache size 0 max 16384 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.147+0000 7f6939fce740 1 Processor -- start 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.147+0000 7f6939fce740 1 -- start start 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7d760 0x55b169ea3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169ea3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69301aab60 con 0x55b169e88020 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69301ad3e0 con 0x55b169e7ce40 2024-09-05T12:07:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6939fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69301ab1d0 con 0x55b169e7d760 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6935600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169ea3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40402/0 (socket says 172.21.15.59:40402) 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 -- 172.21.15.59:0/2518792586 learned_addr learned my addr 172.21.15.59:0/2518792586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f692fe00640 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7d760 0x55b169ea3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 -- 172.21.15.59:0/2518792586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7d760 msgr2=0x55b169ea3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7d760 0x55b169ea3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 -- 172.21.15.59:0/2518792586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 msgr2=0x55b169ea3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169ea3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.148+0000 7f6934c00640 1 -- 172.21.15.59:0/2518792586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b169e7cbf0 con 0x55b169e7ce40 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6934c00640 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69240ba4a0 tx=0x7f69240ba700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdeeb1508bbf8b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6921200640 1 -- 172.21.15.59:0/2518792586 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924049070 con 0x55b169e7ce40 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6921200640 1 -- 172.21.15.59:0/2518792586 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69240f3370 con 0x55b169e7ce40 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6921200640 1 -- 172.21.15.59:0/2518792586 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240bbd30 con 0x55b169e7ce40 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 -- 172.21.15.59:0/2518792586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 msgr2=0x55b169e7f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69240ba4a0 tx=0x7f69240ba700 comp rx=0 tx=0).stop 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 -- 172.21.15.59:0/2518792586 shutdown_connections 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e88020 0x55b169ea3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e7d760 0x55b169ea3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.149+0000 7f6939fce740 1 --2- 172.21.15.59:0/2518792586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e7f8a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.150+0000 7f6939fce740 1 -- 172.21.15.59:0/2518792586 >> 172.21.15.59:0/2518792586 conn(0x55b169e793d0 msgr2=0x55b169e84030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.150+0000 7f6939fce740 1 -- 172.21.15.59:0/2518792586 shutdown_connections 2024-09-05T12:07:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.150+0000 7f6939fce740 1 -- 172.21.15.59:0/2518792586 wait complete. 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.150+0000 7f6939fce740 1 -- start start 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.150+0000 7f6939fce740 1 Processor -- start 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e8f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e88020 0x55b169e8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69301ab8a0 con 0x55b169e7d760 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69301acd10 con 0x55b169e7ce40 2024-09-05T12:07:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69301abf70 con 0x55b169e88020 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6939fce740 10 client.0 _ll_register_callbacks cb 0x7f6924103070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e8f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40410/0 (socket says 172.21.15.59:40410) 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 -- 172.21.15.59:0/3145139380 learned_addr learned my addr 172.21.15.59:0/3145139380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6935600640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e88020 0x55b169e8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 -- 172.21.15.59:0/3145139380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e88020 msgr2=0x55b169e8fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e88020 0x55b169e8fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 -- 172.21.15.59:0/3145139380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 msgr2=0x55b169e8f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e8f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f6934c00640 1 -- 172.21.15.59:0/3145139380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b169e7e3c0 con 0x55b169e7ce40 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.151+0000 7f692fe00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e8f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6934c00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f69240bc180 tx=0x7f69240f3b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1f0495765093a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6916000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924049070 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69240dc0b0 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b169deae60 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69240da050 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6939fce740 10 client.26855 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b169e7e3c0 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.152+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b169deae60 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f69240bc450 con 0x55b169e7ce40 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 10 client.26855 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 20 client.26855 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169f06840 0x55b169f08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 4 client.26855 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b169d67010 con 0x55b169f06840 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.153+0000 7f692fe00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169f06840 0x55b169f08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f692fe00640 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169f06840 0x55b169f08c30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe33064f9721bbe server_cookie=eb7ff9420f8e96c7 in_seq=0 out_seq=0 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6916000640 10 client.26855 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6916000640 1 -- 172.21.15.59:0/3145139380 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b169d67010 con 0x55b169f06840 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6916000640 10 client.26855 handle_command_reply: tid=0 2024-09-05T12:07:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 client.26855 shutdown 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 msgr2=0x55b169e94660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f69240bc180 tx=0x7f69240f3b00 comp rx=0 tx=0).stop 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 shutdown_connections 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b169e88020 0x55b169e8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b169f06840 0x55b169f08c30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b169e7d760 0x55b169e8f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6916000640 0 client.26855 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 --2- 172.21.15.59:0/3145139380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b169e7ce40 0x55b169e94660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 >> 172.21.15.59:0/3145139380 conn(0x55b169e793d0 msgr2=0x55b169e84ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 shutdown_connections 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 1 -- 172.21.15.59:0/3145139380 wait complete. 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.154+0000 7f6939fce740 20 client.26855 trim_cache size 0 max 16384 2024-09-05T12:07:21.160 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 msgr2=0x7f69180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 0x7f69180a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6928005fd0 tx=0x7f6928009500 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 msgr2=0x7f6930198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f692402a940 tx=0x7f6924003b20 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 shutdown_connections 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69180a2b80 0x7f69180a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69301079c0 0x7f693019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6930102360 0x7f69301987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 --2- 172.21.15.59:0/374107405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69301019c0 0x7f6930198290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:21.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 >> 172.21.15.59:0/374107405 conn(0x7f69300fd740 msgr2=0x7f69300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:21.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 shutdown_connections 2024-09-05T12:07:21.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:21.155+0000 7f6936000640 1 -- 172.21.15.59:0/374107405 wait complete. 2024-09-05T12:07:22.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:22 smithi189 ceph-mon[34772]: pgmap v685: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:22.870 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:07:22.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:07:22.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:22 smithi059 ceph-mon[30025]: pgmap v685: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:22.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:22 smithi089 ceph-mon[34499]: pgmap v685: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:23.260 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:23.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 -- 172.21.15.59:0/427582581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44101de0 msgr2=0x7f1f441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 --2- 172.21.15.59:0/427582581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44101de0 0x7f1f441021c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009900 tx=0x7f1f3802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 -- 172.21.15.59:0/427582581 shutdown_connections 2024-09-05T12:07:23.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 --2- 172.21.15.59:0/427582581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44106f10 0x7f1f4410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 --2- 172.21.15.59:0/427582581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44102790 0x7f1f441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 --2- 172.21.15.59:0/427582581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44101de0 0x7f1f441021c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 -- 172.21.15.59:0/427582581 >> 172.21.15.59:0/427582581 conn(0x7f1f440fd8c0 msgr2=0x7f1f440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 -- 172.21.15.59:0/427582581 shutdown_connections 2024-09-05T12:07:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.875+0000 7f1f4c800640 1 -- 172.21.15.59:0/427582581 wait complete. 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.876+0000 7f1f4c800640 1 Processor -- start 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.876+0000 7f1f4c800640 1 -- start start 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.876+0000 7f1f4c800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 0x7f1f4419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 0x7f1f4419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 0x7f1f4419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44111f80 con 0x7f1f44102790 2024-09-05T12:07:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44111e00 con 0x7f1f44106f10 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4c800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44112100 con 0x7f1f44101de0 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 0x7f1f4419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 0x7f1f4419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 0x7f1f4419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38462/0 (socket says 172.21.15.59:38462) 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f49e00640 1 -- 172.21.15.59:0/698867786 learned_addr learned my addr 172.21.15.59:0/698867786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 0x7f1f4419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 -- 172.21.15.59:0/698867786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 msgr2=0x7f1f4419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 0x7f1f4419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 -- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 msgr2=0x7f1f4419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 0x7f1f4419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f4a800640 1 -- 172.21.15.59:0/698867786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4419f610 con 0x7f1f44106f10 2024-09-05T12:07:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.877+0000 7f1f49400640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 0x7f1f4419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f4a800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 0x7f1f4419ef30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000da10 tx=0x7f1f4000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f3ea00640 1 -- 172.21.15.59:0/698867786 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4000bd70 con 0x7f1f44106f10 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f3ea00640 1 -- 172.21.15.59:0/698867786 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f40003e50 con 0x7f1f44106f10 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4419f900 con 0x7f1f44106f10 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f3ea00640 1 -- 172.21.15.59:0/698867786 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f400106a0 con 0x7f1f44106f10 2024-09-05T12:07:23.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.878+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441a71e0 con 0x7f1f44106f10 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.881+0000 7f1f4f236740 1 Processor -- start 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.881+0000 7f1f3ea00640 1 -- 172.21.15.59:0/698867786 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f40003970 con 0x7f1f44106f10 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.881+0000 7f1f4f236740 1 -- start start 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 0x5583b2a97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583b29f3290 con 0x5583b2a8cac0 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5583b299ab40 con 0x5583b2a8d3e0 2024-09-05T12:07:23.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f4f236740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583b29ef250 con 0x5583b2a97ca0 2024-09-05T12:07:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39034/0 (socket says 172.21.15.59:39034) 2024-09-05T12:07:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f49400640 1 -- 172.21.15.59:0/2620754966 learned_addr learned my addr 172.21.15.59:0/2620754966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 0x5583b2a97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.882+0000 7f1f49400640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 -- 172.21.15.59:0/2620754966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 msgr2=0x5583b2a9a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a9a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 -- 172.21.15.59:0/2620754966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 msgr2=0x5583b2a8cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 -- 172.21.15.59:0/2620754966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b27646a0 con 0x5583b2a8d3e0 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49400640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 0x5583b2a97760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f38031550 tx=0x7f1f38005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca8363e7c677bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f3ea00640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 0x7f1f2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f33000640 1 -- 172.21.15.59:0/2620754966 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38040070 con 0x5583b2a8d3e0 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f33000640 1 -- 172.21.15.59:0/2620754966 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f38004480 con 0x5583b2a8d3e0 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f33000640 1 -- 172.21.15.59:0/2620754966 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38035040 con 0x5583b2a8d3e0 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.883+0000 7f1f3ea00640 1 -- 172.21.15.59:0/698867786 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1f400cf7f0 con 0x7f1f44106f10 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f49400640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 0x7f1f2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 -- 172.21.15.59:0/2620754966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 msgr2=0x5583b2a97760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 0x5583b2a97760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f38031550 tx=0x7f1f38005c00 comp rx=0 tx=0).stop 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 -- 172.21.15.59:0/2620754966 shutdown_connections 2024-09-05T12:07:23.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8d3e0 0x5583b2a97760 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2620754966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8cac0 0x5583b2a8cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 -- 172.21.15.59:0/2620754966 >> 172.21.15.59:0/2620754966 conn(0x5583b2a89050 msgr2=0x5583b2a89430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f49400640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 0x7f1f2c0a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5583b2a8a3d0 tx=0x7f1f34009c00 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.884+0000 7f1f4f236740 1 -- 172.21.15.59:0/2620754966 shutdown_connections 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.885+0000 7f1f4f236740 1 -- 172.21.15.59:0/2620754966 wait complete. 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.885+0000 7f1f4f236740 1 -- start start 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.885+0000 7f1f4f236740 1 Processor -- start 2024-09-05T12:07:23.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 0x5583b2a97690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 0x5583b2a94f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583b2a9a8b0 con 0x5583b2a8d3e0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5583b29adfa0 con 0x5583b2a97ca0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583b2a9aa30 con 0x5583b2a8cac0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4f236740 10 client.0 _ll_register_callbacks cb 0x5583b2701ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 0x5583b2a97690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39048/0 (socket says 172.21.15.59:39048) 2024-09-05T12:07:23.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 -- 172.21.15.59:0/2029439526 learned_addr learned my addr 172.21.15.59:0/2029439526 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 0x5583b2a97690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38476/0 (socket says 172.21.15.59:38476) 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f4a800640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 0x5583b2a94f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 -- 172.21.15.59:0/2029439526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 msgr2=0x5583b2a97690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 0x5583b2a97690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 -- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 msgr2=0x5583b2a94f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 0x5583b2a94f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.886+0000 7f1f49400640 1 -- 172.21.15.59:0/2029439526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b2a95760 con 0x5583b2a8d3e0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f4a800640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 0x5583b2a94f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f49400640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1f34010f90 tx=0x7f1f34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d68e5c4ede802d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f27400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34027070 con 0x5583b2a8d3e0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1f34005220 con 0x5583b2a8d3e0 2024-09-05T12:07:23.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f38004480 con 0x5583b2a8d3e0 2024-09-05T12:07:23.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1f34005420 con 0x5583b2a8d3e0 2024-09-05T12:07:23.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.887+0000 7f1f4f236740 10 client.16872 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:23.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5583b2aa68d0 con 0x5583b2a8d3e0 2024-09-05T12:07:23.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f38004480 con 0x5583b2a8d3e0 2024-09-05T12:07:23.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1f3402b6b0 con 0x5583b2a8d3e0 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 10 client.16872 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 20 client.16872 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2aa7de0 0x5583b2aaa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 4 client.16872 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.888+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583b2a00800 con 0x5583b2aa7de0 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.889+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2aa7de0 0x5583b2aaa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.889+0000 7f1f49e00640 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2aa7de0 0x5583b2aaa220 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b253019a0930ac server_cookie=cf04c185466aa5ee in_seq=0 out_seq=0 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.889+0000 7f1f27400640 10 client.16872 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f27400640 1 -- 172.21.15.59:0/2029439526 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583b2a00800 con 0x5583b2aa7de0 2024-09-05T12:07:23.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f27400640 10 client.16872 handle_command_reply: tid=0 2024-09-05T12:07:23.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 client.16872 shutdown 2024-09-05T12:07:23.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 msgr2=0x5583b2a94940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1f34010f90 tx=0x7f1f34004730 comp rx=0 tx=0).stop 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 shutdown_connections 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a97ca0 0x5583b2a94f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2aa7de0 0x5583b2aaa220 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2a94940 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 --2- 172.21.15.59:0/2029439526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8cac0 0x5583b2a97690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 >> 172.21.15.59:0/2029439526 conn(0x5583b2a89050 msgr2=0x5583b2a881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.890+0000 7f1f27400640 0 client.16872 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.891+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 shutdown_connections 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.891+0000 7f1f4f236740 1 -- 172.21.15.59:0/2029439526 wait complete. 2024-09-05T12:07:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.891+0000 7f1f4f236740 20 client.16872 trim_cache size 0 max 16384 2024-09-05T12:07:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 Processor -- start 2024-09-05T12:07:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 -- start start 2024-09-05T12:07:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2ab0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441af100 con 0x5583b2a8d3e0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441b1980 con 0x5583b2a8cac0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.909+0000 7f1f4f236740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441af770 con 0x5583b2a97ca0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2ab0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40452/0 (socket says 172.21.15.59:40452) 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 -- 172.21.15.59:0/3450806451 learned_addr learned my addr 172.21.15.59:0/3450806451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f4a800640 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 -- 172.21.15.59:0/3450806451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 msgr2=0x5583b2a8f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a8f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 -- 172.21.15.59:0/3450806451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 msgr2=0x5583b2ab0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2ab0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 -- 172.21.15.59:0/3450806451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b2aaa8a0 con 0x5583b2a8cac0 2024-09-05T12:07:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.910+0000 7f1f49e00640 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004650 tx=0x7f1f38009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7565f5431bfbef08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f32600640 1 -- 172.21.15.59:0/3450806451 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38040040 con 0x5583b2a8cac0 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f32600640 1 -- 172.21.15.59:0/3450806451 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f380054e0 con 0x5583b2a8cac0 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f32600640 1 -- 172.21.15.59:0/3450806451 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38035040 con 0x5583b2a8cac0 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 -- 172.21.15.59:0/3450806451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 msgr2=0x5583b2aafaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004650 tx=0x7f1f38009260 comp rx=0 tx=0).stop 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 -- 172.21.15.59:0/3450806451 shutdown_connections 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a97ca0 0x5583b2a8f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a8d3e0 0x5583b2ab0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 --2- 172.21.15.59:0/3450806451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aafaf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 -- 172.21.15.59:0/3450806451 >> 172.21.15.59:0/3450806451 conn(0x5583b2a89050 msgr2=0x5583b2a93a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 -- 172.21.15.59:0/3450806451 shutdown_connections 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.911+0000 7f1f4f236740 1 -- 172.21.15.59:0/3450806451 wait complete. 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 -- start start 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 Processor -- start 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aa0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8d3e0 0x5583b2a9b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441afe40 con 0x5583b2a97ca0 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441b12b0 con 0x5583b2a8cac0 2024-09-05T12:07:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f441b0510 con 0x5583b2a8d3e0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.912+0000 7f1f4f236740 10 client.0 _ll_register_callbacks cb 0x7f1f34009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aa0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39068/0 (socket says 172.21.15.59:39068) 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f49400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8d3e0 0x5583b2a9b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 -- 172.21.15.59:0/4094122407 learned_addr learned my addr 172.21.15.59:0/4094122407 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 -- 172.21.15.59:0/4094122407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8d3e0 msgr2=0x5583b2a9b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8d3e0 0x5583b2a9b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 -- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 msgr2=0x5583b2aa0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aa0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 -- 172.21.15.59:0/4094122407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b2a9c160 con 0x5583b2a97ca0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f4a800640 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f400cce80 tx=0x7f1f4009c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a9629a306e57b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.913+0000 7f1f27400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:07:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f400b8070 con 0x5583b2a97ca0 2024-09-05T12:07:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1f400042e0 con 0x5583b2a97ca0 2024-09-05T12:07:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f380054e0 con 0x5583b2a97ca0 2024-09-05T12:07:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1f40086040 con 0x5583b2a97ca0 2024-09-05T12:07:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 10 client.16878 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5583b2b129d0 con 0x5583b2a97ca0 2024-09-05T12:07:23.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f380054e0 con 0x5583b2a97ca0 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1f40024060 con 0x5583b2a97ca0 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 10 client.16878 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 20 client.16878 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2b130f0 0x5583b2b15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 4 client.16878 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.914+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583b29fad10 con 0x5583b2b130f0 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.915+0000 7f1f49e00640 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2b130f0 0x5583b2b15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.915+0000 7f1f49e00640 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2b130f0 0x5583b2b15510 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdda645c22802ec4 server_cookie=137f3de8bf767601 in_seq=0 out_seq=0 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.915+0000 7f1f27400640 10 client.16878 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f27400640 1 -- 172.21.15.59:0/4094122407 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583b29fad10 con 0x5583b2b130f0 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f27400640 10 client.16878 handle_command_reply: tid=0 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 client.16878 shutdown 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 msgr2=0x5583b2a9baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f400cce80 tx=0x7f1f4009c3a0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 shutdown_connections 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5583b2b130f0 0x5583b2b15510 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583b2a97ca0 0x5583b2a9baf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.916+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583b2a8d3e0 0x5583b2a9b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4f236740 1 --2- 172.21.15.59:0/4094122407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583b2a8cac0 0x5583b2aa0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 >> 172.21.15.59:0/4094122407 conn(0x5583b2a89050 msgr2=0x5583b2a879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f27400640 0 client.16878 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 shutdown_connections 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4f236740 1 -- 172.21.15.59:0/4094122407 wait complete. 2024-09-05T12:07:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4f236740 20 client.16878 trim_cache size 0 max 16384 2024-09-05T12:07:23.922 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:23.922 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:07:23.922 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "c2b16710-a1b3-47dd-b408-f7bf6f21a43a", 2024-09-05T12:07:23.922 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:07:23.922 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 msgr2=0x7f1f2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.917+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 0x7f1f2c0a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5583b2a8a3d0 tx=0x7f1f34009c00 comp rx=0 tx=0).stop 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 msgr2=0x7f1f4419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 0x7f1f4419ef30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000da10 tx=0x7f1f4000ded0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 shutdown_connections 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1f2c0a2b80 0x7f1f2c0a5040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f44106f10 0x7f1f4419ef30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f44102790 0x7f1f4419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 --2- 172.21.15.59:0/698867786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44101de0 0x7f1f4419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:23.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 >> 172.21.15.59:0/698867786 conn(0x7f1f440fd8c0 msgr2=0x7f1f440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:23.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 shutdown_connections 2024-09-05T12:07:23.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:23.918+0000 7f1f4c800640 1 -- 172.21.15.59:0/698867786 wait complete. 2024-09-05T12:07:24.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:07:24.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:24 smithi059 ceph-mon[30025]: pgmap v686: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:24.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:24 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:07:24.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:24 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:24.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:24 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:07:24.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:24 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:24.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:24 smithi189 ceph-mon[34772]: pgmap v686: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:24.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:24 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:07:24.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:24 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:24.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:24 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:07:24.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:24 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:24 smithi089 ceph-mon[34499]: pgmap v686: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:24 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:07:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:24 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:24 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:07:24.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:24 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:25.145 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:25.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.707+0000 7f4ce3c00640 1 -- 172.21.15.59:0/3190096468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc104010 msgr2=0x7f4cdc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.707+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/3190096468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc104010 0x7f4cdc1043f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.707+0000 7f4ce3c00640 1 -- 172.21.15.59:0/3190096468 shutdown_connections 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.707+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/3190096468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc109140 0x7f4cdc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.707+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/3190096468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.708+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/3190096468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc104010 0x7f4cdc1043f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.708+0000 7f4ce3c00640 1 -- 172.21.15.59:0/3190096468 >> 172.21.15.59:0/3190096468 conn(0x7f4cdc0ffa50 msgr2=0x7f4cdc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.708+0000 7f4ce3c00640 1 -- 172.21.15.59:0/3190096468 shutdown_connections 2024-09-05T12:07:25.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.708+0000 7f4ce3c00640 1 -- 172.21.15.59:0/3190096468 wait complete. 2024-09-05T12:07:25.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 Processor -- start 2024-09-05T12:07:25.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 -- start start 2024-09-05T12:07:25.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 0x7f4cdc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1141c0 con 0x7f4cdc109140 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc114040 con 0x7f4cdc1049c0 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce3c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc114340 con 0x7f4cdc104010 2024-09-05T12:07:25.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce1200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce1200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38520/0 (socket says 172.21.15.59:38520) 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.709+0000 7f4ce1200640 1 -- 172.21.15.59:0/578879002 learned_addr learned my addr 172.21.15.59:0/578879002 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 0x7f4cdc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 -- 172.21.15.59:0/578879002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 msgr2=0x7f4cdc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 -- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 msgr2=0x7f4cdc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 0x7f4cdc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 -- 172.21.15.59:0/578879002 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc19f5d0 con 0x7f4cdc104010 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 0x7f4cdc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:07:25.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4ce1200640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004770 tx=0x7f4cd0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.710+0000 7f4cd9600640 1 -- 172.21.15.59:0/578879002 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd003f070 con 0x7f4cdc104010 2024-09-05T12:07:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.711+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc19f860 con 0x7f4cdc104010 2024-09-05T12:07:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.711+0000 7f4cd9600640 1 -- 172.21.15.59:0/578879002 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0031ca0 con 0x7f4cdc104010 2024-09-05T12:07:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.711+0000 7f4cd9600640 1 -- 172.21.15.59:0/578879002 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0035030 con 0x7f4cdc104010 2024-09-05T12:07:25.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.711+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1a71a0 con 0x7f4cdc104010 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 Processor -- start 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4cd9600640 1 -- 172.21.15.59:0/578879002 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4cd0042070 con 0x7f4cdc104010 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 -- start start 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0ecd70 0x55717d0ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55717d0483e0 con 0x55717d0e1b90 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55717cfefd80 con 0x55717d0ecd70 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55717d0443a0 con 0x55717d0e24b0 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.714+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39090/0 (socket says 172.21.15.59:39090) 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 -- 172.21.15.59:0/2442404901 learned_addr learned my addr 172.21.15.59:0/2442404901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1200640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0ecd70 0x55717d0ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 -- 172.21.15.59:0/2442404901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 msgr2=0x55717d0ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 -- 172.21.15.59:0/2442404901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0ecd70 msgr2=0x55717d0ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0ecd70 0x55717d0ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 -- 172.21.15.59:0/2442404901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717d00eae0 con 0x55717d0e1b90 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1200640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:07:25.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.715+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009fd0 tx=0x7f4cd4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea263baabf994119 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4cd9600640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4cd9600640 1 -- 172.21.15.59:0/578879002 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4cd00efc20 con 0x7f4cdc104010 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4cce000640 1 -- 172.21.15.59:0/2442404901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4011700 con 0x55717d0e1b90 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4cce000640 1 -- 172.21.15.59:0/2442404901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd4011d80 con 0x55717d0e1b90 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4cce000640 1 -- 172.21.15.59:0/2442404901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400e3c0 con 0x55717d0e1b90 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4ce1200640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4ce65ce740 1 -- 172.21.15.59:0/2442404901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 msgr2=0x55717d0e1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.716+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009fd0 tx=0x7f4cd4010860 comp rx=0 tx=0).stop 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce1200640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 0x7f4cc40a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b8530 tx=0x7f4cd0009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 -- 172.21.15.59:0/2442404901 shutdown_connections 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0ecd70 0x55717d0ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/2442404901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0e1f70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 -- 172.21.15.59:0/2442404901 >> 172.21.15.59:0/2442404901 conn(0x55717d0de120 msgr2=0x55717d0de500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 -- 172.21.15.59:0/2442404901 shutdown_connections 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 -- 172.21.15.59:0/2442404901 wait complete. 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 -- start start 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.717+0000 7f4ce65ce740 1 Processor -- start 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0ecd70 0x55717d0ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55717d0ef9e0 con 0x55717d0ecd70 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55717d0030f0 con 0x55717d0e1b90 2024-09-05T12:07:25.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55717d0efb60 con 0x55717d0e24b0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce65ce740 10 client.0 _ll_register_callbacks cb 0x55717cff49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0ecd70 0x55717d0ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40486/0 (socket says 172.21.15.59:40486) 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.718+0000 7f4ce1c00640 1 -- 172.21.15.59:0/902029658 learned_addr learned my addr 172.21.15.59:0/902029658 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 -- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 msgr2=0x55717d0e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 -- 172.21.15.59:0/902029658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0ecd70 msgr2=0x55717d0ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0ecd70 0x55717d0ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 -- 172.21.15.59:0/902029658 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55717cdb93c0 con 0x55717d0e24b0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ea210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80126d0 tx=0x7f4cc8012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff719ddeface645 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.719+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004020 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4cc80041c0 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd4011d80 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4cc8025050 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4ce65ce740 10 client.26818 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55717d0fb770 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd4011d80 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.720+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4cc8003ad0 con 0x55717d0e24b0 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 10 client.26818 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 10 client.26818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 20 client.26818 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:25.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d0fcc80 0x55717d0ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 4 client.26818 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:25.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55717d055950 con 0x55717d0fcc80 2024-09-05T12:07:25.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.721+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d0fcc80 0x55717d0ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.722+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d0fcc80 0x55717d0ff0a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74157a320bbea7c5 server_cookie=251a459a2b3695df in_seq=0 out_seq=0 2024-09-05T12:07:25.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.722+0000 7f4cc2a00640 10 client.26818 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.722+0000 7f4cc2a00640 1 -- 172.21.15.59:0/902029658 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55717d055950 con 0x55717d0fcc80 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.722+0000 7f4cc2a00640 10 client.26818 handle_command_reply: tid=0 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.722+0000 7f4ce65ce740 1 client.26818 shutdown 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 msgr2=0x55717d0ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ea210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80126d0 tx=0x7f4cc8012ad0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 shutdown_connections 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d0fcc80 0x55717d0ff0a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0ecd70 0x55717d0ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0e24b0 0x55717d0ea210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4cc2a00640 0 client.26818 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/902029658 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 >> 172.21.15.59:0/902029658 conn(0x55717d0de120 msgr2=0x55717d0e3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 shutdown_connections 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 1 -- 172.21.15.59:0/902029658 wait complete. 2024-09-05T12:07:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.723+0000 7f4ce65ce740 20 client.26818 trim_cache size 0 max 16384 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 Processor -- start 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 -- start start 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d104eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1af060 con 0x55717d0e24b0 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1b18e0 con 0x55717d0e1b90 2024-09-05T12:07:25.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.741+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1af6d0 con 0x55717d0ecd70 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40500/0 (socket says 172.21.15.59:40500) 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4cdbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39098/0 (socket says 172.21.15.59:39098) 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3879455861 learned_addr learned my addr 172.21.15.59:0/3879455861 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d104eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3879455861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 msgr2=0x55717d104eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d104eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3879455861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 msgr2=0x55717d104970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3879455861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717d0ec640 con 0x55717d0e1b90 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.742+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4cd401bc60 tx=0x7f4cd4010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7e182640d12e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ccd600640 1 -- 172.21.15.59:0/3879455861 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd401b630 con 0x55717d0e1b90 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ccd600640 1 -- 172.21.15.59:0/3879455861 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd401be50 con 0x55717d0e1b90 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ccd600640 1 -- 172.21.15.59:0/3879455861 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd402fc60 con 0x55717d0e1b90 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3879455861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 msgr2=0x55717d0dd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4cd401bc60 tx=0x7f4cd4010fd0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3879455861 shutdown_connections 2024-09-05T12:07:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e24b0 0x55717d104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3879455861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e1b90 0x55717d0dd0b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.743+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3879455861 >> 172.21.15.59:0/3879455861 conn(0x55717d0de120 msgr2=0x55717d0dc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.744+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3879455861 shutdown_connections 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.744+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3879455861 wait complete. 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.744+0000 7f4ce65ce740 1 -- start start 2024-09-05T12:07:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.744+0000 7f4ce65ce740 1 Processor -- start 2024-09-05T12:07:25.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e24b0 0x55717d0f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d166940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1afda0 con 0x55717d0e1b90 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1b1210 con 0x55717d0e24b0 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1b0470 con 0x55717d0ecd70 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce65ce740 10 client.0 _ll_register_callbacks cb 0x7f4cc8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39106/0 (socket says 172.21.15.59:39106) 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3800817900 learned_addr learned my addr 172.21.15.59:0/3800817900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e24b0 0x55717d0f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d166940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3800817900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 msgr2=0x55717d166940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d166940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e24b0 msgr2=0x55717d0f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e24b0 0x55717d0f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.745+0000 7f4ce1c00640 1 -- 172.21.15.59:0/3800817900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55717d166e80 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4ce1c00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009560 tx=0x7f4cd402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7f9ff0d7bfeede server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4cc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd403e030 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4cd401b430 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd401be50 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4cd404b050 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4ce65ce740 10 client.16896 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.746+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55717d168030 con 0x55717d0e1b90 2024-09-05T12:07:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd401be50 con 0x55717d0e1b90 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4cd40118d0 con 0x55717d0e1b90 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 10 client.16896 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 20 client.16896 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d168400 0x55717d16a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 4 client.16896 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55717d04fe60 con 0x55717d168400 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.747+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d168400 0x55717d16a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4cdbe00640 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d168400 0x55717d16a7f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7200963541470e3 server_cookie=17df16408edfca48 in_seq=0 out_seq=0 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4cc2a00640 10 client.16896 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4cc2a00640 1 -- 172.21.15.59:0/3800817900 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55717d04fe60 con 0x55717d168400 2024-09-05T12:07:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4cc2a00640 10 client.16896 handle_command_reply: tid=0 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 client.16896 shutdown 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 msgr2=0x55717d0f0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009560 tx=0x7f4cd402f840 comp rx=0 tx=0).stop 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 shutdown_connections 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55717d0ecd70 0x55717d166940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55717d0e24b0 0x55717d0f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55717d168400 0x55717d16a7f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 --2- 172.21.15.59:0/3800817900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55717d0e1b90 0x55717d0f0f10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 >> 172.21.15.59:0/3800817900 conn(0x55717d0de120 msgr2=0x55717d0ea4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.748+0000 7f4cc2a00640 0 client.16896 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 shutdown_connections 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce65ce740 1 -- 172.21.15.59:0/3800817900 wait complete. 2024-09-05T12:07:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce65ce740 20 client.16896 trim_cache size 0 max 16384 2024-09-05T12:07:25.755 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:07:25.755 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:07:25.755 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 msgr2=0x7f4cc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 0x7f4cc40a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b8530 tx=0x7f4cd0009260 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 msgr2=0x7f4cdc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004770 tx=0x7f4cd0003c10 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 shutdown_connections 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cdc109140 0x7f4cdc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cdc1049c0 0x7f4cdc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 --2- 172.21.15.59:0/578879002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cdc104010 0x7f4cdc19a620 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.749+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 >> 172.21.15.59:0/578879002 conn(0x7f4cdc0ffa50 msgr2=0x7f4cdc100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.750+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 shutdown_connections 2024-09-05T12:07:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:25.750+0000 7f4ce3c00640 1 -- 172.21.15.59:0/578879002 wait complete. 2024-09-05T12:07:25.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:25 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:07:25.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:25 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:07:25.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:25 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:07:26.612 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:07:26.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:07:26.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:26 smithi059 ceph-mon[30025]: pgmap v687: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:26.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:26 smithi189 ceph-mon[34772]: pgmap v687: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:26.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:26 smithi089 ceph-mon[34499]: pgmap v687: 129 pgs: 129 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:26.995 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 -- 172.21.15.59:0/2609106665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc108f60 msgr2=0x7f44dc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 --2- 172.21.15.59:0/2609106665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc108f60 0x7f44dc110ae0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009ca0 tx=0x7f44d402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 -- 172.21.15.59:0/2609106665 shutdown_connections 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 --2- 172.21.15.59:0/2609106665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc108f60 0x7f44dc110ae0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 --2- 172.21.15.59:0/2609106665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc1047e0 0x7f44dc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 --2- 172.21.15.59:0/2609106665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc103e30 0x7f44dc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 -- 172.21.15.59:0/2609106665 >> 172.21.15.59:0/2609106665 conn(0x7f44dc0ff8d0 msgr2=0x7f44dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:27.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 -- 172.21.15.59:0/2609106665 shutdown_connections 2024-09-05T12:07:27.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.524+0000 7f44e3000640 1 -- 172.21.15.59:0/2609106665 wait complete. 2024-09-05T12:07:27.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.525+0000 7f44e3000640 1 Processor -- start 2024-09-05T12:07:27.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.525+0000 7f44e3000640 1 -- start start 2024-09-05T12:07:27.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 0x7f44dc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:27.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc1047e0 0x7f44dc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:27.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 0x7f44dc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc114010 con 0x7f44dc103e30 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc113e90 con 0x7f44dc108f60 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e3000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc114190 con 0x7f44dc1047e0 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 0x7f44dc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 0x7f44dc076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39118/0 (socket says 172.21.15.59:39118) 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e1c00640 1 -- 172.21.15.59:0/3586424640 learned_addr learned my addr 172.21.15.59:0/3586424640 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e2600640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 0x7f44dc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.526+0000 7f44e1200640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc1047e0 0x7f44dc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 -- 172.21.15.59:0/3586424640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc1047e0 msgr2=0x7f44dc076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc1047e0 0x7f44dc076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 -- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 msgr2=0x7f44dc076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 0x7f44dc076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 -- 172.21.15.59:0/3586424640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc1aaa80 con 0x7f44dc108f60 2024-09-05T12:07:27.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e2600640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 0x7f44dc077180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f44d402aeb0 tx=0x7f44d4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:27.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4031650 con 0x7f44dc108f60 2024-09-05T12:07:27.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc1aad10 con 0x7f44dc108f60 2024-09-05T12:07:27.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d40317f0 con 0x7f44dc108f60 2024-09-05T12:07:27.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d402f950 con 0x7f44dc108f60 2024-09-05T12:07:27.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.527+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1ab2a0 con 0x7f44dc108f60 2024-09-05T12:07:27.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.528+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc105310 con 0x7f44dc108f60 2024-09-05T12:07:27.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.531+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f44d402fbd0 con 0x7f44dc108f60 2024-09-05T12:07:27.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.532+0000 7f44da000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 0x7f44c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:27.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.532+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f44d40f07a0 con 0x7f44dc108f60 2024-09-05T12:07:27.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.532+0000 7f44e1c00640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 0x7f44c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:27.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.533+0000 7f44e1c00640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 0x7f44c40a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009cd0 tx=0x7f44d0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:27.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.534+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44d40bd290 con 0x7f44dc108f60 2024-09-05T12:07:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.760+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44dc077c30 con 0x7f44dc108f60 2024-09-05T12:07:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.761+0000 7f44da000640 1 -- 172.21.15.59:0/3586424640 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f44d40bcb90 con 0x7f44dc108f60 2024-09-05T12:07:27.763 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:07:27.763 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:07:27.764 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 msgr2=0x7f44c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 0x7f44c40a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009cd0 tx=0x7f44d0005f90 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 msgr2=0x7f44dc077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 0x7f44dc077180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f44d402aeb0 tx=0x7f44d4003f10 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 shutdown_connections 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f44c40a2c00 0x7f44c40a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44dc108f60 0x7f44dc077180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc1047e0 0x7f44dc076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 --2- 172.21.15.59:0/3586424640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44dc103e30 0x7f44dc076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 >> 172.21.15.59:0/3586424640 conn(0x7f44dc0ff8d0 msgr2=0x7f44dc101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.764+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 shutdown_connections 2024-09-05T12:07:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:27.765+0000 7f44e3000640 1 -- 172.21.15.59:0/3586424640 wait complete. 2024-09-05T12:07:28.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:07:28.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:28 smithi189 ceph-mon[34772]: pgmap v688: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:07:28.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:28 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3586424640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:28 smithi089 ceph-mon[34499]: pgmap v688: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:07:28.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:28 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3586424640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:28 smithi059 ceph-mon[30025]: pgmap v688: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:07:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:28 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3586424640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:28.966 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:29.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 -- 172.21.15.59:0/1018494329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d441019c0 msgr2=0x7f7d44101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 --2- 172.21.15.59:0/1018494329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d441019c0 0x7f7d44101e20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009930 tx=0x7f7d2c02a420 comp rx=0 tx=0).stop 2024-09-05T12:07:29.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 -- 172.21.15.59:0/1018494329 shutdown_connections 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 --2- 172.21.15.59:0/1018494329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d44102360 0x7f7d4410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 --2- 172.21.15.59:0/1018494329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d441019c0 0x7f7d44101e20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 --2- 172.21.15.59:0/1018494329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 0x7f7d44107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 -- 172.21.15.59:0/1018494329 >> 172.21.15.59:0/1018494329 conn(0x7f7d440fd740 msgr2=0x7f7d440ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 -- 172.21.15.59:0/1018494329 shutdown_connections 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.540+0000 7f7d43200640 1 -- 172.21.15.59:0/1018494329 wait complete. 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 Processor -- start 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 -- start start 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 0x7f7d441982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 0x7f7d4419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44111e10 con 0x7f7d44102360 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44111c90 con 0x7f7d441079c0 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.541+0000 7f7d43200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44111f90 con 0x7f7d441019c0 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39132/0 (socket says 172.21.15.59:39132) 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 -- 172.21.15.59:0/1966572140 learned_addr learned my addr 172.21.15.59:0/1966572140 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 0x7f7d441982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d42800640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 0x7f7d4419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 -- 172.21.15.59:0/1966572140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 msgr2=0x7f7d441982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 0x7f7d441982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 -- 172.21.15.59:0/1966572140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 msgr2=0x7f7d4419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 0x7f7d4419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d39400640 1 -- 172.21.15.59:0/1966572140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4419d280 con 0x7f7d44102360 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.542+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 0x7f7d441982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:29.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c02a940 tx=0x7f7d2c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d3b400640 1 -- 172.21.15.59:0/1966572140 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c03f070 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d3b400640 1 -- 172.21.15.59:0/1966572140 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c031c90 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4419d510 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d3b400640 1 -- 172.21.15.59:0/1966572140 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c035030 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.543+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441a4e50 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.546+0000 7f7d3b400640 1 -- 172.21.15.59:0/1966572140 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d2c042070 con 0x7f7d44102360 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 Processor -- start 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 -- start start 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 0x559e72c6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c7e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e72bd33e0 con 0x559e72c6d640 2024-09-05T12:07:29.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e72b7ad80 con 0x559e72c6cd20 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.547+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e72bcf3a0 con 0x559e72c7c430 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d41e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d42800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 0x559e72c6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c7e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d42800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 0x559e72c6d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40548/0 (socket says 172.21.15.59:40548) 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d41e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39148/0 (socket says 172.21.15.59:39148) 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d42800640 1 -- 172.21.15.59:0/1568791429 learned_addr learned my addr 172.21.15.59:0/1568791429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d3b400640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 0x7f7d280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 -- 172.21.15.59:0/1568791429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 msgr2=0x559e72c6d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 0x559e72c6d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 -- 172.21.15.59:0/1568791429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 msgr2=0x559e72c7bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d3b400640 1 -- 172.21.15.59:0/1966572140 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7d2c0efb80 con 0x7f7d44102360 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d39400640 1 -- 172.21.15.59:0/1568791429 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e72b99ae0 con 0x559e72c7c430 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.548+0000 7f7d42800640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 0x7f7d280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.549+0000 7f7d39400640 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c7e820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ed1b0 tx=0x7f7d2c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f4bb13d27b6629 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.549+0000 7f7d33400640 1 -- 172.21.15.59:0/1568791429 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0bb980 con 0x559e72c7c430 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.549+0000 7f7d33400640 1 -- 172.21.15.59:0/1568791429 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c0f7040 con 0x559e72c7c430 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.549+0000 7f7d33400640 1 -- 172.21.15.59:0/1568791429 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c031e40 con 0x559e72c7c430 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1568791429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 msgr2=0x559e72c7e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c7e820 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ed1b0 tx=0x7f7d2c009260 comp rx=0 tx=0).stop 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d42800640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 0x7f7d280a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c005f40 tx=0x7f7d3c004a40 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1568791429 shutdown_connections 2024-09-05T12:07:29.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c7e820 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c7bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1568791429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6cd20 0x559e72c6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1568791429 >> 172.21.15.59:0/1568791429 conn(0x559e72c693d0 msgr2=0x559e72c697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1568791429 shutdown_connections 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1568791429 wait complete. 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 -- start start 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.550+0000 7f7d4ba02740 1 Processor -- start 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e72c7edc0 con 0x559e72c6d640 2024-09-05T12:07:29.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e72b8e0f0 con 0x559e72c7c430 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e72c7ef40 con 0x559e72c6cd20 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d4ba02740 10 client.0 _ll_register_callbacks cb 0x559e72b75bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39154/0 (socket says 172.21.15.59:39154) 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d39400640 1 -- 172.21.15.59:0/1406904477 learned_addr learned my addr 172.21.15.59:0/1406904477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.551+0000 7f7d42800640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 -- 172.21.15.59:0/1406904477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 msgr2=0x559e72c792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 -- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 msgr2=0x559e72c79d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c79d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 -- 172.21.15.59:0/1406904477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e729443c0 con 0x559e72c6d640 2024-09-05T12:07:29.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d39400640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006010 tx=0x7f7d2c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0a9e6d60279762 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d27e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c049070 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7d2c03b460 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2c0f7040 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c0d6050 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d4ba02740 10 client.16914 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.552+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e72c7edc0 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c0f7040 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7d2c03bce0 con 0x559e72c6d640 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 10 client.16914 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 20 client.16914 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72c8c020 0x559e72c8e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 4 client.16914 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.553+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e72be0950 con 0x559e72c8c020 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.554+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72c8c020 0x559e72c8e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.554+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72c8c020 0x559e72c8e410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512acad35c2a357e server_cookie=f98e1af05ae0ad23 in_seq=0 out_seq=0 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.554+0000 7f7d27e00640 10 client.16914 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d27e00640 1 -- 172.21.15.59:0/1406904477 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e72be0950 con 0x559e72c8c020 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d27e00640 10 client.16914 handle_command_reply: tid=0 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 client.16914 shutdown 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 msgr2=0x559e72c797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006010 tx=0x7f7d2c02a420 comp rx=0 tx=0).stop 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 shutdown_connections 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c79d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72c8c020 0x559e72c8e410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c797f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1406904477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 >> 172.21.15.59:0/1406904477 conn(0x559e72c693d0 msgr2=0x559e72c6f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d27e00640 0 client.16914 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 shutdown_connections 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.555+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1406904477 wait complete. 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.556+0000 7f7d4ba02740 20 client.16914 trim_cache size 0 max 16384 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.569+0000 7f7d4ba02740 1 Processor -- start 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.569+0000 7f7d4ba02740 1 -- start start 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c68380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c94180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441aaac0 con 0x559e72c6d640 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441ad340 con 0x559e72c7c430 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441ab130 con 0x559e72c6cd20 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39162/0 (socket says 172.21.15.59:39162) 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 -- 172.21.15.59:0/1966667808 learned_addr learned my addr 172.21.15.59:0/1966667808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d42800640 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c94180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d41e00640 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c68380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 -- 172.21.15.59:0/1966667808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 msgr2=0x559e72c68380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c68380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 -- 172.21.15.59:0/1966667808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 msgr2=0x559e72c94180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c94180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.570+0000 7f7d39400640 1 -- 172.21.15.59:0/1966667808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e72c6e2f0 con 0x559e72c6d640 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.571+0000 7f7d39400640 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0bbfb0 tx=0x7f7d2c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6962a0baeec3db66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.571+0000 7f7d32a00640 1 -- 172.21.15.59:0/1966667808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c049070 con 0x559e72c6d640 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.571+0000 7f7d32a00640 1 -- 172.21.15.59:0/1966667808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c031e40 con 0x559e72c6d640 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.571+0000 7f7d32a00640 1 -- 172.21.15.59:0/1966667808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c040260 con 0x559e72c6d640 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1966667808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 msgr2=0x559e72c93c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0bbfb0 tx=0x7f7d2c005e50 comp rx=0 tx=0).stop 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1966667808 shutdown_connections 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c7c430 0x559e72c94180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6d640 0x559e72c93c40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/1966667808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c6cd20 0x559e72c68380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1966667808 >> 172.21.15.59:0/1966667808 conn(0x559e72c693d0 msgr2=0x559e72c67c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1966667808 shutdown_connections 2024-09-05T12:07:29.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- 172.21.15.59:0/1966667808 wait complete. 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.572+0000 7f7d4ba02740 1 -- start start 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 Processor -- start 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441ab800 con 0x559e72c6cd20 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441acc70 con 0x559e72c6d640 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441abed0 con 0x559e72c7c430 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d4ba02740 10 client.0 _ll_register_callbacks cb 0x7f7d34009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d41e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d39400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d39400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40574/0 (socket says 172.21.15.59:40574) 2024-09-05T12:07:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d39400640 1 -- 172.21.15.59:0/864966057 learned_addr learned my addr 172.21.15.59:0/864966057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d41e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39176/0 (socket says 172.21.15.59:39176) 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.573+0000 7f7d42800640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 -- 172.21.15.59:0/864966057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 msgr2=0x559e72c766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 -- 172.21.15.59:0/864966057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 msgr2=0x559e72c849c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 -- 172.21.15.59:0/864966057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e72c76f10 con 0x559e72c6d640 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d41e00640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d39400640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0f3600 tx=0x7f7d2c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67732af2d64633f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d27e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c049070 con 0x559e72c6d640 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e72bdae60 con 0x559e72c6d640 2024-09-05T12:07:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7d2c0c4750 con 0x559e72c6d640 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c0bb980 con 0x559e72c6d640 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d4ba02740 10 client.26915 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.574+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e72c76f10 con 0x559e72c6d640 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559e72bdae60 con 0x559e72c6d640 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7d2c0dabb0 con 0x559e72c6d640 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 10 client.26915 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 20 client.26915 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72cf6f90 0x559e72cf9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 4 client.26915 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e72b57010 con 0x559e72cf6f90 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.575+0000 7f7d41e00640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72cf6f90 0x559e72cf9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:29.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d41e00640 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72cf6f90 0x559e72cf9380 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eefae7cb79462e server_cookie=c5fcd0e219fb62e7 in_seq=0 out_seq=0 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d27e00640 10 client.26915 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d27e00640 1 -- 172.21.15.59:0/864966057 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e72b57010 con 0x559e72cf6f90 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d27e00640 10 client.26915 handle_command_reply: tid=0 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d4ba02740 1 client.26915 shutdown 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 msgr2=0x559e72c760f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.576+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0f3600 tx=0x7f7d2c009260 comp rx=0 tx=0).stop 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 shutdown_connections 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e72c7c430 0x559e72c766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e72c6d640 0x559e72c760f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e72cf6f90 0x559e72cf9380 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 --2- 172.21.15.59:0/864966057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e72c6cd20 0x559e72c849c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d27e00640 0 client.26915 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:29.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 >> 172.21.15.59:0/864966057 conn(0x559e72c693d0 msgr2=0x559e72c7b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 shutdown_connections 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 1 -- 172.21.15.59:0/864966057 wait complete. 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d4ba02740 20 client.26915 trim_cache size 0 max 16384 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 msgr2=0x7f7d280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 0x7f7d280a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c005f40 tx=0x7f7d3c004a40 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 msgr2=0x7f7d44198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.577+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c02a940 tx=0x7f7d2c003b90 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 shutdown_connections 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d280a2b80 0x7f7d280a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d441079c0 0x7f7d4419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44102360 0x7f7d44198810 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 --2- 172.21.15.59:0/1966572140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d441019c0 0x7f7d441982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 >> 172.21.15.59:0/1966572140 conn(0x7f7d440fd740 msgr2=0x7f7d440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 shutdown_connections 2024-09-05T12:07:29.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:29.578+0000 7f7d43200640 1 -- 172.21.15.59:0/1966572140 wait complete. 2024-09-05T12:07:30.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:30 smithi059 ceph-mon[30025]: pgmap v689: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:30.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:30 smithi189 ceph-mon[34772]: pgmap v689: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:30.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:30 smithi089 ceph-mon[34499]: pgmap v689: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:31.435 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:07:31.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:07:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 -- 172.21.15.59:0/1066288620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a4074ee0 msgr2=0x7f59a410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 --2- 172.21.15.59:0/1066288620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a4074ee0 0x7f59a410def0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5990009900 tx=0x7f599002a480 comp rx=0 tx=0).stop 2024-09-05T12:07:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 -- 172.21.15.59:0/1066288620 shutdown_connections 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 --2- 172.21.15.59:0/1066288620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 0x7f59a4110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 --2- 172.21.15.59:0/1066288620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a4074ee0 0x7f59a410def0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 --2- 172.21.15.59:0/1066288620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a40745c0 0x7f59a40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.363+0000 7f59aba00640 1 -- 172.21.15.59:0/1066288620 >> 172.21.15.59:0/1066288620 conn(0x7f59a40fd8b0 msgr2=0x7f59a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.364+0000 7f59aba00640 1 -- 172.21.15.59:0/1066288620 shutdown_connections 2024-09-05T12:07:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.364+0000 7f59aba00640 1 -- 172.21.15.59:0/1066288620 wait complete. 2024-09-05T12:07:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.365+0000 7f59aba00640 1 Processor -- start 2024-09-05T12:07:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.365+0000 7f59aba00640 1 -- start start 2024-09-05T12:07:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.365+0000 7f59aba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.365+0000 7f59aba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a4074ee0 0x7f59a419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59aba00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 0x7f59a419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59aba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114190 con 0x7f59a40745c0 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59aba00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114010 con 0x7f59a410e430 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59aba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114310 con 0x7f59a4074ee0 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a4074ee0 0x7f59a419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39896/0 (socket says 172.21.15.59:39896) 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 -- 172.21.15.59:0/3628222677 learned_addr learned my addr 172.21.15.59:0/3628222677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9a00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 0x7f59a419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 -- 172.21.15.59:0/3628222677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a4074ee0 msgr2=0x7f59a419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a4074ee0 0x7f59a419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 -- 172.21.15.59:0/3628222677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 msgr2=0x7f59a419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 0x7f59a419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.366+0000 7f59a9000640 1 -- 172.21.15.59:0/3628222677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a419f600 con 0x7f59a40745c0 2024-09-05T12:07:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59a9000640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f599800e9f0 tx=0x7f599800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59a1600640 1 -- 172.21.15.59:0/3628222677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800cd80 con 0x7f59a40745c0 2024-09-05T12:07:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59a1600640 1 -- 172.21.15.59:0/3628222677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5998005500 con 0x7f59a40745c0 2024-09-05T12:07:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a419f8f0 con 0x7f59a40745c0 2024-09-05T12:07:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59a1600640 1 -- 172.21.15.59:0/3628222677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998010690 con 0x7f59a40745c0 2024-09-05T12:07:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.367+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41a71d0 con 0x7f59a40745c0 2024-09-05T12:07:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.370+0000 7f59ae491740 1 Processor -- start 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.370+0000 7f59a1600640 1 -- 172.21.15.59:0/3628222677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59980046c0 con 0x7f59a40745c0 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.370+0000 7f59ae491740 1 -- start start 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5c2252290 con 0x55b5c22ebac0 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b5c21f9b40 con 0x55b5c22f6ca0 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59ae491740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b5c224e250 con 0x55b5c22ec3e0 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39912/0 (socket says 172.21.15.59:39912) 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59a9a00640 1 -- 172.21.15.59:0/1685310369 learned_addr learned my addr 172.21.15.59:0/1685310369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59a9000640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.371+0000 7f59a3e00640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 -- 172.21.15.59:0/1685310369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 msgr2=0x55b5c22f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 -- 172.21.15.59:0/1685310369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 msgr2=0x55b5c22ebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 -- 172.21.15.59:0/1685310369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c1fc36a0 con 0x55b5c22f6ca0 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a9a00640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a1600640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 0x7f598c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a3e00640 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f9090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5990005940 tx=0x7f5990005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278c3fdbb33873d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a1600640 1 -- 172.21.15.59:0/3628222677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f59980cf7d0 con 0x7f59a40745c0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f59a9a00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 0x7f598c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.372+0000 7f5996000640 1 -- 172.21.15.59:0/1685310369 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990040070 con 0x55b5c22f6ca0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.373+0000 7f5996000640 1 -- 172.21.15.59:0/1685310369 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990031870 con 0x55b5c22f6ca0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.373+0000 7f5996000640 1 -- 172.21.15.59:0/1685310369 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990035040 con 0x55b5c22f6ca0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.373+0000 7f59ae491740 1 -- 172.21.15.59:0/1685310369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 msgr2=0x55b5c22f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.373+0000 7f59a9a00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 0x7f598c0a50c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f599c009fd0 tx=0x7f599c009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.373+0000 7f59ae491740 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f9090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5990005940 tx=0x7f5990005be0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 -- 172.21.15.59:0/1685310369 shutdown_connections 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f9090 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 --2- 172.21.15.59:0/1685310369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22ebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 -- 172.21.15.59:0/1685310369 >> 172.21.15.59:0/1685310369 conn(0x55b5c22e8050 msgr2=0x55b5c22e8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 -- 172.21.15.59:0/1685310369 shutdown_connections 2024-09-05T12:07:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 -- 172.21.15.59:0/1685310369 wait complete. 2024-09-05T12:07:32.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 -- start start 2024-09-05T12:07:32.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.374+0000 7f59ae491740 1 Processor -- start 2024-09-05T12:07:32.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c22f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b5c22f98b0 con 0x55b5c22f6ca0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b5c220cfa0 con 0x55b5c22ebac0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b5c22f9a30 con 0x55b5c22ec3e0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59ae491740 10 client.0 _ll_register_callbacks cb 0x55b5c1f60ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39914/0 (socket says 172.21.15.59:39914) 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59a9a00640 1 -- 172.21.15.59:0/118577164 learned_addr learned my addr 172.21.15.59:0/118577164 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.375+0000 7f59a9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c22f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 -- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 msgr2=0x55b5c22f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c22f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 -- 172.21.15.59:0/118577164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 msgr2=0x55b5c22f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 -- 172.21.15.59:0/118577164 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c2302d20 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59a3e00640 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f3990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5990006010 tx=0x7f5990005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797d9c2c653baf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f598aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990052060 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5990031870 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.376+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5990031670 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5990059050 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 10 client.26860 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b5c22f98b0 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990031870 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f59900042d0 con 0x55b5c22ec3e0 2024-09-05T12:07:32.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 10 client.26860 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 20 client.26860 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.377+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c2306a40 0x55b5c2308e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.378+0000 7f59ae491740 4 client.26860 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.378+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5c225f800 con 0x55b5c2306a40 2024-09-05T12:07:32.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.378+0000 7f59a9000640 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c2306a40 0x55b5c2308e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.378+0000 7f59a9000640 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c2306a40 0x55b5c2308e30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f004299b82986f9 server_cookie=7a62ff4809416898 in_seq=0 out_seq=0 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.378+0000 7f598aa00640 10 client.26860 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.379+0000 7f598aa00640 1 -- 172.21.15.59:0/118577164 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5c225f800 con 0x55b5c2306a40 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.379+0000 7f598aa00640 10 client.26860 handle_command_reply: tid=0 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.379+0000 7f59ae491740 1 client.26860 shutdown 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.379+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 msgr2=0x55b5c22f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.379+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f3990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5990006010 tx=0x7f5990005250 comp rx=0 tx=0).stop 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 shutdown_connections 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c2306a40 0x55b5c2308e30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c22f3990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 --2- 172.21.15.59:0/118577164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c22f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f598aa00640 0 client.26860 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 >> 172.21.15.59:0/118577164 conn(0x55b5c22e8050 msgr2=0x55b5c22edd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 shutdown_connections 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 1 -- 172.21.15.59:0/118577164 wait complete. 2024-09-05T12:07:32.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.380+0000 7f59ae491740 20 client.26860 trim_cache size 0 max 16384 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.398+0000 7f59ae491740 1 Processor -- start 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.398+0000 7f59ae491740 1 -- start start 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.398+0000 7f59ae491740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c230e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.398+0000 7f59ae491740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c230ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.398+0000 7f59ae491740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59ae491740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41af060 con 0x55b5c22f6ca0 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59ae491740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41b18e0 con 0x55b5c22ebac0 2024-09-05T12:07:32.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59ae491740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41af6d0 con 0x55b5c22ec3e0 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c230ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39928/0 (socket says 172.21.15.59:39928) 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a9a00640 1 -- 172.21.15.59:0/2315186986 learned_addr learned my addr 172.21.15.59:0/2315186986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a9000640 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c230e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 -- 172.21.15.59:0/2315186986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 msgr2=0x55b5c230e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c230e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 -- 172.21.15.59:0/2315186986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 msgr2=0x55b5c22fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.399+0000 7f59a3e00640 1 -- 172.21.15.59:0/2315186986 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c22eb8b0 con 0x55b5c22ec3e0 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f59a3e00640 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c230ea60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f599002a9a0 tx=0x7f599002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d606691327ed41 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f5995600640 1 -- 172.21.15.59:0/2315186986 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990052040 con 0x55b5c22ec3e0 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f5995600640 1 -- 172.21.15.59:0/2315186986 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990009520 con 0x55b5c22ec3e0 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f5995600640 1 -- 172.21.15.59:0/2315186986 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990040070 con 0x55b5c22ec3e0 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f59ae491740 1 -- 172.21.15.59:0/2315186986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 msgr2=0x55b5c230ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.400+0000 7f59ae491740 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c230ea60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f599002a9a0 tx=0x7f599002a480 comp rx=0 tx=0).stop 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 -- 172.21.15.59:0/2315186986 shutdown_connections 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22ec3e0 0x55b5c230ea60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 --2- 172.21.15.59:0/2315186986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ebac0 0x55b5c230e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 -- 172.21.15.59:0/2315186986 >> 172.21.15.59:0/2315186986 conn(0x55b5c22e8050 msgr2=0x55b5c22f5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 -- 172.21.15.59:0/2315186986 shutdown_connections 2024-09-05T12:07:32.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 -- 172.21.15.59:0/2315186986 wait complete. 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.401+0000 7f59ae491740 1 -- start start 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 Processor -- start 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 0x55b5c22fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41afda0 con 0x55b5c22ebac0 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41b1210 con 0x55b5c22ec3e0 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41b0470 con 0x55b5c22f6ca0 2024-09-05T12:07:32.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59ae491740 10 client.0 _ll_register_callbacks cb 0x7f599c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59a9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39940/0 (socket says 172.21.15.59:39940) 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59a9000640 1 -- 172.21.15.59:0/635986143 learned_addr learned my addr 172.21.15.59:0/635986143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.402+0000 7f59a3e00640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 0x55b5c22fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 -- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 msgr2=0x55b5c22fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 0x55b5c22fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 -- 172.21.15.59:0/635986143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 msgr2=0x55b5c22fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 -- 172.21.15.59:0/635986143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c2370750 con 0x55b5c22f6ca0 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a3e00640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 0x55b5c22fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9000640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:32.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f59a9a00640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fd6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55b5c2297bb0 tx=0x7f599c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ee67feb8255809 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f598aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.403+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c004b20 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5990009520 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f599c004cc0 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f599c034050 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 10 client.26866 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b5c2371b40 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990009520 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f599c005300 con 0x55b5c22f6ca0 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 10 client.26866 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 20 client.26866 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c23722c0 0x55b5c23746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:32.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 4 client.26866 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:07:32.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.404+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b5c2259d10 con 0x55b5c23722c0 2024-09-05T12:07:32.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.405+0000 7f59a9000640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c23722c0 0x55b5c23746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.405+0000 7f59a9000640 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c23722c0 0x55b5c23746e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbc52ba8ac80a73 server_cookie=c2bd26bf158655b in_seq=0 out_seq=0 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.405+0000 7f598aa00640 10 client.26866 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.406+0000 7f598aa00640 1 -- 172.21.15.59:0/635986143 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b5c2259d10 con 0x55b5c23722c0 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.406+0000 7f598aa00640 10 client.26866 handle_command_reply: tid=0 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.406+0000 7f59ae491740 1 client.26866 shutdown 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.406+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 msgr2=0x55b5c22fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.406+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fd6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55b5c2297bb0 tx=0x7f599c004a40 comp rx=0 tx=0).stop 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 shutdown_connections 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5c22f6ca0 0x55b5c22fd6e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b5c22ec3e0 0x55b5c22fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b5c23722c0 0x55b5c23746e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 --2- 172.21.15.59:0/635986143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b5c22ebac0 0x55b5c22fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 >> 172.21.15.59:0/635986143 conn(0x55b5c22e8050 msgr2=0x55b5c22f2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f598aa00640 0 client.26866 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 shutdown_connections 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 1 -- 172.21.15.59:0/635986143 wait complete. 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.407+0000 7f59ae491740 20 client.26866 trim_cache size 0 max 16384 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "740ca461-1ddc-421a-9790-befaf99054f7", 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:07:32.411 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 msgr2=0x7f598c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 0x7f598c0a50c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f599c009fd0 tx=0x7f599c009500 comp rx=0 tx=0).stop 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 msgr2=0x7f59a419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f599800e9f0 tx=0x7f599800edf0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 shutdown_connections 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f598c0a2c00 0x7f598c0a50c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a410e430 0x7f59a419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a4074ee0 0x7f59a419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 --2- 172.21.15.59:0/3628222677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a40745c0 0x7f59a419a650 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:32.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 >> 172.21.15.59:0/3628222677 conn(0x7f59a40fd8b0 msgr2=0x7f59a40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:32.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 shutdown_connections 2024-09-05T12:07:32.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:32.408+0000 7f59aba00640 1 -- 172.21.15.59:0/3628222677 wait complete. 2024-09-05T12:07:32.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:32 smithi189 ceph-mon[34772]: pgmap v690: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:32.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:32 smithi089 ceph-mon[34499]: pgmap v690: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:32.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:32 smithi059 ceph-mon[30025]: pgmap v690: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:33.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:07:33.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:33 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:07:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:33 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:33 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:07:33.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:33 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:33.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:33 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:07:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:33 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:33 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:07:33.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:33 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:33 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:07:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:33 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:33 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:07:33.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:33 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:07:34.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 -- 172.21.15.59:0/2008100424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd341049c0 msgr2=0x7efd34108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 --2- 172.21.15.59:0/2008100424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd341049c0 0x7efd34108a10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009900 tx=0x7efd1c02a410 comp rx=0 tx=0).stop 2024-09-05T12:07:34.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 -- 172.21.15.59:0/2008100424 shutdown_connections 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 --2- 172.21.15.59:0/2008100424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd34109140 0x7efd34110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 --2- 172.21.15.59:0/2008100424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd341049c0 0x7efd34108a10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 --2- 172.21.15.59:0/2008100424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34104010 0x7efd341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 -- 172.21.15.59:0/2008100424 >> 172.21.15.59:0/2008100424 conn(0x7efd340ffa50 msgr2=0x7efd34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 -- 172.21.15.59:0/2008100424 shutdown_connections 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.312+0000 7efd39800640 1 -- 172.21.15.59:0/2008100424 wait complete. 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.313+0000 7efd39800640 1 Processor -- start 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.313+0000 7efd39800640 1 -- start start 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd341049c0 0x7efd3419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 0x7efd3419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd34114160 con 0x7efd34104010 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd34113fe0 con 0x7efd34109140 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd39800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd341142e0 con 0x7efd341049c0 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd2a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd341049c0 0x7efd3419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39960/0 (socket says 172.21.15.59:39960) 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd32a00640 1 -- 172.21.15.59:0/3408626889 learned_addr learned my addr 172.21.15.59:0/3408626889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.314+0000 7efd33400640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 0x7efd3419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 -- 172.21.15.59:0/3408626889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd341049c0 msgr2=0x7efd3419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd341049c0 0x7efd3419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 -- 172.21.15.59:0/3408626889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 msgr2=0x7efd3419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 0x7efd3419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 -- 172.21.15.59:0/3408626889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3419f5f0 con 0x7efd34104010 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd33400640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 0x7efd3419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd32a00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd2400ebc0 tx=0x7efd2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd2be00640 1 -- 172.21.15.59:0/3408626889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24010630 con 0x7efd34104010 2024-09-05T12:07:34.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd2be00640 1 -- 172.21.15.59:0/3408626889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd24010cb0 con 0x7efd34104010 2024-09-05T12:07:34.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd2be00640 1 -- 172.21.15.59:0/3408626889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24018bd0 con 0x7efd34104010 2024-09-05T12:07:34.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3419f8e0 con 0x7efd34104010 2024-09-05T12:07:34.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.315+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd341a71c0 con 0x7efd34104010 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.318+0000 7efd3c2c8740 1 Processor -- start 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.318+0000 7efd2be00640 1 -- 172.21.15.59:0/3408626889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd240107d0 con 0x7efd34104010 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 -- start start 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a719f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a724830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a727160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c1a6803e0 con 0x563c1a71a4b0 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563c1a627d80 con 0x563c1a719b90 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c1a67c3a0 con 0x563c1a724d70 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a724830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a724830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39968/0 (socket says 172.21.15.59:39968) 2024-09-05T12:07:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd32a00640 1 -- 172.21.15.59:0/2390623566 learned_addr learned my addr 172.21.15.59:0/2390623566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a719f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.319+0000 7efd2a000640 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a727160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 -- 172.21.15.59:0/2390623566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 msgr2=0x563c1a727160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a727160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 -- 172.21.15.59:0/2390623566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 msgr2=0x563c1a724830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a724830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 -- 172.21.15.59:0/2390623566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1a646ae0 con 0x563c1a719b90 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd33400640 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a719f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009f30 tx=0x7efd2c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9285b286c50311dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd2be00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 0x7efd180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd23e00640 1 -- 172.21.15.59:0/2390623566 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c004310 con 0x563c1a719b90 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.320+0000 7efd23e00640 1 -- 172.21.15.59:0/2390623566 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd2c0044b0 con 0x563c1a719b90 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd23e00640 1 -- 172.21.15.59:0/2390623566 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0035f0 con 0x563c1a719b90 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd2be00640 1 -- 172.21.15.59:0/3408626889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd240cfa50 con 0x7efd34104010 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd32a00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 0x7efd180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 -- 172.21.15.59:0/2390623566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 msgr2=0x563c1a719f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a719f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009f30 tx=0x7efd2c0118d0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd32a00640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 0x7efd180a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd240cfcf0 tx=0x7efd240d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 -- 172.21.15.59:0/2390623566 shutdown_connections 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a727160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a724830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/2390623566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a719f70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.321+0000 7efd3c2c8740 1 -- 172.21.15.59:0/2390623566 >> 172.21.15.59:0/2390623566 conn(0x563c1a716120 msgr2=0x563c1a716500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.322+0000 7efd3c2c8740 1 -- 172.21.15.59:0/2390623566 shutdown_connections 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.322+0000 7efd3c2c8740 1 -- 172.21.15.59:0/2390623566 wait complete. 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.322+0000 7efd3c2c8740 1 -- start start 2024-09-05T12:07:34.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.322+0000 7efd3c2c8740 1 Processor -- start 2024-09-05T12:07:34.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 0x563c1a7221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a724d70 0x563c1a722730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563c1a7279e0 con 0x563c1a719b90 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563c1a63b0f0 con 0x563c1a724d70 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c1a727b60 con 0x563c1a71a4b0 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd3c2c8740 10 client.0 _ll_register_callbacks cb 0x563c1a62c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39978/0 (socket says 172.21.15.59:39978) 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd33400640 1 -- 172.21.15.59:0/1743859787 learned_addr learned my addr 172.21.15.59:0/1743859787 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd2a000640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 0x563c1a7221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd33400640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a724d70 0x563c1a722730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd2a000640 1 -- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a724d70 msgr2=0x563c1a722730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd2a000640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a724d70 0x563c1a722730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd2a000640 1 -- 172.21.15.59:0/1743859787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 msgr2=0x563c1a721cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.323+0000 7efd2a000640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd2a000640 1 -- 172.21.15.59:0/1743859787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1a3f13c0 con 0x563c1a71a4b0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd33400640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd2a000640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 0x563c1a7221f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd1c005940 tx=0x7efd1c0317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e298b765a7cfc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd20c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c035040 con 0x563c1a71a4b0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c0044b0 con 0x563c1a71a4b0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.324+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd1c004460 con 0x563c1a71a4b0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd1c04e050 con 0x563c1a71a4b0 2024-09-05T12:07:34.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 10 client.26878 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c1a733730 con 0x563c1a71a4b0 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd2c0044b0 con 0x563c1a71a4b0 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd1c004bb0 con 0x563c1a71a4b0 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 10 client.26878 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 20 client.26878 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a734b40 0x563c1a736f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 4 client.26878 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.325+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c1a68d950 con 0x563c1a734b40 2024-09-05T12:07:34.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.326+0000 7efd33400640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a734b40 0x563c1a736f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.326+0000 7efd33400640 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a734b40 0x563c1a736f30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1e7d2af0df9a16 server_cookie=674818dd3f91f911 in_seq=0 out_seq=0 2024-09-05T12:07:34.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.326+0000 7efd20c00640 10 client.26878 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd20c00640 1 -- 172.21.15.59:0/1743859787 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c1a68d950 con 0x563c1a734b40 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd20c00640 10 client.26878 handle_command_reply: tid=0 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 client.26878 shutdown 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 msgr2=0x563c1a7221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 0x563c1a7221f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd1c005940 tx=0x7efd1c0317e0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 shutdown_connections 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a724d70 0x563c1a722730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a71a4b0 0x563c1a7221f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a734b40 0x563c1a736f30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1743859787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a719b90 0x563c1a721cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 >> 172.21.15.59:0/1743859787 conn(0x563c1a716120 msgr2=0x563c1a71be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.327+0000 7efd20c00640 0 client.26878 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.328+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 shutdown_connections 2024-09-05T12:07:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.328+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1743859787 wait complete. 2024-09-05T12:07:34.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.328+0000 7efd3c2c8740 20 client.26878 trim_cache size 0 max 16384 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.345+0000 7efd3c2c8740 1 Processor -- start 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.345+0000 7efd3c2c8740 1 -- start start 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a719b90 0x563c1a7150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a71a4b0 0x563c1a73c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd341af0a0 con 0x563c1a724d70 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd341b1920 con 0x563c1a71a4b0 2024-09-05T12:07:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd341af710 con 0x563c1a719b90 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39988/0 (socket says 172.21.15.59:39988) 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 -- 172.21.15.59:0/1193704849 learned_addr learned my addr 172.21.15.59:0/1193704849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a719b90 0x563c1a7150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd2a000640 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a71a4b0 0x563c1a73c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 -- 172.21.15.59:0/1193704849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a719b90 msgr2=0x563c1a7150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a719b90 0x563c1a7150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 -- 172.21.15.59:0/1193704849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a71a4b0 msgr2=0x563c1a73c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a71a4b0 0x563c1a73c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.346+0000 7efd33400640 1 -- 172.21.15.59:0/1193704849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1a7243a0 con 0x563c1a724d70 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.347+0000 7efd33400640 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd2c004b50 tx=0x7efd2c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869fdb14b758481 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.347+0000 7efd23400640 1 -- 172.21.15.59:0/1193704849 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c037070 con 0x563c1a724d70 2024-09-05T12:07:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.347+0000 7efd23400640 1 -- 172.21.15.59:0/1193704849 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd2c003de0 con 0x563c1a724d70 2024-09-05T12:07:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.347+0000 7efd23400640 1 -- 172.21.15.59:0/1193704849 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c028c00 con 0x563c1a724d70 2024-09-05T12:07:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1193704849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 msgr2=0x563c1a73cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd2c004b50 tx=0x7efd2c004b80 comp rx=0 tx=0).stop 2024-09-05T12:07:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1193704849 shutdown_connections 2024-09-05T12:07:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a724d70 0x563c1a73cd40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a71a4b0 0x563c1a73c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/1193704849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a719b90 0x563c1a7150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1193704849 >> 172.21.15.59:0/1193704849 conn(0x563c1a716120 msgr2=0x563c1a7149a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1193704849 shutdown_connections 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- 172.21.15.59:0/1193704849 wait complete. 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.348+0000 7efd3c2c8740 1 -- start start 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 Processor -- start 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a728e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a79e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd341afde0 con 0x563c1a71a4b0 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd341b1250 con 0x563c1a719b90 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd341b04b0 con 0x563c1a724d70 2024-09-05T12:07:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd3c2c8740 10 client.0 _ll_register_callbacks cb 0x7efd1c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd2a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd2a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39998/0 (socket says 172.21.15.59:39998) 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd2a000640 1 -- 172.21.15.59:0/860039035 learned_addr learned my addr 172.21.15.59:0/860039035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.349+0000 7efd33400640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a79e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd33400640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a728e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 -- 172.21.15.59:0/860039035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 msgr2=0x563c1a79e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a79e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 -- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 msgr2=0x563c1a728e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a728e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 -- 172.21.15.59:0/860039035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1a729c70 con 0x563c1a71a4b0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd33400640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a728e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd2a000640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efd1c003690 tx=0x7efd1c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99133f0f1902233 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd20c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:07:34.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c04b040 con 0x563c1a71a4b0 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efd1c004fe0 con 0x563c1a71a4b0 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.350+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c003de0 con 0x563c1a71a4b0 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd1c061050 con 0x563c1a71a4b0 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 10 client.16956 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c1a79fe00 con 0x563c1a71a4b0 2024-09-05T12:07:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd2c003de0 con 0x563c1a71a4b0 2024-09-05T12:07:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd1c0046c0 con 0x563c1a71a4b0 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 10 client.16956 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 20 client.16956 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a7a0600 0x563c1a7a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 4 client.16956 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.351+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c1a687e60 con 0x563c1a7a0600 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd33400640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a7a0600 0x563c1a7a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd33400640 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a7a0600 0x563c1a7a2a20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab1e903c1aac431 server_cookie=a2a206a26c3d84d6 in_seq=0 out_seq=0 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd20c00640 10 client.16956 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd20c00640 1 -- 172.21.15.59:0/860039035 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563c1a687e60 con 0x563c1a7a0600 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd20c00640 10 client.16956 handle_command_reply: tid=0 2024-09-05T12:07:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.352+0000 7efd3c2c8740 1 client.16956 shutdown 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 msgr2=0x563c1a729390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efd1c003690 tx=0x7efd1c004470 comp rx=0 tx=0).stop 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 shutdown_connections 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1a724d70 0x563c1a79e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563c1a7a0600 0x563c1a7a2a20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563c1a71a4b0 0x563c1a729390 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 --2- 172.21.15.59:0/860039035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c1a719b90 0x563c1a728e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 >> 172.21.15.59:0/860039035 conn(0x563c1a716120 msgr2=0x563c1a71be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd20c00640 0 client.16956 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 shutdown_connections 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 1 -- 172.21.15.59:0/860039035 wait complete. 2024-09-05T12:07:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.353+0000 7efd3c2c8740 20 client.16956 trim_cache size 0 max 16384 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "740ca461-1ddc-421a-9790-befaf99054f7": { 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "740ca461-1ddc-421a-9790-befaf99054f7", 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:34.359 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:34.360 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:34.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 msgr2=0x7efd180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 0x7efd180a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd240cfcf0 tx=0x7efd240d5530 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 msgr2=0x7efd3419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd2400ebc0 tx=0x7efd2400c6a0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 shutdown_connections 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7efd180a2c00 0x7efd180a50c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd34109140 0x7efd3419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd341049c0 0x7efd3419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 --2- 172.21.15.59:0/3408626889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd34104010 0x7efd3419a640 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 >> 172.21.15.59:0/3408626889 conn(0x7efd340ffa50 msgr2=0x7efd34100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 shutdown_connections 2024-09-05T12:07:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:34.354+0000 7efd39800640 1 -- 172.21.15.59:0/3408626889 wait complete. 2024-09-05T12:07:34.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:34 smithi189 ceph-mon[34772]: pgmap v691: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:34.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:34 smithi059 ceph-mon[30025]: pgmap v691: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:34.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:34 smithi089 ceph-mon[34499]: pgmap v691: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:35.243 INFO:tasks.cephfs.filesystem:scrub status for tag:740ca461-1ddc-421a-9790-befaf99054f7 - {'path': '/', 'tag': '740ca461-1ddc-421a-9790-befaf99054f7', 'options': 'recursive,force'} 2024-09-05T12:07:35.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:35 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:07:35.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:35 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:07:35.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:35 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:07:36.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:36 smithi189 ceph-mon[34772]: pgmap v692: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:36.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:36 smithi089 ceph-mon[34499]: pgmap v692: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:36.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:36 smithi059 ceph-mon[30025]: pgmap v692: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:07:38.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:38 smithi189 ceph-mon[34772]: pgmap v693: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:38.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:38 smithi089 ceph-mon[34499]: pgmap v693: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:38 smithi059 ceph-mon[30025]: pgmap v693: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:313/600: open fd = 3 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:314/600: open fd = 3 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.147 INFO:tasks.workunit.client.0.smithi059.stdout:315/600: open fd = 3 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:316/600: open fd = 3 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:317/600: open fd = 3 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:318/600: open fd = 3 2024-09-05T12:07:39.148 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:319/600: open fd = 3 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:320/600: open fd = 3 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:321/600: open fd = 3 2024-09-05T12:07:39.149 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:322/600: open fd = 3 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:323/600: open fd = 3 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:324/600: open fd = 3 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.150 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:325/600: open fd = 3 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:326/600: open fd = 3 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:327/600: open fd = 3 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:328/600: open fd = 3 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.151 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:329/600: open fd = 3 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:330/600: open fd = 3 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:331/600: open fd = 3 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.152 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:332/600: open fd = 3 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:333/600: open fd = 3 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:334/600: open fd = 3 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.153 INFO:tasks.workunit.client.0.smithi059.stdout:335/600: open fd = 3 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:336/600: open fd = 3 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:337/600: open fd = 3 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:338/600: open fd = 3 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.154 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:339/600: open fd = 3 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:340/600: open fd = 3 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:341/600: open fd = 3 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.155 INFO:tasks.workunit.client.0.smithi059.stdout:342/600: open fd = 3 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:343/600: open fd = 3 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:344/600: open fd = 3 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:345/600: open fd = 3 2024-09-05T12:07:39.156 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:346/600: open fd = 3 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:347/600: open fd = 3 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:348/600: open fd = 3 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.157 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:349/600: open fd = 3 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:350/600: open fd = 3 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:351/600: open fd = 3 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.158 INFO:tasks.workunit.client.0.smithi059.stdout:352/600: open fd = 3 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:353/600: open fd = 3 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:354/600: open fd = 3 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:355/600: open fd = 3 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.159 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:356/600: open fd = 3 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:357/600: open fd = 3 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:358/600: open fd = 3 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.160 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.161 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.161 INFO:tasks.workunit.client.0.smithi059.stdout:359/600: open fd = 3 2024-09-05T12:07:39.161 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.161 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.164 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.164 INFO:tasks.workunit.client.0.smithi059.stdout:360/600: open fd = 3 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:361/600: open fd = 3 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:362/600: open fd = 3 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:363/600: open fd = 3 2024-09-05T12:07:39.165 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:364/600: open fd = 3 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:365/600: open fd = 3 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:366/600: open fd = 3 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.166 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:367/600: open fd = 3 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:368/600: open fd = 3 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:369/600: open fd = 3 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:370/600: open fd = 3 2024-09-05T12:07:39.167 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:371/600: open fd = 3 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:372/600: open fd = 3 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:373/600: open fd = 3 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.168 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:39.169 INFO:tasks.workunit.client.0.smithi059.stdout:374/600: open fd = 3 2024-09-05T12:07:39.169 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.169 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:07:39.169 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:07:40.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:40 smithi089 ceph-mon[34499]: pgmap v694: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:40.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:40 smithi059 ceph-mon[30025]: pgmap v694: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:40 smithi189 ceph-mon[34772]: pgmap v694: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:42.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:42 smithi089 ceph-mon[34499]: pgmap v695: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:42.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:42 smithi059 ceph-mon[30025]: pgmap v695: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:42 smithi189 ceph-mon[34772]: pgmap v695: 129 pgs: 129 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:44.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:44 smithi089 ceph-mon[34499]: pgmap v696: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:44.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:44 smithi059 ceph-mon[30025]: pgmap v696: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:44 smithi189 ceph-mon[34772]: pgmap v696: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:46.334 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.3 2024-09-05T12:07:46.335 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:07:46.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:46 smithi059 ceph-mon[30025]: pgmap v697: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:46.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:46 smithi089 ceph-mon[34499]: pgmap v697: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:46 smithi189 ceph-mon[34772]: pgmap v697: 129 pgs: 129 active+clean; 5.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.321+0000 7f2a98c00640 1 -- 172.21.15.59:0/3065408507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a941049a0 msgr2=0x7f2a941089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 --2- 172.21.15.59:0/3065408507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a941049a0 0x7f2a941089f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002a410 comp rx=0 tx=0).stop 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 -- 172.21.15.59:0/3065408507 shutdown_connections 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 --2- 172.21.15.59:0/3065408507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 0x7f2a94110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 --2- 172.21.15.59:0/3065408507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a941049a0 0x7f2a941089f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 --2- 172.21.15.59:0/3065408507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a94103ff0 0x7f2a941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 -- 172.21.15.59:0/3065408507 >> 172.21.15.59:0/3065408507 conn(0x7f2a940ffa70 msgr2=0x7f2a94101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 -- 172.21.15.59:0/3065408507 shutdown_connections 2024-09-05T12:07:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.322+0000 7f2a98c00640 1 -- 172.21.15.59:0/3065408507 wait complete. 2024-09-05T12:07:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 Processor -- start 2024-09-05T12:07:47.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 -- start start 2024-09-05T12:07:47.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a941049a0 0x7f2a9419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 0x7f2a9419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94114170 con 0x7f2a94103ff0 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94113ff0 con 0x7f2a94109120 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a98c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941142f0 con 0x7f2a941049a0 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a91e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.323+0000 7f2a91e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47784/0 (socket says 172.21.15.59:47784) 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 -- 172.21.15.59:0/27179118 learned_addr learned my addr 172.21.15.59:0/27179118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a92800640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 0x7f2a9419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91400640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a941049a0 0x7f2a9419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:47.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 -- 172.21.15.59:0/27179118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a941049a0 msgr2=0x7f2a9419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a941049a0 0x7f2a9419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 -- 172.21.15.59:0/27179118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 msgr2=0x7f2a9419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 0x7f2a9419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 -- 172.21.15.59:0/27179118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9419f5a0 con 0x7f2a94103ff0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.324+0000 7f2a91e00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400d950 tx=0x7f2a8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.325+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84017070 con 0x7f2a94103ff0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.325+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84009c30 con 0x7f2a94103ff0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.325+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9419f890 con 0x7f2a94103ff0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.325+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400f6b0 con 0x7f2a94103ff0 2024-09-05T12:07:47.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.325+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941a7170 con 0x7f2a94103ff0 2024-09-05T12:07:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.326+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a941043d0 con 0x7f2a94103ff0 2024-09-05T12:07:47.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.328+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a840109d0 con 0x7f2a94103ff0 2024-09-05T12:07:47.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.329+0000 7f2a8aa00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:47.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.329+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2a840cf090 con 0x7f2a94103ff0 2024-09-05T12:07:47.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.329+0000 7f2a91400640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:47.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.330+0000 7f2a91400640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a80000c00 tx=0x7f2a80005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:47.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.334+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a84009de0 con 0x7f2a94103ff0 2024-09-05T12:07:47.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.553+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9402d7c0 con 0x7f2a94103ff0 2024-09-05T12:07:47.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.554+0000 7f2a8aa00640 1 -- 172.21.15.59:0/27179118 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f2a8409b530 con 0x7f2a94103ff0 2024-09-05T12:07:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:07:47.557 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:07:47.557 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:07:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 msgr2=0x7f2a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a80000c00 tx=0x7f2a80005b50 comp rx=0 tx=0).stop 2024-09-05T12:07:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 msgr2=0x7f2a9419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400d950 tx=0x7f2a8400de10 comp rx=0 tx=0).stop 2024-09-05T12:07:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 shutdown_connections 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a94109120 0x7f2a9419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a941049a0 0x7f2a9419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 --2- 172.21.15.59:0/27179118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a94103ff0 0x7f2a9419a5f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 >> 172.21.15.59:0/27179118 conn(0x7f2a940ffa70 msgr2=0x7f2a94100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 shutdown_connections 2024-09-05T12:07:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:47.557+0000 7f2a98c00640 1 -- 172.21.15.59:0/27179118 wait complete. 2024-09-05T12:07:47.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/27179118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:47.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:47 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/27179118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:48.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:47 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/27179118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:07:48.371 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 139.3 --await-for 45.0' 2024-09-05T12:07:48.371 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 139.3 --await-for 45.0 2024-09-05T12:07:48.761 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:07:48.902 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:48 smithi059 ceph-mon[30025]: pgmap v698: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:48.902 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:48 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:07:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:48 smithi089 ceph-mon[34499]: pgmap v698: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:48.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:48 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:07:49.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:48 smithi189 ceph-mon[34772]: pgmap v698: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:07:49.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:48 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 -- 172.21.15.59:0/1945488752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 msgr2=0x7f2c1c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 --2- 172.21.15.59:0/1945488752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c10dd20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a480 comp rx=0 tx=0).stop 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 -- 172.21.15.59:0/1945488752 shutdown_connections 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 --2- 172.21.15.59:0/1945488752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c10e260 0x7f2c1c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 --2- 172.21.15.59:0/1945488752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c10dd20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 --2- 172.21.15.59:0/1945488752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 -- 172.21.15.59:0/1945488752 >> 172.21.15.59:0/1945488752 conn(0x7f2c1c0fd740 msgr2=0x7f2c1c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 -- 172.21.15.59:0/1945488752 shutdown_connections 2024-09-05T12:07:49.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.380+0000 7f2c22600640 1 -- 172.21.15.59:0/1945488752 wait complete. 2024-09-05T12:07:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.381+0000 7f2c22600640 1 Processor -- start 2024-09-05T12:07:49.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.381+0000 7f2c22600640 1 -- start start 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c10e260 0x7f2c1c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c114000 con 0x7f2c1c074ee0 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c113e80 con 0x7f2c1c10e260 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c22600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c114180 con 0x7f2c1c0745c0 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52894/0 (socket says 172.21.15.59:52894) 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 -- 172.21.15.59:0/4230788781 learned_addr learned my addr 172.21.15.59:0/4230788781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c1be00640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21c00640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c10e260 0x7f2c1c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 -- 172.21.15.59:0/4230788781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c10e260 msgr2=0x7f2c1c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c10e260 0x7f2c1c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 -- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 msgr2=0x7f2c1c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.382+0000 7f2c21200640 1 -- 172.21.15.59:0/4230788781 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c19f470 con 0x7f2c1c0745c0 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c21200640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000d950 tx=0x7f2c1000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c19600640 1 -- 172.21.15.59:0/4230788781 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10017070 con 0x7f2c1c0745c0 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c19600640 1 -- 172.21.15.59:0/4230788781 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10009c30 con 0x7f2c1c0745c0 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c19f760 con 0x7f2c1c0745c0 2024-09-05T12:07:49.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c19600640 1 -- 172.21.15.59:0/4230788781 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000f6b0 con 0x7f2c1c0745c0 2024-09-05T12:07:49.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.383+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1a7040 con 0x7f2c1c0745c0 2024-09-05T12:07:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.386+0000 7f2c19600640 1 -- 172.21.15.59:0/4230788781 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c100109d0 con 0x7f2c1c0745c0 2024-09-05T12:07:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.386+0000 7f2c2662b740 1 Processor -- start 2024-09-05T12:07:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 -- start start 2024-09-05T12:07:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f21cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56050f175530 con 0x56050f20f560 2024-09-05T12:07:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56050f11c860 con 0x56050f20fe80 2024-09-05T12:07:49.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c2662b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56050f1714f0 con 0x56050f21a740 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.387+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47804/0 (socket says 172.21.15.59:47804) 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 -- 172.21.15.59:0/2003362348 learned_addr learned my addr 172.21.15.59:0/2003362348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21200640 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c1be00640 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f21cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 -- 172.21.15.59:0/2003362348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 msgr2=0x56050f21cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f21cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 -- 172.21.15.59:0/2003362348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 msgr2=0x56050f21a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c21c00640 1 -- 172.21.15.59:0/2003362348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050eee6b90 con 0x56050f20f560 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.388+0000 7f2c19600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 0x7f2c040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009f00 tx=0x7f2c140107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f351e81342bc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c19600640 1 -- 172.21.15.59:0/4230788781 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2c100cf140 con 0x7f2c1c0745c0 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c21200640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 0x7f2c040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c0e000640 1 -- 172.21.15.59:0/2003362348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14003b70 con 0x56050f20f560 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c0e000640 1 -- 172.21.15.59:0/2003362348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14003d10 con 0x56050f20f560 2024-09-05T12:07:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.389+0000 7f2c0e000640 1 -- 172.21.15.59:0/2003362348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14011a40 con 0x56050f20f560 2024-09-05T12:07:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 -- 172.21.15.59:0/2003362348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 msgr2=0x56050f20f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c21200640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 0x7f2c040a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c10097f00 tx=0x7f2c100d2b80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009f00 tx=0x7f2c140107a0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 -- 172.21.15.59:0/2003362348 shutdown_connections 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f21cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2003362348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f20f940 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 -- 172.21.15.59:0/2003362348 >> 172.21.15.59:0/2003362348 conn(0x56050f20baf0 msgr2=0x56050f20bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 -- 172.21.15.59:0/2003362348 shutdown_connections 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.390+0000 7f2c2662b740 1 -- 172.21.15.59:0/2003362348 wait complete. 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.391+0000 7f2c2662b740 1 -- start start 2024-09-05T12:07:49.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.391+0000 7f2c2662b740 1 Processor -- start 2024-09-05T12:07:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f217670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f217bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f2180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56050f21d310 con 0x56050f20f560 2024-09-05T12:07:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56050f12fe60 con 0x56050f21a740 2024-09-05T12:07:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56050f21d490 con 0x56050f20fe80 2024-09-05T12:07:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c2662b740 10 client.0 _ll_register_callbacks cb 0x56050f134cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:49.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f217670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f217670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47818/0 (socket says 172.21.15.59:47818) 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c21c00640 1 -- 172.21.15.59:0/310743093 learned_addr learned my addr 172.21.15.59:0/310743093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.392+0000 7f2c1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f217bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c21c00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f2180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 -- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 msgr2=0x56050f2180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f2180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 -- 172.21.15.59:0/310743093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 msgr2=0x56050f217670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f217670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 -- 172.21.15.59:0/310743093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050f13b990 con 0x56050f20fe80 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c1be00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f217bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c0802aa70 tx=0x7f2c08004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b42b9c3067bfae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c02a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08035040 con 0x56050f20fe80 2024-09-05T12:07:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.393+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14003d10 con 0x56050f20fe80 2024-09-05T12:07:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2c08005450 con 0x56050f20fe80 2024-09-05T12:07:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c0804f050 con 0x56050f20fe80 2024-09-05T12:07:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 10 client.26902 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56050f228fa0 con 0x56050f20fe80 2024-09-05T12:07:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14003d10 con 0x56050f20fe80 2024-09-05T12:07:49.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2c08003a30 con 0x56050f20fe80 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 10 client.26902 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 10 client.26902 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 20 client.26902 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.394+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f22a3b0 0x56050f22c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.395+0000 7f2c2662b740 4 client.26902 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.395+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050f118310 con 0x56050f22a3b0 2024-09-05T12:07:49.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.395+0000 7f2c21c00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f22a3b0 0x56050f22c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.395+0000 7f2c21c00640 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f22a3b0 0x56050f22c7a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b316fe5408cd452c server_cookie=735c0ef2f44469d1 in_seq=0 out_seq=0 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.395+0000 7f2c02a00640 10 client.26902 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.396+0000 7f2c02a00640 1 -- 172.21.15.59:0/310743093 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56050f118310 con 0x56050f22a3b0 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.396+0000 7f2c02a00640 10 client.26902 handle_command_reply: tid=0 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.396+0000 7f2c2662b740 1 client.26902 shutdown 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 msgr2=0x56050f217bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f217bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c0802aa70 tx=0x7f2c08004440 comp rx=0 tx=0).stop 2024-09-05T12:07:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 shutdown_connections 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f2180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f217bb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f22a3b0 0x56050f22c7a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 --2- 172.21.15.59:0/310743093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f217670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 >> 172.21.15.59:0/310743093 conn(0x56050f20baf0 msgr2=0x56050f211820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c02a00640 0 client.26902 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 shutdown_connections 2024-09-05T12:07:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 1 -- 172.21.15.59:0/310743093 wait complete. 2024-09-05T12:07:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.397+0000 7f2c2662b740 20 client.26902 trim_cache size 0 max 16384 2024-09-05T12:07:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 Processor -- start 2024-09-05T12:07:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 -- start start 2024-09-05T12:07:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f20ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f232000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1aee90 con 0x56050f20f560 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b1710 con 0x56050f21a740 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c2662b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1af500 con 0x56050f20fe80 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.429+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47826/0 (socket says 172.21.15.59:47826) 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 -- 172.21.15.59:0/226985322 learned_addr learned my addr 172.21.15.59:0/226985322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c1be00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f20ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f232000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 -- 172.21.15.59:0/226985322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 msgr2=0x56050f20ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f20ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 -- 172.21.15.59:0/226985322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 msgr2=0x56050f232000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f232000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 -- 172.21.15.59:0/226985322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050f20b920 con 0x56050f20f560 2024-09-05T12:07:49.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c1be00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f20ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:07:49.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.430+0000 7f2c21c00640 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c140107e0 tx=0x7f2c14011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd4438ee9ef7b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c0d600640 1 -- 172.21.15.59:0/226985322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14032070 con 0x56050f20f560 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c0d600640 1 -- 172.21.15.59:0/226985322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14027040 con 0x56050f20f560 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c0d600640 1 -- 172.21.15.59:0/226985322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14026040 con 0x56050f20f560 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 -- 172.21.15.59:0/226985322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 msgr2=0x56050f211fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c140107e0 tx=0x7f2c14011440 comp rx=0 tx=0).stop 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 -- 172.21.15.59:0/226985322 shutdown_connections 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f21a740 0x56050f232000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f20fe80 0x56050f20ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 --2- 172.21.15.59:0/226985322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f211fc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.431+0000 7f2c2662b740 1 -- 172.21.15.59:0/226985322 >> 172.21.15.59:0/226985322 conn(0x56050f20baf0 msgr2=0x56050f20a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.432+0000 7f2c2662b740 1 -- 172.21.15.59:0/226985322 shutdown_connections 2024-09-05T12:07:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.432+0000 7f2c2662b740 1 -- 172.21.15.59:0/226985322 wait complete. 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.432+0000 7f2c2662b740 1 -- start start 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.432+0000 7f2c2662b740 1 Processor -- start 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f21f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f28cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1afbd0 con 0x56050f20f560 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b1040 con 0x56050f20fe80 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b02a0 con 0x56050f21a740 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c2662b740 10 client.0 _ll_register_callbacks cb 0x7f2c08009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f21f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c1be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c1be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40454/0 (socket says 172.21.15.59:40454) 2024-09-05T12:07:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c1be00640 1 -- 172.21.15.59:0/2314007827 learned_addr learned my addr 172.21.15.59:0/2314007827 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.433+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f28cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 -- 172.21.15.59:0/2314007827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 msgr2=0x56050f28cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f28cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 -- 172.21.15.59:0/2314007827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 msgr2=0x56050f21f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f21f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 -- 172.21.15.59:0/2314007827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050f28d7c0 con 0x56050f20fe80 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f21f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c1be00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c080041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d378f9460b6c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c02a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08035070 con 0x56050f20fe80 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2c0804c040 con 0x56050f20fe80 2024-09-05T12:07:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14027040 con 0x56050f20fe80 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c08061050 con 0x56050f20fe80 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.434+0000 7f2c2662b740 10 client.26975 fetch_fsmap learned FSMap version 21 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56050f295ab0 con 0x56050f20fe80 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c14027040 con 0x56050f20fe80 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2c08003710 con 0x56050f20fe80 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 10 client.26975 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 10 client.26975 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 20 client.26975 populate_metadata read hostname 'smithi059' 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f296290 0x56050f298680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 4 client.26975 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.3, "await_for": 45.0}] 2024-09-05T12:07:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.3, "await_for": 45.0}) -- 0x56050f213290 con 0x56050f296290 2024-09-05T12:07:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.435+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f296290 0x56050f298680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:07:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.436+0000 7f2c21c00640 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f296290 0x56050f298680 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee66bb0b7b7ca0c server_cookie=bbbeb4b15b99dd94 in_seq=0 out_seq=0 2024-09-05T12:07:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.436+0000 7f2c02a00640 10 client.26975 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c02a00640 1 -- 172.21.15.59:0/2314007827 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56050f213290 con 0x56050f296290 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c02a00640 10 client.26975 handle_command_reply: tid=0 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 client.26975 shutdown 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 msgr2=0x56050f21f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c080041a0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 shutdown_connections 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56050f21a740 0x56050f28cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56050f20fe80 0x56050f21f550 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56050f296290 0x56050f298680 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 --2- 172.21.15.59:0/2314007827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56050f20f560 0x56050f21f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.445+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 >> 172.21.15.59:0/2314007827 conn(0x56050f20baf0 msgr2=0x56050f20ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.446+0000 7f2c02a00640 0 client.26975 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:07:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.446+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 shutdown_connections 2024-09-05T12:07:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.446+0000 7f2c2662b740 1 -- 172.21.15.59:0/2314007827 wait complete. 2024-09-05T12:07:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.446+0000 7f2c2662b740 20 client.26975 trim_cache size 0 max 16384 2024-09-05T12:07:49.449 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:07:49.449 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:07:49.449 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 34, 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "71976967": { 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 34, 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:07:49.450 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 139.3, 2024-09-05T12:07:49.451 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:49.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 msgr2=0x7f2c040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 0x7f2c040a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2c10097f00 tx=0x7f2c100d2b80 comp rx=0 tx=0).stop 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 msgr2=0x7f2c1c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000d950 tx=0x7f2c1000de10 comp rx=0 tx=0).stop 2024-09-05T12:07:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 shutdown_connections 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c040a2b80 0x7f2c040a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c1c10e260 0x7f2c1c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c1c074ee0 0x7f2c1c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 --2- 172.21.15.59:0/4230788781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c1c0745c0 0x7f2c1c19a4c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 >> 172.21.15.59:0/4230788781 conn(0x7f2c1c0fd740 msgr2=0x7f2c1c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 shutdown_connections 2024-09-05T12:07:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:07:49.447+0000 7f2c22600640 1 -- 172.21.15.59:0/4230788781 wait complete. 2024-09-05T12:07:50.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:07:50.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:07:50.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:49 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:07:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:07:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:49 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.211 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '71976967', quiesce took 3.9 seconds. Will release after: 15.4 2024-09-05T12:07:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:07:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:07:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:49 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:07:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:50 smithi089 ceph-mon[34499]: pgmap v699: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:50 smithi059 ceph-mon[30025]: pgmap v699: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:50 smithi189 ceph-mon[34772]: pgmap v699: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:52 smithi089 ceph-mon[34499]: pgmap v700: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:53.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:52 smithi059 ceph-mon[30025]: pgmap v700: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:52 smithi189 ceph-mon[34772]: pgmap v700: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:07:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:54 smithi089 ceph-mon[34499]: pgmap v701: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:07:55.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:54 smithi059 ceph-mon[30025]: pgmap v701: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:07:55.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:54 smithi189 ceph-mon[34772]: pgmap v701: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:07:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:56 smithi089 ceph-mon[34499]: pgmap v702: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:07:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:56 smithi059 ceph-mon[30025]: pgmap v702: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:07:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:56 smithi189 ceph-mon[34772]: pgmap v702: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:07:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:07:58 smithi089 ceph-mon[34499]: pgmap v703: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:07:58 smithi059 ceph-mon[30025]: pgmap v703: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:07:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:07:58 smithi189 ceph-mon[34772]: pgmap v703: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:00 smithi089 ceph-mon[34499]: pgmap v704: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:08:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:00 smithi059 ceph-mon[30025]: pgmap v704: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:08:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:00 smithi189 ceph-mon[34772]: pgmap v704: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-09-05T12:08:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:02 smithi089 ceph-mon[34499]: pgmap v705: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:03.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:02 smithi059 ceph-mon[30025]: pgmap v705: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:02 smithi189 ceph-mon[34772]: pgmap v705: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:04 smithi089 ceph-mon[34499]: pgmap v706: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:04 smithi059 ceph-mon[30025]: pgmap v706: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:05.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:08:05.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:04 smithi189 ceph-mon[34772]: pgmap v706: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:08:05.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:08:05.636 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:06.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:05 smithi059 ceph-mon[30025]: pgmap v707: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-05T12:08:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 -- 172.21.15.59:0/2541920642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 msgr2=0x7fa538108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 --2- 172.21.15.59:0/2541920642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa538108960 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa520009900 tx=0x7fa52002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 -- 172.21.15.59:0/2541920642 shutdown_connections 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 --2- 172.21.15.59:0/2541920642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 0x7fa538110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 --2- 172.21.15.59:0/2541920642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa538108960 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 --2- 172.21.15.59:0/2541920642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa538104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 -- 172.21.15.59:0/2541920642 >> 172.21.15.59:0/2541920642 conn(0x7fa5380ff8b0 msgr2=0x7fa538101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 -- 172.21.15.59:0/2541920642 shutdown_connections 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.179+0000 7fa53dc00640 1 -- 172.21.15.59:0/2541920642 wait complete. 2024-09-05T12:08:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.180+0000 7fa53dc00640 1 Processor -- start 2024-09-05T12:08:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 -- start start 2024-09-05T12:08:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa53819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa53819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 0x7fa53819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa538114090 con 0x7fa538104800 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa538113f10 con 0x7fa538109090 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa53dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa538114210 con 0x7fa538103e50 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.181+0000 7fa52f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa53819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa52f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa53819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56110/0 (socket says 172.21.15.59:56110) 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa52f400640 1 -- 172.21.15.59:0/769949946 learned_addr learned my addr 172.21.15.59:0/769949946 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa53819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa53d200640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 0x7fa53819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 -- 172.21.15.59:0/769949946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 msgr2=0x7fa53819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 0x7fa53819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 -- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 msgr2=0x7fa53819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa53819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.182+0000 7fa537e00640 1 -- 172.21.15.59:0/769949946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53819f520 con 0x7fa538103e50 2024-09-05T12:08:06.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa537e00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa53819a570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa535600640 1 -- 172.21.15.59:0/769949946 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52800ccc0 con 0x7fa538103e50 2024-09-05T12:08:06.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa535600640 1 -- 172.21.15.59:0/769949946 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa528010910 con 0x7fa538103e50 2024-09-05T12:08:06.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53819f810 con 0x7fa538103e50 2024-09-05T12:08:06.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa535600640 1 -- 172.21.15.59:0/769949946 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528018960 con 0x7fa538103e50 2024-09-05T12:08:06.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.183+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a70f0 con 0x7fa538103e50 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.186+0000 7fa535600640 1 -- 172.21.15.59:0/769949946 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa528010ac0 con 0x7fa538103e50 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.186+0000 7fa541e48740 1 Processor -- start 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 -- start start 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c47a5093e0 con 0x55c47a5adf30 2024-09-05T12:08:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c47a4b0d80 con 0x55c47a5a2d50 2024-09-05T12:08:06.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.187+0000 7fa541e48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c47a5053a0 con 0x55c47a5a3670 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40790/0 (socket says 172.21.15.59:40790) 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa537e00640 1 -- 172.21.15.59:0/3368687327 learned_addr learned my addr 172.21.15.59:0/3368687327 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa52f400640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa535600640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 -- 172.21.15.59:0/3368687327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 msgr2=0x55c47a5ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 -- 172.21.15.59:0/3368687327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 msgr2=0x55c47a5b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa53d200640 1 -- 172.21.15.59:0/3368687327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47a4cfae0 con 0x55c47a5a2d50 2024-09-05T12:08:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa537e00640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.188+0000 7fa535600640 1 -- 172.21.15.59:0/769949946 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa5280cf7a0 con 0x7fa538103e50 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.189+0000 7fa53d200640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.189+0000 7fa53d200640 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5a3130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa530009f00 tx=0x7fa530011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550891ca771621db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.189+0000 7fa52d600640 1 -- 172.21.15.59:0/3368687327 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530004050 con 0x55c47a5a2d50 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.189+0000 7fa52d600640 1 -- 172.21.15.59:0/3368687327 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530003e50 con 0x55c47a5a2d50 2024-09-05T12:08:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.189+0000 7fa52d600640 1 -- 172.21.15.59:0/3368687327 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53000e3b0 con 0x55c47a5a2d50 2024-09-05T12:08:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 -- 172.21.15.59:0/3368687327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 msgr2=0x55c47a5a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa53d200640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa53001d000 tx=0x7fa53000e6e0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5a3130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa530009f00 tx=0x7fa530011810 comp rx=0 tx=0).stop 2024-09-05T12:08:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 -- 172.21.15.59:0/3368687327 shutdown_connections 2024-09-05T12:08:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 --2- 172.21.15.59:0/3368687327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5a3130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 -- 172.21.15.59:0/3368687327 >> 172.21.15.59:0/3368687327 conn(0x55c47a59f2e0 msgr2=0x55c47a59f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 -- 172.21.15.59:0/3368687327 shutdown_connections 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.190+0000 7fa541e48740 1 -- 172.21.15.59:0/3368687327 wait complete. 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.191+0000 7fa541e48740 1 -- start start 2024-09-05T12:08:06.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.191+0000 7fa541e48740 1 Processor -- start 2024-09-05T12:08:06.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c47a5b0ba0 con 0x55c47a5adf30 2024-09-05T12:08:06.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c47a4c40f0 con 0x55c47a5a2d50 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c47a5b0d20 con 0x55c47a5a3670 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa541e48740 10 client.0 _ll_register_callbacks cb 0x55c47a4b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.192+0000 7fa537e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50284/0 (socket says 172.21.15.59:50284) 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 -- 172.21.15.59:0/2152798897 learned_addr learned my addr 172.21.15.59:0/2152798897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa52f400640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa53d200640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 -- 172.21.15.59:0/2152798897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 msgr2=0x55c47a5ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 -- 172.21.15.59:0/2152798897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 msgr2=0x55c47a5ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 -- 172.21.15.59:0/2152798897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47a27a3c0 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa53d200640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa537e00640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5280a42d0 tx=0x7fa52809bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20054791f526c8b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa526000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.193+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52809bc40 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa5280cf3f0 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa530003e50 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa52809ff40 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa541e48740 10 client.26987 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c47a5c50a0 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530003e50 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa5280ce030 con 0x55c47a5a2d50 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa541e48740 10 client.26987 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.194+0000 7fa541e48740 10 client.26987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa541e48740 20 client.26987 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a5c6560 0x55c47a5c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa541e48740 4 client.26987 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c47a516950 con 0x55c47a5c6560 2024-09-05T12:08:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa52f400640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a5c6560 0x55c47a5c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa52f400640 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a5c6560 0x55c47a5c8990 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a758d9bad8aadc server_cookie=4285bd91aaab8b64 in_seq=0 out_seq=0 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.195+0000 7fa526000640 10 client.26987 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.196+0000 7fa526000640 1 -- 172.21.15.59:0/2152798897 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c47a516950 con 0x55c47a5c6560 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.196+0000 7fa526000640 10 client.26987 handle_command_reply: tid=0 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 client.26987 shutdown 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 msgr2=0x55c47a5aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5280a42d0 tx=0x7fa52809bad0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 shutdown_connections 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a5c6560 0x55c47a5c8990 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5adf30 0x55c47a5ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 --2- 172.21.15.59:0/2152798897 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5a2d50 0x55c47a5aae20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 >> 172.21.15.59:0/2152798897 conn(0x55c47a59f2e0 msgr2=0x55c47a5a5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa526000640 0 client.26987 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 shutdown_connections 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 1 -- 172.21.15.59:0/2152798897 wait complete. 2024-09-05T12:08:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.197+0000 7fa541e48740 20 client.26987 trim_cache size 0 max 16384 2024-09-05T12:08:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.216+0000 7fa541e48740 1 Processor -- start 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.216+0000 7fa541e48740 1 -- start start 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 0x55c47a5ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a5aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381aab30 con 0x55c47a5a3670 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ad3b0 con 0x55c47a5adf30 2024-09-05T12:08:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa541e48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ab1a0 con 0x55c47a5a2d50 2024-09-05T12:08:06.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40820/0 (socket says 172.21.15.59:40820) 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa53d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a5aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 -- 172.21.15.59:0/1310657903 learned_addr learned my addr 172.21.15.59:0/1310657903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa52f400640 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 0x55c47a5ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 -- 172.21.15.59:0/1310657903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 msgr2=0x55c47a5aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a5aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 -- 172.21.15.59:0/1310657903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 msgr2=0x55c47a5ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 0x55c47a5ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa537e00640 1 -- 172.21.15.59:0/1310657903 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47a59f110 con 0x55c47a5a2d50 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.217+0000 7fa52f400640 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 0x55c47a5ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.218+0000 7fa537e00640 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa528000c00 tx=0x7fa52809bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a81e30e80fe01 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.218+0000 7fa52cc00640 1 -- 172.21.15.59:0/1310657903 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280b8070 con 0x55c47a5a2d50 2024-09-05T12:08:06.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.218+0000 7fa52cc00640 1 -- 172.21.15.59:0/1310657903 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5280a4800 con 0x55c47a5a2d50 2024-09-05T12:08:06.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.218+0000 7fa52cc00640 1 -- 172.21.15.59:0/1310657903 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528097990 con 0x55c47a5a2d50 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 -- 172.21.15.59:0/1310657903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 msgr2=0x55c47a5a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa528000c00 tx=0x7fa52809bb40 comp rx=0 tx=0).stop 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 -- 172.21.15.59:0/1310657903 shutdown_connections 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a5aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a3670 0x55c47a5ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 --2- 172.21.15.59:0/1310657903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a2d50 0x55c47a5a5800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 -- 172.21.15.59:0/1310657903 >> 172.21.15.59:0/1310657903 conn(0x55c47a59f2e0 msgr2=0x55c47a5aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 -- 172.21.15.59:0/1310657903 shutdown_connections 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.219+0000 7fa541e48740 1 -- 172.21.15.59:0/1310657903 wait complete. 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 -- start start 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 Processor -- start 2024-09-05T12:08:06.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a630230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ab870 con 0x55c47a5a2d50 2024-09-05T12:08:06.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381acce0 con 0x55c47a5adf30 2024-09-05T12:08:06.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.220+0000 7fa541e48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381abf40 con 0x55c47a5a3670 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa541e48740 10 client.0 _ll_register_callbacks cb 0x7fa528010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa537e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40824/0 (socket says 172.21.15.59:40824) 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 -- 172.21.15.59:0/3757455962 learned_addr learned my addr 172.21.15.59:0/3757455962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa537e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56152/0 (socket says 172.21.15.59:56152) 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa53d200640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a630230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 -- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 msgr2=0x55c47a630230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a630230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 -- 172.21.15.59:0/3757455962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 msgr2=0x55c47a5ba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa52f400640 1 -- 172.21.15.59:0/3757455962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47a630770 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.221+0000 7fa537e00640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa52f400640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa520031550 tx=0x7fa520009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9ac32dd49f648e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa526000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520035040 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa520005040 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5280a4800 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa520053050 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa541e48740 10 client.26926 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.222+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c47a6319b0 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5280a4800 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa5200035f0 con 0x55c47a5a3670 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 10 client.26926 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:06.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 10 client.26926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:06.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 20 client.26926 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:06.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a6321b0 0x55c47a6345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:06.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 4 client.26926 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:08:06.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c47a510e60 con 0x55c47a6321b0 2024-09-05T12:08:06.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.223+0000 7fa537e00640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a6321b0 0x55c47a6345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:06.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.224+0000 7fa537e00640 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a6321b0 0x55c47a6345d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cce883083893a8 server_cookie=c5710f396d7c2e23 in_seq=0 out_seq=0 2024-09-05T12:08:06.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.224+0000 7fa526000640 10 client.26926 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:06.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.224+0000 7fa526000640 1 -- 172.21.15.59:0/3757455962 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c47a510e60 con 0x55c47a6321b0 2024-09-05T12:08:06.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.224+0000 7fa526000640 10 client.26926 handle_command_reply: tid=0 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.224+0000 7fa541e48740 1 client.26926 shutdown 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 msgr2=0x55c47a5baea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa520031550 tx=0x7fa520009580 comp rx=0 tx=0).stop 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 shutdown_connections 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c47a5adf30 0x55c47a630230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c47a5a3670 0x55c47a5baea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c47a6321b0 0x55c47a6345d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 --2- 172.21.15.59:0/3757455962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c47a5a2d50 0x55c47a5ba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa526000640 0 client.26926 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 >> 172.21.15.59:0/3757455962 conn(0x55c47a59f2e0 msgr2=0x55c47a5c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 shutdown_connections 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 1 -- 172.21.15.59:0/3757455962 wait complete. 2024-09-05T12:08:06.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.225+0000 7fa541e48740 20 client.26926 trim_cache size 0 max 16384 2024-09-05T12:08:06.230 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:06.230 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:08:06.230 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:08:06.230 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 msgr2=0x7fa51c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa53001d000 tx=0x7fa53000e6e0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 msgr2=0x7fa53819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa53819a570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 shutdown_connections 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa538109090 0x7fa53819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa538104800 0x7fa53819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 --2- 172.21.15.59:0/769949946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538103e50 0x7fa53819a570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:06.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 >> 172.21.15.59:0/769949946 conn(0x7fa5380ff8b0 msgr2=0x7fa538100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:06.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.226+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 shutdown_connections 2024-09-05T12:08:06.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:06.227+0000 7fa53dc00640 1 -- 172.21.15.59:0/769949946 wait complete. 2024-09-05T12:08:06.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:05 smithi189 ceph-mon[34772]: pgmap v707: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-05T12:08:06.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:05 smithi089 ceph-mon[34499]: pgmap v707: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-05T12:08:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.138+0000 7f5f62a00640 1 -- 172.21.15.59:0/3901738975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 msgr2=0x7f5f5c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.138+0000 7f5f62a00640 1 --2- 172.21.15.59:0/3901738975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c1043d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009900 tx=0x7f5f4c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 -- 172.21.15.59:0/3901738975 shutdown_connections 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 --2- 172.21.15.59:0/3901738975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c109120 0x7f5f5c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 --2- 172.21.15.59:0/3901738975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 --2- 172.21.15.59:0/3901738975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c1043d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 -- 172.21.15.59:0/3901738975 >> 172.21.15.59:0/3901738975 conn(0x7f5f5c0ffa90 msgr2=0x7f5f5c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 -- 172.21.15.59:0/3901738975 shutdown_connections 2024-09-05T12:08:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.139+0000 7f5f62a00640 1 -- 172.21.15.59:0/3901738975 wait complete. 2024-09-05T12:08:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 Processor -- start 2024-09-05T12:08:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 -- start start 2024-09-05T12:08:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c109120 0x7f5f5c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1141a0 con 0x7f5f5c103ff0 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c114020 con 0x7f5f5c109120 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f62a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c114320 con 0x7f5f5c1049a0 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f5be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f5be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56156/0 (socket says 172.21.15.59:56156) 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f53e00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.140+0000 7f5f5be00640 1 -- 172.21.15.59:0/2654522876 learned_addr learned my addr 172.21.15.59:0/2654522876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f60a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c109120 0x7f5f5c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 -- 172.21.15.59:0/2654522876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c109120 msgr2=0x7f5f5c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c109120 0x7f5f5c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 -- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 msgr2=0x7f5f5c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 -- 172.21.15.59:0/2654522876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c19f5f0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.141+0000 7f5f53e00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c19ab80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e9f0 tx=0x7f5f4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.142+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400cd80 con 0x7f5f5c1049a0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.142+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c19f8e0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.142+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f440035f0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.142+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44010670 con 0x7f5f5c1049a0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.142+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a71c0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.143+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c1043d0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.145+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f440037a0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.146+0000 7f5f59600640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 0x7f5f400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.146+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5f440cf770 con 0x7f5f5c1049a0 2024-09-05T12:08:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.147+0000 7f5f5be00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 0x7f5f400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.147+0000 7f5f5be00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 0x7f5f400a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0096c0 tx=0x7f5f4c0046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:07.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.149+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f4409c260 con 0x7f5f5c1049a0 2024-09-05T12:08:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.373+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f5c19b9a0 con 0x7f5f5c1049a0 2024-09-05T12:08:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.374+0000 7f5f59600640 1 -- 172.21.15.59:0/2654522876 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f5f4409bb60 con 0x7f5f5c1049a0 2024-09-05T12:08:07.376 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:08:07.376 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:08:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:08:07.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 msgr2=0x7f5f400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:07.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 0x7f5f400a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0096c0 tx=0x7f5f4c0046c0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 msgr2=0x7f5f5c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c19ab80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e9f0 tx=0x7f5f4400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 shutdown_connections 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f400a2b80 0x7f5f400a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f5c109120 0x7f5f5c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f5c1049a0 0x7f5f5c19ab80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 --2- 172.21.15.59:0/2654522876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c103ff0 0x7f5f5c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 >> 172.21.15.59:0/2654522876 conn(0x7f5f5c0ffa90 msgr2=0x7f5f5c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 shutdown_connections 2024-09-05T12:08:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:07.377+0000 7f5f62a00640 1 -- 172.21.15.59:0/2654522876 wait complete. 2024-09-05T12:08:07.641 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:08:07.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:08:08.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:08 smithi059 ceph-mon[30025]: pgmap v708: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:08:08.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2654522876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:08.424 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 71976967 --release --await' 2024-09-05T12:08:08.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 71976967 --release --await 2024-09-05T12:08:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:08 smithi089 ceph-mon[34499]: pgmap v708: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:08:08.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:08 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2654522876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:08.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:08 smithi189 ceph-mon[34772]: pgmap v708: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:08:08.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:08 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2654522876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 -- 172.21.15.59:0/1662923165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56901049a0 msgr2=0x7f56901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 --2- 172.21.15.59:0/1662923165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56901049a0 0x7f56901089f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f567c009900 tx=0x7f567c02a410 comp rx=0 tx=0).stop 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 -- 172.21.15.59:0/1662923165 shutdown_connections 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 --2- 172.21.15.59:0/1662923165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 0x7f5690110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 --2- 172.21.15.59:0/1662923165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56901049a0 0x7f56901089f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 --2- 172.21.15.59:0/1662923165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5690103ff0 0x7f56901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 -- 172.21.15.59:0/1662923165 >> 172.21.15.59:0/1662923165 conn(0x7f56900ffa90 msgr2=0x7f5690101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 -- 172.21.15.59:0/1662923165 shutdown_connections 2024-09-05T12:08:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.094+0000 7f5697e00640 1 -- 172.21.15.59:0/1662923165 wait complete. 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 Processor -- start 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 -- start start 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 0x7f569019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 0x7f569019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 0x7f569019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5690114170 con 0x7f5690103ff0 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5690113ff0 con 0x7f56901049a0 2024-09-05T12:08:09.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.095+0000 7f5697e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56901142f0 con 0x7f5690109120 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 0x7f569019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 0x7f569019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 0x7f569019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50342/0 (socket says 172.21.15.59:50342) 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5694a00640 1 -- 172.21.15.59:0/3552458271 learned_addr learned my addr 172.21.15.59:0/3552458271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 0x7f569019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 -- 172.21.15.59:0/3552458271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 msgr2=0x7f569019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 0x7f569019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 -- 172.21.15.59:0/3552458271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 msgr2=0x7f569019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 0x7f569019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.096+0000 7f5695400640 1 -- 172.21.15.59:0/3552458271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569019f600 con 0x7f5690103ff0 2024-09-05T12:08:09.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f5695400640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 0x7f569019a650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f568400e9f0 tx=0x7f568400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400ccc0 con 0x7f5690103ff0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684010910 con 0x7f5690103ff0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684018960 con 0x7f5690103ff0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569019f8f0 con 0x7f5690103ff0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.097+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a71d0 con 0x7f5690103ff0 2024-09-05T12:08:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.098+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56901043d0 con 0x7f5690103ff0 2024-09-05T12:08:09.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.100+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5684010ac0 con 0x7f5690103ff0 2024-09-05T12:08:09.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.102+0000 7f568e000640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.102+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f56840cf8f0 con 0x7f5690103ff0 2024-09-05T12:08:09.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.102+0000 7f5694a00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.103+0000 7f5694a00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 0x7f56780a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f567c0096c0 tx=0x7f567c0046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.106+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f568409c3e0 con 0x7f5690103ff0 2024-09-05T12:08:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.346+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f569010c390 con 0x7f5690103ff0 2024-09-05T12:08:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.347+0000 7f568e000640 1 -- 172.21.15.59:0/3552458271 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f568409bce0 con 0x7f5690103ff0 2024-09-05T12:08:09.349 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:08:09.349 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:08:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:08:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 msgr2=0x7f56780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 0x7f56780a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f567c0096c0 tx=0x7f567c0046c0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 msgr2=0x7f569019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 0x7f569019a650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f568400e9f0 tx=0x7f568400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 shutdown_connections 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5690109120 0x7f569019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56901049a0 0x7f569019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 --2- 172.21.15.59:0/3552458271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5690103ff0 0x7f569019a650 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 >> 172.21.15.59:0/3552458271 conn(0x7f56900ffa90 msgr2=0x7f5690100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 shutdown_connections 2024-09-05T12:08:09.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.350+0000 7f5697e00640 1 -- 172.21.15.59:0/3552458271 wait complete. 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 -- 172.21.15.59:0/107696815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4109120 msgr2=0x7f6fb4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 --2- 172.21.15.59:0/107696815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4109120 0x7f6fb4110ca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009ca0 tx=0x7f6fac02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 -- 172.21.15.59:0/107696815 shutdown_connections 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 --2- 172.21.15.59:0/107696815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4109120 0x7f6fb4110ca0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 --2- 172.21.15.59:0/107696815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 0x7f6fb41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 --2- 172.21.15.59:0/107696815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4103ff0 0x7f6fb41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 -- 172.21.15.59:0/107696815 >> 172.21.15.59:0/107696815 conn(0x7f6fb40ffa90 msgr2=0x7f6fb4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 -- 172.21.15.59:0/107696815 shutdown_connections 2024-09-05T12:08:09.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.882+0000 7f6fba200640 1 -- 172.21.15.59:0/107696815 wait complete. 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 Processor -- start 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 -- start start 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 0x7f6fb419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 0x7f6fb419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 0x7f6fb419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41141a0 con 0x7f6fb4103ff0 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.883+0000 7f6fba200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4114020 con 0x7f6fb4109120 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fba200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4114320 con 0x7f6fb41049a0 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 0x7f6fb419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 0x7f6fb419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56196/0 (socket says 172.21.15.59:56196) 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3400640 1 -- 172.21.15.59:0/1807856041 learned_addr learned my addr 172.21.15.59:0/1807856041 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 0x7f6fb419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 0x7f6fb419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 -- 172.21.15.59:0/1807856041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 msgr2=0x7f6fb419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 0x7f6fb419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 -- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 msgr2=0x7f6fb419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 0x7f6fb419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.884+0000 7f6fb3e00640 1 -- 172.21.15.59:0/1807856041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb419f650 con 0x7f6fb4109120 2024-09-05T12:08:09.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 0x7f6fb419ef70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fac005e30 tx=0x7f6fac003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fabe00640 1 -- 172.21.15.59:0/1807856041 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac031650 con 0x7f6fb4109120 2024-09-05T12:08:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb419f8e0 con 0x7f6fb4109120 2024-09-05T12:08:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fabe00640 1 -- 172.21.15.59:0/1807856041 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fac0317f0 con 0x7f6fb4109120 2024-09-05T12:08:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fabe00640 1 -- 172.21.15.59:0/1807856041 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac02f990 con 0x7f6fb4109120 2024-09-05T12:08:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.885+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41a7220 con 0x7f6fb4109120 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.888+0000 7f6fbccd4740 1 Processor -- start 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.888+0000 7f6fabe00640 1 -- 172.21.15.59:0/1807856041 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6fac02fb50 con 0x7f6fb4109120 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.888+0000 7f6fbccd4740 1 -- start start 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 0x559211f486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f52fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559211eae650 con 0x559211f534f0 2024-09-05T12:08:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559211e55f10 con 0x559211f48310 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559211eaa610 con 0x559211f48c30 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fb2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 0x559211f486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f52fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fb2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 0x559211f486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50370/0 (socket says 172.21.15.59:50370) 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1017399960 learned_addr learned my addr 172.21.15.59:0/1017399960 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.889+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 -- 172.21.15.59:0/1017399960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 msgr2=0x559211f486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 0x559211f486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 -- 172.21.15.59:0/1017399960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 msgr2=0x559211f558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 -- 172.21.15.59:0/1017399960 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559211bbd580 con 0x559211f48c30 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fabe00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f52fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009c80 tx=0x7f6fa4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36cfdab63537f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fabe00640 1 -- 172.21.15.59:0/1807856041 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6fac0f0660 con 0x7f6fb4109120 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6f9fe00640 1 -- 172.21.15.59:0/1017399960 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4004230 con 0x559211f48c30 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6f9fe00640 1 -- 172.21.15.59:0/1017399960 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa40043d0 con 0x559211f48c30 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.890+0000 7f6f9fe00640 1 -- 172.21.15.59:0/1017399960 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4004cd0 con 0x559211f48c30 2024-09-05T12:08:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1017399960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 msgr2=0x559211f52fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 0x7f6f980a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009fd0 tx=0x7f6fa0005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f52fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009c80 tx=0x7f6fa4011810 comp rx=0 tx=0).stop 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1017399960 shutdown_connections 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f52fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1017399960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48310 0x559211f486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1017399960 >> 172.21.15.59:0/1017399960 conn(0x559211f448a0 msgr2=0x559211f44c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1017399960 shutdown_connections 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.891+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1017399960 wait complete. 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.892+0000 7f6fbccd4740 1 -- start start 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.892+0000 7f6fbccd4740 1 Processor -- start 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f534f0 0x559211f50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559211f560c0 con 0x559211f48310 2024-09-05T12:08:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559211e69350 con 0x559211f48c30 2024-09-05T12:08:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559211f56240 con 0x559211f534f0 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fbccd4740 10 client.0 _ll_register_callbacks cb 0x559211e8c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56216/0 (socket says 172.21.15.59:56216) 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fb3400640 1 -- 172.21.15.59:0/1513501594 learned_addr learned my addr 172.21.15.59:0/1513501594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.893+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f534f0 0x559211f50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1513501594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f534f0 msgr2=0x559211f50e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f534f0 0x559211f50e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1513501594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 msgr2=0x559211f503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1513501594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559211c87960 con 0x559211f48c30 2024-09-05T12:08:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f50900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x559211f45c20 tx=0x7f6fa0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cadd51afe549c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6f9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004480 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa40043d0 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.894+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6fa001a050 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0035050 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 10 client.27017 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559211f61d10 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa40043d0 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6fa0020620 con 0x559211f48c30 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 10 client.27017 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 10 client.27017 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 20 client.27017 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.895+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211f63120 0x559211f655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.896+0000 7f6fbccd4740 4 client.27017 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.896+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559211ebbbc0 con 0x559211f63120 2024-09-05T12:08:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.896+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211f63120 0x559211f655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.897+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211f63120 0x559211f655e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9ca8d7c5b2d89b server_cookie=9a5be72f25e08c1c in_seq=0 out_seq=0 2024-09-05T12:08:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.897+0000 7f6f9cc00640 10 client.27017 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.897+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1513501594 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559211ebbbc0 con 0x559211f63120 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.897+0000 7f6f9cc00640 10 client.27017 handle_command_reply: tid=0 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.897+0000 7f6fbccd4740 1 client.27017 shutdown 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 msgr2=0x559211f50900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f50900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x559211f45c20 tx=0x7f6fa0004730 comp rx=0 tx=0).stop 2024-09-05T12:08:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 shutdown_connections 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f534f0 0x559211f50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f50900 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211f63120 0x559211f655e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1513501594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 >> 172.21.15.59:0/1513501594 conn(0x559211f448a0 msgr2=0x559211f43aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6f9cc00640 0 client.27017 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 shutdown_connections 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1513501594 wait complete. 2024-09-05T12:08:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.898+0000 7f6fbccd4740 20 client.27017 trim_cache size 0 max 16384 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.916+0000 7f6fbccd4740 1 Processor -- start 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 -- start start 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f4a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f6aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41af060 con 0x559211f48310 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41b18e0 con 0x559211f534f0 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41af6d0 con 0x559211f48c30 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f4a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:50380/0 (socket says 172.21.15.59:50380) 2024-09-05T12:08:09.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fb3e00640 1 -- 172.21.15.59:0/2144459699 learned_addr learned my addr 172.21.15.59:0/2144459699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.917+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f4a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56230/0 (socket says 172.21.15.59:56230) 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f6aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 -- 172.21.15.59:0/2144459699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 msgr2=0x559211f6aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f6aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 -- 172.21.15.59:0/2144459699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 msgr2=0x559211f4a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f4a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 -- 172.21.15.59:0/2144459699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559211f446d0 con 0x559211f534f0 2024-09-05T12:08:09.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009430 tx=0x7f6fac0a7040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aea537f9078bb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6f9f400640 1 -- 172.21.15.59:0/2144459699 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0d4020 con 0x559211f534f0 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6f9f400640 1 -- 172.21.15.59:0/2144459699 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fac036050 con 0x559211f534f0 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.918+0000 7f6f9f400640 1 -- 172.21.15.59:0/2144459699 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0bab80 con 0x559211f534f0 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 -- 172.21.15.59:0/2144459699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 msgr2=0x559211f6b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009430 tx=0x7f6fac0a7040 comp rx=0 tx=0).stop 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 -- 172.21.15.59:0/2144459699 shutdown_connections 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f534f0 0x559211f6b3f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48c30 0x559211f6aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/2144459699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f48310 0x559211f4a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 -- 172.21.15.59:0/2144459699 >> 172.21.15.59:0/2144459699 conn(0x559211f448a0 msgr2=0x559211f49d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 -- 172.21.15.59:0/2144459699 shutdown_connections 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.919+0000 7f6fbccd4740 1 -- 172.21.15.59:0/2144459699 wait complete. 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 -- start start 2024-09-05T12:08:09.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 Processor -- start 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41afda0 con 0x559211f534f0 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41b1210 con 0x559211f48c30 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb41b0470 con 0x559211f48310 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.920+0000 7f6fbccd4740 10 client.0 _ll_register_callbacks cb 0x7f6fa0009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56240/0 (socket says 172.21.15.59:56240) 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3e00640 1 -- 172.21.15.59:0/1763552648 learned_addr learned my addr 172.21.15.59:0/1763552648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:40922/0 (socket says 172.21.15.59:40922) 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1763552648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 msgr2=0x559211f5b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1763552648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 msgr2=0x559211f57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 -- 172.21.15.59:0/1763552648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559211f4b180 con 0x559211f48c30 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb3e00640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6fb2a00640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f5b9e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0021fd0 tx=0x7f6fa0011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d219d00470ebd6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:09.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.921+0000 7f6f9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa003e060 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6fa001a040 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fac036050 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0020830 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6fbccd4740 10 client.27023 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559211fd6f40 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.922+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fac036050 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6fa002bb40 con 0x559211f48c30 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 10 client.27023 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 10 client.27023 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 20 client.27023 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211fd76f0 0x559211fd9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 4 client.27023 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "71976967", "release": true, "await": true}] 2024-09-05T12:08:09.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "71976967", "release": true, "await": true}) -- 0x559211eb60d0 con 0x559211fd76f0 2024-09-05T12:08:09.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.923+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211fd76f0 0x559211fd9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:09.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.924+0000 7f6fb3400640 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211fd76f0 0x559211fd9b10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d4a8ad57ccf574 server_cookie=f154ab709637e9b3 in_seq=0 out_seq=0 2024-09-05T12:08:09.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.924+0000 7f6f9cc00640 10 client.27023 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6f9cc00640 1 -- 172.21.15.59:0/1763552648 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559211eb60d0 con 0x559211fd76f0 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6f9cc00640 10 client.27023 handle_command_reply: tid=0 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 client.27023 shutdown 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 msgr2=0x559211f5b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f5b9e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0021fd0 tx=0x7f6fa0011ec0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 shutdown_connections 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559211fd76f0 0x559211fd9b10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559211f534f0 0x559211f57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559211f48c30 0x559211f5b9e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 --2- 172.21.15.59:0/1763552648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559211f48310 0x559211f5b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6f9cc00640 0 client.27023 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:09.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 >> 172.21.15.59:0/1763552648 conn(0x559211f448a0 msgr2=0x559211f43aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 shutdown_connections 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.925+0000 7f6fbccd4740 1 -- 172.21.15.59:0/1763552648 wait complete. 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.926+0000 7f6fbccd4740 20 client.27023 trim_cache size 0 max 16384 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 36, 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "71976967": { 2024-09-05T12:08:09.930 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 36, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 139.3, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:08:09.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.926+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 msgr2=0x7f6f980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.926+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 0x7f6f980a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009fd0 tx=0x7f6fa0005f90 comp rx=0 tx=0).stop 2024-09-05T12:08:09.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.926+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 msgr2=0x7f6fb419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.926+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 0x7f6fb419ef70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fac005e30 tx=0x7f6fac003f10 comp rx=0 tx=0).stop 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 shutdown_connections 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fb4109120 0x7f6fb419ef70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fb41049a0 0x7f6fb419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 --2- 172.21.15.59:0/1807856041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb4103ff0 0x7f6fb419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 >> 172.21.15.59:0/1807856041 conn(0x7f6fb40ffa90 msgr2=0x7f6fb4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 shutdown_connections 2024-09-05T12:08:09.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:09.927+0000 7f6fba200640 1 -- 172.21.15.59:0/1807856041 wait complete. 2024-09-05T12:08:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:10 smithi089 ceph-mon[34499]: pgmap v709: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:08:10.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:10 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3552458271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:10 smithi059 ceph-mon[30025]: pgmap v709: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:08:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:10 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3552458271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:10.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:10 smithi189 ceph-mon[34772]: pgmap v709: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:08:10.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:10 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3552458271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:11.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:08:11.620 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '71976967', total seconds elapsed: 25.3 2024-09-05T12:08:11.715 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:12.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 -- 172.21.15.59:0/1169170039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 msgr2=0x7f5a8c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 --2- 172.21.15.59:0/1169170039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c10eaa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009ca0 tx=0x7f5a8402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 -- 172.21.15.59:0/1169170039 shutdown_connections 2024-09-05T12:08:12.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 --2- 172.21.15.59:0/1169170039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c10eaa0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 --2- 172.21.15.59:0/1169170039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 0x7f5a8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 --2- 172.21.15.59:0/1169170039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.414+0000 7f5a91000640 1 -- 172.21.15.59:0/1169170039 >> 172.21.15.59:0/1169170039 conn(0x7f5a8c0fd8b0 msgr2=0x7f5a8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.415+0000 7f5a91000640 1 -- 172.21.15.59:0/1169170039 shutdown_connections 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.415+0000 7f5a91000640 1 -- 172.21.15.59:0/1169170039 wait complete. 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 Processor -- start 2024-09-05T12:08:12.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 -- start start 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 0x7f5a8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c111fd0 con 0x7f5a8c106f20 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c111e50 con 0x7f5a8c1027a0 2024-09-05T12:08:12.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.416+0000 7f5a91000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c112150 con 0x7f5a8c101df0 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38730/0 (socket says 172.21.15.59:38730) 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 -- 172.21.15.59:0/3619142882 learned_addr learned my addr 172.21.15.59:0/3619142882 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89400640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 0x7f5a8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 -- 172.21.15.59:0/3619142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 msgr2=0x7f5a8c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 0x7f5a8c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 -- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 msgr2=0x7f5a8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 -- 172.21.15.59:0/3619142882 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8c1a6880 con 0x7f5a8c101df0 2024-09-05T12:08:12.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.417+0000 7f5a89e00640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00e9f0 tx=0x7f5a7c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.420 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:12 smithi059 ceph-mon[30025]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:08:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.418+0000 7f5a82a00640 1 -- 172.21.15.59:0/3619142882 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00cd80 con 0x7f5a8c101df0 2024-09-05T12:08:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.418+0000 7f5a82a00640 1 -- 172.21.15.59:0/3619142882 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c005500 con 0x7f5a8c101df0 2024-09-05T12:08:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.418+0000 7f5a82a00640 1 -- 172.21.15.59:0/3619142882 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c010690 con 0x7f5a8c101df0 2024-09-05T12:08:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.418+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8c1a6b70 con 0x7f5a8c101df0 2024-09-05T12:08:12.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.418+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8c1a70b0 con 0x7f5a8c101df0 2024-09-05T12:08:12.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.421+0000 7f5a82a00640 1 -- 172.21.15.59:0/3619142882 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a7c0046c0 con 0x7f5a8c101df0 2024-09-05T12:08:12.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.422+0000 7f5a93bef740 1 Processor -- start 2024-09-05T12:08:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.422+0000 7f5a82a00640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 0x7f5a700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 -- start start 2024-09-05T12:08:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a82a00640 1 -- 172.21.15.59:0/3619142882 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5a7c0cf830 con 0x7f5a8c101df0 2024-09-05T12:08:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a8a800640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 0x7f5a700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e125910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e125e50 0x55ef2e128240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e0813e0 con 0x55ef2e11ac70 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e028d80 con 0x55ef2e125e50 2024-09-05T12:08:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a93bef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e07d3a0 con 0x55ef2e11b590 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36626/0 (socket says 172.21.15.59:36626) 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.423+0000 7f5a89e00640 1 -- 172.21.15.59:0/142795464 learned_addr learned my addr 172.21.15.59:0/142795464 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a8a800640 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e125e50 0x55ef2e128240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89400640 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e125910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a8a800640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 0x7f5a700a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006010 tx=0x7f5a84005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 -- 172.21.15.59:0/142795464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 msgr2=0x55ef2e125910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e125910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 -- 172.21.15.59:0/142795464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e125e50 msgr2=0x55ef2e128240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e125e50 0x55ef2e128240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 -- 172.21.15.59:0/142795464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e047ae0 con 0x55ef2e11ac70 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.424+0000 7f5a89e00640 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c09bbb0 tx=0x7f5a7c09b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd5f579f07a3be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a77400640 1 -- 172.21.15.59:0/142795464 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c09ffd0 con 0x55ef2e11ac70 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a77400640 1 -- 172.21.15.59:0/142795464 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c0a4890 con 0x55ef2e11ac70 2024-09-05T12:08:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a77400640 1 -- 172.21.15.59:0/142795464 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c09b270 con 0x55ef2e11ac70 2024-09-05T12:08:12.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 -- 172.21.15.59:0/142795464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 msgr2=0x55ef2e11b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c09bbb0 tx=0x7f5a7c09b580 comp rx=0 tx=0).stop 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 -- 172.21.15.59:0/142795464 shutdown_connections 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e125e50 0x55ef2e128240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e125910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 --2- 172.21.15.59:0/142795464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11b050 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 -- 172.21.15.59:0/142795464 >> 172.21.15.59:0/142795464 conn(0x55ef2e117200 msgr2=0x55ef2e1175e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 -- 172.21.15.59:0/142795464 shutdown_connections 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.425+0000 7f5a93bef740 1 -- 172.21.15.59:0/142795464 wait complete. 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.426+0000 7f5a93bef740 1 -- start start 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.426+0000 7f5a93bef740 1 Processor -- start 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 0x55ef2e122da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e1232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e128a20 con 0x55ef2e125e50 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e03c0f0 con 0x55ef2e11ac70 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e128ba0 con 0x55ef2e11b590 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a93bef740 10 client.0 _ll_register_callbacks cb 0x55ef2e023bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 0x55ef2e122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36640/0 (socket says 172.21.15.59:36640) 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a8a800640 1 -- 172.21.15.59:0/59390913 learned_addr learned my addr 172.21.15.59:0/59390913 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89400640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e1232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 -- 172.21.15.59:0/59390913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 msgr2=0x55ef2e1232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e1232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 -- 172.21.15.59:0/59390913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 msgr2=0x55ef2e123820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89e00640 1 -- 172.21.15.59:0/59390913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2ddf23c0 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a89400640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e1232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.427+0000 7f5a8a800640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a89e00640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 0x55ef2e122da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0985f0 tx=0x7f5a7c0a4400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd37df42c995822 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a6be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0ba050 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a7c0a4890 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5a7c098290 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5a7c09ffd0 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a93bef740 10 client.27035 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ef2e134ac0 con 0x55ef2e11ac70 2024-09-05T12:08:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.428+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c0a4890 con 0x55ef2e11ac70 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5a7c0a1160 con 0x55ef2e11ac70 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 10 client.27035 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 10 client.27035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 20 client.27035 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e135e10 0x55ef2e138200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 4 client.27035 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2e08e950 con 0x55ef2e135e10 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.429+0000 7f5a89400640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e135e10 0x55ef2e138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.430+0000 7f5a89400640 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e135e10 0x55ef2e138200 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3444ea0b5fb042bd server_cookie=3f349237ac05b052 in_seq=0 out_seq=0 2024-09-05T12:08:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.430+0000 7f5a6be00640 10 client.27035 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.430+0000 7f5a6be00640 1 -- 172.21.15.59:0/59390913 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef2e08e950 con 0x55ef2e135e10 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.430+0000 7f5a6be00640 10 client.27035 handle_command_reply: tid=0 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.430+0000 7f5a93bef740 1 client.27035 shutdown 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 msgr2=0x55ef2e122da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 0x55ef2e122da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0985f0 tx=0x7f5a7c0a4400 comp rx=0 tx=0).stop 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 shutdown_connections 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e135e10 0x55ef2e138200 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e125e50 0x55ef2e123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e11b590 0x55ef2e1232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 --2- 172.21.15.59:0/59390913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11ac70 0x55ef2e122da0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a6be00640 0 client.27035 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 >> 172.21.15.59:0/59390913 conn(0x55ef2e117200 msgr2=0x55ef2e116380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 shutdown_connections 2024-09-05T12:08:12.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 1 -- 172.21.15.59:0/59390913 wait complete. 2024-09-05T12:08:12.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.431+0000 7f5a93bef740 20 client.27035 trim_cache size 0 max 16384 2024-09-05T12:08:12.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:12 smithi089 ceph-mon[34499]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:08:12.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 Processor -- start 2024-09-05T12:08:12.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 -- start start 2024-09-05T12:08:12.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e121c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e13d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1af020 con 0x55ef2e11ac70 2024-09-05T12:08:12.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1b18a0 con 0x55ef2e11b590 2024-09-05T12:08:12.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a93bef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1af690 con 0x55ef2e125e50 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36646/0 (socket says 172.21.15.59:36646) 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.445+0000 7f5a89e00640 1 -- 172.21.15.59:0/900204230 learned_addr learned my addr 172.21.15.59:0/900204230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89400640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a8a800640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e13d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 -- 172.21.15.59:0/900204230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 msgr2=0x55ef2e13d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e13d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 -- 172.21.15.59:0/900204230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 msgr2=0x55ef2e121c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e121c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 -- 172.21.15.59:0/900204230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e138570 con 0x55ef2e11ac70 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a8a800640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e13d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a89e00640 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c09b920 tx=0x7f5a7c09b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825390786f177be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a76a00640 1 -- 172.21.15.59:0/900204230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c09b380 con 0x55ef2e11ac70 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a76a00640 1 -- 172.21.15.59:0/900204230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c0a9070 con 0x55ef2e11ac70 2024-09-05T12:08:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.446+0000 7f5a76a00640 1 -- 172.21.15.59:0/900204230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c098360 con 0x55ef2e11ac70 2024-09-05T12:08:12.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 -- 172.21.15.59:0/900204230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 msgr2=0x55ef2e11d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c09b920 tx=0x7f5a7c09b6b0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 -- 172.21.15.59:0/900204230 shutdown_connections 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e13d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e121c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 --2- 172.21.15.59:0/900204230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e11d6d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 -- 172.21.15.59:0/900204230 >> 172.21.15.59:0/900204230 conn(0x55ef2e117200 msgr2=0x55ef2e121540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 -- 172.21.15.59:0/900204230 shutdown_connections 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.447+0000 7f5a93bef740 1 -- 172.21.15.59:0/900204230 wait complete. 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 -- start start 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 Processor -- start 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e198a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e129350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1afd60 con 0x55ef2e11ac70 2024-09-05T12:08:12.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1b11d0 con 0x55ef2e11b590 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1b0430 con 0x55ef2e125e50 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.448+0000 7f5a93bef740 10 client.0 _ll_register_callbacks cb 0x7f5a7c09cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a8a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e129350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36652/0 (socket says 172.21.15.59:36652) 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89e00640 1 -- 172.21.15.59:0/1443124659 learned_addr learned my addr 172.21.15.59:0/1443124659 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40324/0 (socket says 172.21.15.59:40324) 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 -- 172.21.15.59:0/1443124659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 msgr2=0x55ef2e129350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e129350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 -- 172.21.15.59:0/1443124659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 msgr2=0x55ef2e198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:12.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 -- 172.21.15.59:0/1443124659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2e129b80 con 0x55ef2e11b590 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a89400640 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55ef2e128cb0 tx=0x7f5a78010ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c4cde50c3fc7791 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.449+0000 7f5a6be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800fa90 con 0x55ef2e11b590 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a7c0a9070 con 0x55ef2e11b590 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5a7800fc30 con 0x55ef2e11b590 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5a7800fe30 con 0x55ef2e11b590 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a93bef740 10 client.27041 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:12.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ef2e1a0320 con 0x55ef2e11b590 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.450+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7c0a9070 con 0x55ef2e11b590 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5a7801e5f0 con 0x55ef2e11b590 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 10 client.27041 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 20 client.27041 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e1a0a70 0x55ef2e1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 4 client.27041 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef2e088e60 con 0x55ef2e1a0a70 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.451+0000 7f5a89e00640 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e1a0a70 0x55ef2e1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a89e00640 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e1a0a70 0x55ef2e1a2e90 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc04dbe2d61c258 server_cookie=b18d338414303378 in_seq=0 out_seq=0 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a6be00640 10 client.27041 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a6be00640 1 -- 172.21.15.59:0/1443124659 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef2e088e60 con 0x55ef2e1a0a70 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a6be00640 10 client.27041 handle_command_reply: tid=0 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 client.27041 shutdown 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 msgr2=0x55ef2e198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55ef2e128cb0 tx=0x7f5a78010ad0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 shutdown_connections 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e125e50 0x55ef2e129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef2e11b590 0x55ef2e198fa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ef2e1a0a70 0x55ef2e1a2e90 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 --2- 172.21.15.59:0/1443124659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef2e11ac70 0x55ef2e198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 >> 172.21.15.59:0/1443124659 conn(0x55ef2e117200 msgr2=0x55ef2e116380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a6be00640 0 client.27041 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 shutdown_connections 2024-09-05T12:08:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 1 -- 172.21.15.59:0/1443124659 wait complete. 2024-09-05T12:08:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.452+0000 7f5a93bef740 20 client.27041 trim_cache size 0 max 16384 2024-09-05T12:08:12.458 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:08:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 msgr2=0x7f5a700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 0x7f5a700a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006010 tx=0x7f5a84005d80 comp rx=0 tx=0).stop 2024-09-05T12:08:12.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 msgr2=0x7f5a8c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00e9f0 tx=0x7f5a7c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 shutdown_connections 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5a700a2c00 0x7f5a700a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a8c106f20 0x7f5a8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a8c1027a0 0x7f5a8c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 --2- 172.21.15.59:0/3619142882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a8c101df0 0x7f5a8c0744f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 >> 172.21.15.59:0/3619142882 conn(0x7f5a8c0fd8b0 msgr2=0x7f5a8c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 shutdown_connections 2024-09-05T12:08:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:12.453+0000 7f5a91000640 1 -- 172.21.15.59:0/3619142882 wait complete. 2024-09-05T12:08:12.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:12 smithi189 ceph-mon[34772]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-05T12:08:14.207 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:08:14.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:08:14.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:14 smithi089 ceph-mon[34499]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:14.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:14 smithi059 ceph-mon[30025]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:14.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:14 smithi189 ceph-mon[34772]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:14.598 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 -- 172.21.15.59:0/2732185615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658109120 msgr2=0x7f5658110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 --2- 172.21.15.59:0/2732185615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658109120 0x7f5658110ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 -- 172.21.15.59:0/2732185615 shutdown_connections 2024-09-05T12:08:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 --2- 172.21.15.59:0/2732185615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658109120 0x7f5658110ca0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 --2- 172.21.15.59:0/2732185615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 0x7f56581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 --2- 172.21.15.59:0/2732185615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658103ff0 0x7f56581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 -- 172.21.15.59:0/2732185615 >> 172.21.15.59:0/2732185615 conn(0x7f56580ffa90 msgr2=0x7f5658101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 -- 172.21.15.59:0/2732185615 shutdown_connections 2024-09-05T12:08:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.286+0000 7f565d400640 1 -- 172.21.15.59:0/2732185615 wait complete. 2024-09-05T12:08:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.287+0000 7f565d400640 1 Processor -- start 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.287+0000 7f565d400640 1 -- start start 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.287+0000 7f565d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.287+0000 7f565d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 0x7f565819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.287+0000 7f565d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658109120 0x7f565819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f565d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5658114150 con 0x7f5658103ff0 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f565d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5658113fd0 con 0x7f56581049a0 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f565d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56581142d0 con 0x7f5658109120 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5657200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658109120 0x7f565819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36670/0 (socket says 172.21.15.59:36670) 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 -- 172.21.15.59:0/3402967394 learned_addr learned my addr 172.21.15.59:0/3402967394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f564de00640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 0x7f565819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 -- 172.21.15.59:0/3402967394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658109120 msgr2=0x7f565819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658109120 0x7f565819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 -- 172.21.15.59:0/3402967394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 msgr2=0x7f565819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 0x7f565819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.288+0000 7f5656800640 1 -- 172.21.15.59:0/3402967394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565819f5e0 con 0x7f5658103ff0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f5656800640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f564800eb90 tx=0x7f564800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f564fe00640 1 -- 172.21.15.59:0/3402967394 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648004d20 con 0x7f5658103ff0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f564fe00640 1 -- 172.21.15.59:0/3402967394 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564800ee90 con 0x7f5658103ff0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565819f8d0 con 0x7f5658103ff0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56581a71b0 con 0x7f5658103ff0 2024-09-05T12:08:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.289+0000 7f564fe00640 1 -- 172.21.15.59:0/3402967394 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648010830 con 0x7f5658103ff0 2024-09-05T12:08:15.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.292+0000 7f564fe00640 1 -- 172.21.15.59:0/3402967394 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5648005020 con 0x7f5658103ff0 2024-09-05T12:08:15.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.292+0000 7f565ffeb740 1 Processor -- start 2024-09-05T12:08:15.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.292+0000 7f565ffeb740 1 -- start start 2024-09-05T12:08:15.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812978aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129795760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558129795ca0 0x558129798090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5581296f1290 con 0x55812978b3e0 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558129698b40 con 0x558129795ca0 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f565ffeb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581296ed250 con 0x55812978aac0 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f5656800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129795760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f5657200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812978aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f5656800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129795760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36680/0 (socket says 172.21.15.59:36680) 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f5656800640 1 -- 172.21.15.59:0/212717444 learned_addr learned my addr 172.21.15.59:0/212717444 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:15.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.293+0000 7f564de00640 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558129795ca0 0x558129798090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 -- 172.21.15.59:0/212717444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558129795ca0 msgr2=0x558129798090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558129795ca0 0x558129798090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 -- 172.21.15.59:0/212717444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 msgr2=0x558129795760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129795760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 -- 172.21.15.59:0/212717444 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581294626a0 con 0x55812978aac0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f564fe00640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 0x7f563c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f564fe00640 1 -- 172.21.15.59:0/3402967394 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f56480cfa00 con 0x7f5658103ff0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 0x7f563c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.294+0000 7f5657200640 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812978aea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5650004770 tx=0x7f5650005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc1644aee398c96b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.295+0000 7f5647400640 1 -- 172.21.15.59:0/212717444 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565004c070 con 0x55812978aac0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.295+0000 7f5647400640 1 -- 172.21.15.59:0/212717444 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5650003cd0 con 0x55812978aac0 2024-09-05T12:08:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.295+0000 7f5647400640 1 -- 172.21.15.59:0/212717444 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650040070 con 0x55812978aac0 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- 172.21.15.59:0/212717444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 msgr2=0x55812978aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812978aea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5650004770 tx=0x7f5650005e00 comp rx=0 tx=0).stop 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f5657200640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 0x7f563c0a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f565004c000 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- 172.21.15.59:0/212717444 shutdown_connections 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558129795ca0 0x558129798090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129795760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 --2- 172.21.15.59:0/212717444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812978aea0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- 172.21.15.59:0/212717444 >> 172.21.15.59:0/212717444 conn(0x558129787050 msgr2=0x558129787430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- 172.21.15.59:0/212717444 shutdown_connections 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- 172.21.15.59:0/212717444 wait complete. 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.296+0000 7f565ffeb740 1 -- start start 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 Processor -- start 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 0x558129792dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558129798910 con 0x55812978b3e0 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5581296abfa0 con 0x55812978aac0 2024-09-05T12:08:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558129798a90 con 0x558129795ca0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f565ffeb740 10 client.0 _ll_register_callbacks cb 0x5581293ffba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f5656800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f564de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f5657200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 0x558129792dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f5656800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:40374/0 (socket says 172.21.15.59:40374) 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f564de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36696/0 (socket says 172.21.15.59:36696) 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.297+0000 7f5656800640 1 -- 172.21.15.59:0/117077849 learned_addr learned my addr 172.21.15.59:0/117077849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 -- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 msgr2=0x5581297955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 -- 172.21.15.59:0/117077849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 msgr2=0x558129792890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 -- 172.21.15.59:0/117077849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558129793600 con 0x558129795ca0 2024-09-05T12:08:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5656800640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f564de00640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f5657200640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 0x558129792dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f565002a910 tx=0x7f5650031460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b090f35537de5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.298+0000 7f563be00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:08:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650058020 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5650003cd0 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5650003800 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5650063050 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f565ffeb740 10 client.26986 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.299+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558129798910 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5650003cd0 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f56500509c0 con 0x558129795ca0 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 10 client.26986 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 20 client.26986 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581297a5a30 0x5581297a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 4 client.26986 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581296fe800 con 0x5581297a5a30 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.300+0000 7f5656800640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581297a5a30 0x5581297a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.301+0000 7f5656800640 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581297a5a30 0x5581297a7e20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd79184c5c99c48 server_cookie=7bb0bc8db194f22b in_seq=0 out_seq=0 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.301+0000 7f563be00640 10 client.26986 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f563be00640 1 -- 172.21.15.59:0/117077849 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581296fe800 con 0x5581297a5a30 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f563be00640 10 client.26986 handle_command_reply: tid=0 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 client.26986 shutdown 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 msgr2=0x558129792dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 0x558129792dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f565002a910 tx=0x7f5650031460 comp rx=0 tx=0).stop 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 shutdown_connections 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558129795ca0 0x558129792dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581297a5a30 0x5581297a7e20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812978b3e0 0x558129792890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 --2- 172.21.15.59:0/117077849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x5581297955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f563be00640 0 client.26986 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 >> 172.21.15.59:0/117077849 conn(0x558129787050 msgr2=0x5581297861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 shutdown_connections 2024-09-05T12:08:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 1 -- 172.21.15.59:0/117077849 wait complete. 2024-09-05T12:08:15.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.302+0000 7f565ffeb740 20 client.26986 trim_cache size 0 max 16384 2024-09-05T12:08:15.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.320+0000 7f565ffeb740 1 Processor -- start 2024-09-05T12:08:15.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 -- start start 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x558129792cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978b3e0 0x558129794800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f563400dd80 con 0x558129795ca0 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5634010600 con 0x55812978aac0 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f565ffeb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f563400e3f0 con 0x55812978b3e0 2024-09-05T12:08:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f5657200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f564de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978b3e0 0x558129794800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f5657200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36698/0 (socket says 172.21.15.59:36698) 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f5657200640 1 -- 172.21.15.59:0/509108115 learned_addr learned my addr 172.21.15.59:0/509108115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.321+0000 7f5656800640 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x558129792cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 -- 172.21.15.59:0/509108115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978b3e0 msgr2=0x558129794800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978b3e0 0x558129794800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 -- 172.21.15.59:0/509108115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 msgr2=0x558129792cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x558129792cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 -- 172.21.15.59:0/509108115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581297a8450 con 0x558129795ca0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5657200640 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f565002a430 tx=0x7f56500058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ae77851f9a0293 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5646a00640 1 -- 172.21.15.59:0/509108115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650058050 con 0x558129795ca0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5646a00640 1 -- 172.21.15.59:0/509108115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5650003800 con 0x558129795ca0 2024-09-05T12:08:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.322+0000 7f5646a00640 1 -- 172.21.15.59:0/509108115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650040070 con 0x558129795ca0 2024-09-05T12:08:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 -- 172.21.15.59:0/509108115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 msgr2=0x5581297ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f565002a430 tx=0x7f56500058f0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 -- 172.21.15.59:0/509108115 shutdown_connections 2024-09-05T12:08:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x5581297ad680 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978b3e0 0x558129794800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 --2- 172.21.15.59:0/509108115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978aac0 0x558129792cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 -- 172.21.15.59:0/509108115 >> 172.21.15.59:0/509108115 conn(0x558129787050 msgr2=0x5581297940f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 -- 172.21.15.59:0/509108115 shutdown_connections 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.323+0000 7f565ffeb740 1 -- 172.21.15.59:0/509108115 wait complete. 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 -- start start 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 Processor -- start 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812979b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 0x55812979ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f563400eac0 con 0x558129795ca0 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f563400ff30 con 0x55812978b3e0 2024-09-05T12:08:15.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f563400f190 con 0x55812978aac0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.324+0000 7f565ffeb740 10 client.0 _ll_register_callbacks cb 0x7f565004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5656800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812979b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36704/0 (socket says 172.21.15.59:36704) 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 -- 172.21.15.59:0/1049377385 learned_addr learned my addr 172.21.15.59:0/1049377385 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f564de00640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 0x55812979ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 -- 172.21.15.59:0/1049377385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 msgr2=0x55812979b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812979b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 -- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 msgr2=0x55812979ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 0x55812979ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 -- 172.21.15.59:0/1049377385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558129808cc0 con 0x558129795ca0 2024-09-05T12:08:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f564de00640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 0x55812979ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f5657200640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5650050f40 tx=0x7f565003c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee80e932689c06e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.325+0000 7f563be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650040030 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5650003bc0 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5650003800 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5650058040 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f565ffeb740 10 client.17061 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5581298100b0 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.326+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5650003800 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5650047600 con 0x558129795ca0 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 10 client.17061 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 20 client.17061 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:15.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581298106f0 0x558129812b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:15.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 4 client.17061 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:08:15.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581296f8d10 con 0x5581298106f0 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.327+0000 7f5656800640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581298106f0 0x558129812b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.328+0000 7f5656800640 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581298106f0 0x558129812b10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451ddd0fc2c22279 server_cookie=a2197c05e5d75d8d in_seq=0 out_seq=0 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.328+0000 7f563be00640 10 client.17061 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f563be00640 1 -- 172.21.15.59:0/1049377385 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581296f8d10 con 0x5581298106f0 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f563be00640 10 client.17061 handle_command_reply: tid=0 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 client.17061 shutdown 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 msgr2=0x558129808620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5650050f40 tx=0x7f565003c900 comp rx=0 tx=0).stop 2024-09-05T12:08:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 shutdown_connections 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581298106f0 0x558129812b10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558129795ca0 0x558129808620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55812978b3e0 0x55812979ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 --2- 172.21.15.59:0/1049377385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55812978aac0 0x55812979b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 >> 172.21.15.59:0/1049377385 conn(0x558129787050 msgr2=0x558129794910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f563be00640 0 client.17061 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 shutdown_connections 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 1 -- 172.21.15.59:0/1049377385 wait complete. 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.329+0000 7f565ffeb740 20 client.17061 trim_cache size 0 max 16384 2024-09-05T12:08:15.334 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "772f0804-f2d4-454e-a301-19646ab3ce53", 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 msgr2=0x7f563c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 0x7f563c0a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f565004c000 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 msgr2=0x7f565819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f564800eb90 tx=0x7f564800c6a0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 shutdown_connections 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f563c0a2b80 0x7f563c0a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5658109120 0x7f565819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56581049a0 0x7f565819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 --2- 172.21.15.59:0/3402967394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5658103ff0 0x7f565819a630 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 >> 172.21.15.59:0/3402967394 conn(0x7f56580ffa90 msgr2=0x7f56581005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 shutdown_connections 2024-09-05T12:08:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:15.330+0000 7f565d400640 1 -- 172.21.15.59:0/3402967394 wait complete. 2024-09-05T12:08:16.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:08:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:16 smithi089 ceph-mon[34499]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:16 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:08:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:16 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:16 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:08:16.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:16 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:16 smithi059 ceph-mon[30025]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:16 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:08:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:16 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:16 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:08:16.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:16 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:16.557 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:16 smithi189 ceph-mon[34772]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-09-05T12:08:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:16 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:08:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:16 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:16 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:08:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:16 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:08:17.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 -- 172.21.15.59:0/3339898185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2084104010 msgr2=0x7f20841043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 --2- 172.21.15.59:0/3339898185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2084104010 0x7f20841043f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2078009900 tx=0x7f207802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 -- 172.21.15.59:0/3339898185 shutdown_connections 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 --2- 172.21.15.59:0/3339898185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084109140 0x7f2084110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 --2- 172.21.15.59:0/3339898185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20841049c0 0x7f2084108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 --2- 172.21.15.59:0/3339898185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2084104010 0x7f20841043f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 -- 172.21.15.59:0/3339898185 >> 172.21.15.59:0/3339898185 conn(0x7f20840ffa50 msgr2=0x7f2084101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 -- 172.21.15.59:0/3339898185 shutdown_connections 2024-09-05T12:08:17.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.153+0000 7f208c200640 1 -- 172.21.15.59:0/3339898185 wait complete. 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.154+0000 7f208c200640 1 Processor -- start 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.154+0000 7f208c200640 1 -- start start 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 0x7f208419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 0x7f208419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2084109140 0x7f208419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2084114170 con 0x7f20841049c0 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2084113ff0 con 0x7f2084109140 2024-09-05T12:08:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20841142f0 con 0x7f2084104010 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f2088e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 0x7f208419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f2089800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 0x7f208419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f2089800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 0x7f208419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38838/0 (socket says 172.21.15.59:38838) 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f2089800640 1 -- 172.21.15.59:0/1855153472 learned_addr learned my addr 172.21.15.59:0/1855153472 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.155+0000 7f208a200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2084109140 0x7f208419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 -- 172.21.15.59:0/1855153472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 msgr2=0x7f208419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 0x7f208419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 -- 172.21.15.59:0/1855153472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2084109140 msgr2=0x7f208419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2084109140 0x7f208419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 -- 172.21.15.59:0/1855153472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208419f620 con 0x7f20841049c0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2088e00640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 0x7f208419abb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f207000e9f0 tx=0x7f207000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2082000640 1 -- 172.21.15.59:0/1855153472 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207000cd80 con 0x7f20841049c0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2082000640 1 -- 172.21.15.59:0/1855153472 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20700035f0 con 0x7f20841049c0 2024-09-05T12:08:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f2082000640 1 -- 172.21.15.59:0/1855153472 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070010690 con 0x7f20841049c0 2024-09-05T12:08:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208419f910 con 0x7f20841049c0 2024-09-05T12:08:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.156+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20841a71f0 con 0x7f20841049c0 2024-09-05T12:08:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.159+0000 7f208ec86740 1 Processor -- start 2024-09-05T12:08:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 -- start start 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62243830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62243d70 0x55ae62246160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae6219f3e0 con 0x55ae622394b0 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ae62146d80 con 0x55ae62243d70 2024-09-05T12:08:17.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208ec86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae6219b3a0 con 0x55ae62238b90 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f2082000640 1 -- 172.21.15.59:0/1855153472 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2070003b30 con 0x7f20841049c0 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f2089800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62243830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38844/0 (socket says 172.21.15.59:38844) 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.160+0000 7f208a200640 1 -- 172.21.15.59:0/3670283536 learned_addr learned my addr 172.21.15.59:0/3670283536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f2088e00640 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62243d70 0x55ae62246160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 -- 172.21.15.59:0/3670283536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62243d70 msgr2=0x55ae62246160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62243d70 0x55ae62246160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 -- 172.21.15.59:0/3670283536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 msgr2=0x55ae62243830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62243830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 -- 172.21.15.59:0/3670283536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae62165ae0 con 0x55ae62238b90 2024-09-05T12:08:17.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f2089800640 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62243830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.161+0000 7f208a200640 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f207c009f50 tx=0x7f207c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ef719e5c5be25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2076a00640 1 -- 172.21.15.59:0/3670283536 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c004d20 con 0x55ae62238b90 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2076a00640 1 -- 172.21.15.59:0/3670283536 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c004050 con 0x55ae62238b90 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2076a00640 1 -- 172.21.15.59:0/3670283536 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c005260 con 0x55ae62238b90 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2082000640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 0x7f206c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2082000640 1 -- 172.21.15.59:0/1855153472 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f20700cf740 con 0x7f20841049c0 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f2089800640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 0x7f206c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.162+0000 7f208ec86740 1 -- 172.21.15.59:0/3670283536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 msgr2=0x55ae62238f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f207c009f50 tx=0x7f207c0118d0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 -- 172.21.15.59:0/3670283536 shutdown_connections 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62243d70 0x55ae62246160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62243830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 --2- 172.21.15.59:0/3670283536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62238f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 -- 172.21.15.59:0/3670283536 >> 172.21.15.59:0/3670283536 conn(0x55ae62235120 msgr2=0x55ae62235500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 -- 172.21.15.59:0/3670283536 shutdown_connections 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 -- 172.21.15.59:0/3670283536 wait complete. 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f2089800640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 0x7f206c0a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2078031550 tx=0x7f2078005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.163+0000 7f208ec86740 1 -- start start 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 Processor -- start 2024-09-05T12:08:17.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62240c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae62241190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 0x55ae622416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ae622469e0 con 0x55ae62243d70 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ae6215a0f0 con 0x55ae622394b0 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae62246b60 con 0x55ae62238b90 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f208ec86740 10 client.0 _ll_register_callbacks cb 0x55ae6214b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.164+0000 7f2089800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62240c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 0x55ae622416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f2089800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62240c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:38852/0 (socket says 172.21.15.59:38852) 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f2089800640 1 -- 172.21.15.59:0/1853319176 learned_addr learned my addr 172.21.15.59:0/1853319176 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:17.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f2088e00640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae62241190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 -- 172.21.15.59:0/1853319176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 msgr2=0x55ae62240c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62240c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 -- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 msgr2=0x55ae62241190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae62241190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 -- 172.21.15.59:0/1853319176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae61f103c0 con 0x55ae62243d70 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f208a200640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 0x55ae622416d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f207c01b040 tx=0x7f207c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814e56bb7709133b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.165+0000 7f206b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c028020 con 0x55ae62243d70 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f207c005050 con 0x55ae62243d70 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c004050 con 0x55ae62243d70 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f207c009be0 con 0x55ae62243d70 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f208ec86740 10 client.17073 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae622526f0 con 0x55ae62243d70 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.166+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c004050 con 0x55ae62243d70 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f207c0246e0 con 0x55ae62243d70 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 10 client.17073 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 20 client.17073 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62253bb0 0x55ae62255fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 4 client.17073 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae621ac950 con 0x55ae62253bb0 2024-09-05T12:08:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.167+0000 7f2089800640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62253bb0 0x55ae62255fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.168+0000 7f2089800640 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62253bb0 0x55ae62255fa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe9f65092b02cff server_cookie=e2c9c7584f596233 in_seq=0 out_seq=0 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.168+0000 7f206b400640 10 client.17073 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.168+0000 7f206b400640 1 -- 172.21.15.59:0/1853319176 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae621ac950 con 0x55ae62253bb0 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.168+0000 7f206b400640 10 client.17073 handle_command_reply: tid=0 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 client.17073 shutdown 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 msgr2=0x55ae622416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 0x55ae622416d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f207c01b040 tx=0x7f207c004430 comp rx=0 tx=0).stop 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 shutdown_connections 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62253bb0 0x55ae62255fa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62243d70 0x55ae622416d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae62241190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 --2- 172.21.15.59:0/1853319176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62238b90 0x55ae62240c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 >> 172.21.15.59:0/1853319176 conn(0x55ae62235120 msgr2=0x55ae622341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f206b400640 0 client.17073 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 shutdown_connections 2024-09-05T12:08:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 1 -- 172.21.15.59:0/1853319176 wait complete. 2024-09-05T12:08:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.169+0000 7f208ec86740 20 client.17073 trim_cache size 0 max 16384 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 Processor -- start 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 -- start start 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae6223ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62238b90 0x55ae62240530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20841af070 con 0x55ae622394b0 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20841b18f0 con 0x55ae62238b90 2024-09-05T12:08:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.188+0000 7f208ec86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20841af6e0 con 0x55ae62235120 2024-09-05T12:08:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36748/0 (socket says 172.21.15.59:36748) 2024-09-05T12:08:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 -- 172.21.15.59:0/964240070 learned_addr learned my addr 172.21.15.59:0/964240070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f2089800640 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae6223ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f2088e00640 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62238b90 0x55ae62240530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 -- 172.21.15.59:0/964240070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 msgr2=0x55ae6223ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae6223ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 -- 172.21.15.59:0/964240070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62238b90 msgr2=0x55ae62240530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62238b90 0x55ae62240530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 -- 172.21.15.59:0/964240070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae62235c60 con 0x55ae622394b0 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.189+0000 7f208a200640 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f207c024cf0 tx=0x7f207c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65196a273ae8145 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f2076000640 1 -- 172.21.15.59:0/964240070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c009b30 con 0x55ae622394b0 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f2076000640 1 -- 172.21.15.59:0/964240070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c03c070 con 0x55ae622394b0 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f2076000640 1 -- 172.21.15.59:0/964240070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00f9b0 con 0x55ae622394b0 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 -- 172.21.15.59:0/964240070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 msgr2=0x55ae62242bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f207c024cf0 tx=0x7f207c0021d0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 -- 172.21.15.59:0/964240070 shutdown_connections 2024-09-05T12:08:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae622394b0 0x55ae62242bc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae62238b90 0x55ae62240530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 --2- 172.21.15.59:0/964240070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae6223ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.190+0000 7f208ec86740 1 -- 172.21.15.59:0/964240070 >> 172.21.15.59:0/964240070 conn(0x55ae6223c640 msgr2=0x55ae622424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.191+0000 7f208ec86740 1 -- 172.21.15.59:0/964240070 shutdown_connections 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.191+0000 7f208ec86740 1 -- 172.21.15.59:0/964240070 wait complete. 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.191+0000 7f208ec86740 1 -- start start 2024-09-05T12:08:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.191+0000 7f208ec86740 1 Processor -- start 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae62247540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae6224bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20841afdb0 con 0x55ae62238b90 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20841b1220 con 0x55ae622394b0 2024-09-05T12:08:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20841b0480 con 0x55ae62235120 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208ec86740 10 client.0 _ll_register_callbacks cb 0x7f207c003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36754/0 (socket says 172.21.15.59:36754) 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 -- 172.21.15.59:0/1071369780 learned_addr learned my addr 172.21.15.59:0/1071369780 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2089800640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae62247540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f208a200640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae6224bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 -- 172.21.15.59:0/1071369780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 msgr2=0x55ae62247540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae62247540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 -- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 msgr2=0x55ae6224bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae6224bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.192+0000 7f2088e00640 1 -- 172.21.15.59:0/1071369780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6224c2b0 con 0x55ae62238b90 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f2088e00640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f207009b830 tx=0x7f207009c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c185a750d543e5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f206b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207009ce60 con 0x55ae62238b90 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f20700981a0 con 0x55ae62238b90 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c03c070 con 0x55ae62238b90 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f207009fee0 con 0x55ae62238b90 2024-09-05T12:08:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f208ec86740 10 client.17079 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae622be0b0 con 0x55ae62238b90 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.193+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c03c070 con 0x55ae62238b90 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f20700cf070 con 0x55ae62238b90 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 10 client.17079 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 20 client.17079 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62243d70 0x55ae622c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 4 client.17079 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae621a6e60 con 0x55ae62243d70 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.194+0000 7f2089800640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62243d70 0x55ae622c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f2089800640 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62243d70 0x55ae622c0870 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991042bf90ddb267 server_cookie=4e30148134654716 in_seq=0 out_seq=0 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f206b400640 10 client.17079 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f206b400640 1 -- 172.21.15.59:0/1071369780 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae621a6e60 con 0x55ae62243d70 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f206b400640 10 client.17079 handle_command_reply: tid=0 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 client.17079 shutdown 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 msgr2=0x55ae62247a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f207009b830 tx=0x7f207009c230 comp rx=0 tx=0).stop 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 shutdown_connections 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ae622394b0 0x55ae6224bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ae62243d70 0x55ae622c0870 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ae62238b90 0x55ae62247a80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 --2- 172.21.15.59:0/1071369780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae62235120 0x55ae62247540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 >> 172.21.15.59:0/1071369780 conn(0x55ae6223c640 msgr2=0x55ae62242cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f206b400640 0 client.17079 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 shutdown_connections 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 1 -- 172.21.15.59:0/1071369780 wait complete. 2024-09-05T12:08:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.195+0000 7f208ec86740 20 client.17079 trim_cache size 0 max 16384 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "772f0804-f2d4-454e-a301-19646ab3ce53": { 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "772f0804-f2d4-454e-a301-19646ab3ce53", 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:08:17.203 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:17.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:17.204 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 msgr2=0x7f206c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 0x7f206c0a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2078031550 tx=0x7f2078005a20 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 msgr2=0x7f208419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 0x7f208419abb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f207000e9f0 tx=0x7f207000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 shutdown_connections 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f206c0a2c00 0x7f206c0a50c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2084109140 0x7f208419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20841049c0 0x7f208419abb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 --2- 172.21.15.59:0/1855153472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2084104010 0x7f208419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 >> 172.21.15.59:0/1855153472 conn(0x7f20840ffa50 msgr2=0x7f2084100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 shutdown_connections 2024-09-05T12:08:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:17.196+0000 7f208c200640 1 -- 172.21.15.59:0/1855153472 wait complete. 2024-09-05T12:08:18.006 INFO:tasks.cephfs.filesystem:scrub status for tag:772f0804-f2d4-454e-a301-19646ab3ce53 - {'path': '/', 'tag': '772f0804-f2d4-454e-a301-19646ab3ce53', 'options': 'recursive,force'} 2024-09-05T12:08:18.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:18 smithi089 ceph-mon[34499]: pgmap v713: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-09-05T12:08:18.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:18 smithi059 ceph-mon[30025]: pgmap v713: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-09-05T12:08:18.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:18 smithi189 ceph-mon[34772]: pgmap v713: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-09-05T12:08:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:20 smithi089 ceph-mon[34499]: pgmap v714: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 4 op/s 2024-09-05T12:08:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:20 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:08:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:20 smithi059 ceph-mon[30025]: pgmap v714: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 4 op/s 2024-09-05T12:08:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:20 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:08:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:20 smithi189 ceph-mon[34772]: pgmap v714: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 4 op/s 2024-09-05T12:08:20.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:20 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:08:22.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:22 smithi059 ceph-mon[30025]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:08:22.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:22 smithi189 ceph-mon[34772]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:08:22.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:22 smithi089 ceph-mon[34499]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 5 op/s 2024-09-05T12:08:24.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:24 smithi059 ceph-mon[30025]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:08:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:24 smithi189 ceph-mon[34772]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:08:24.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:24 smithi089 ceph-mon[34499]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:08:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:26 smithi189 ceph-mon[34772]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:26 smithi089 ceph-mon[34499]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:26 smithi059 ceph-mon[30025]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:28 smithi189 ceph-mon[34772]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:08:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:28 smithi089 ceph-mon[34499]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:08:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:28 smithi059 ceph-mon[30025]: pgmap v718: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-09-05T12:08:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:30 smithi189 ceph-mon[34772]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:30 smithi089 ceph-mon[34499]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:30 smithi059 ceph-mon[30025]: pgmap v719: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:33.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:32 smithi189 ceph-mon[34772]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:32 smithi089 ceph-mon[34499]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:32 smithi059 ceph-mon[30025]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:08:35.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:34 smithi189 ceph-mon[34772]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:34 smithi089 ceph-mon[34499]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:34 smithi059 ceph-mon[30025]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:37.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:36 smithi189 ceph-mon[34772]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:36 smithi089 ceph-mon[34499]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:37.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:36 smithi059 ceph-mon[30025]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:39.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:38 smithi189 ceph-mon[34772]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:38 smithi089 ceph-mon[34499]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:38 smithi059 ceph-mon[30025]: pgmap v723: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:41.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:40 smithi189 ceph-mon[34772]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:40 smithi089 ceph-mon[34499]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:41.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:40 smithi059 ceph-mon[30025]: pgmap v724: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:08:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:42 smithi189 ceph-mon[34772]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:08:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:42 smithi089 ceph-mon[34499]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:08:43.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:42 smithi059 ceph-mon[30025]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:08:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:44 smithi189 ceph-mon[34772]: pgmap v726: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:44 smithi089 ceph-mon[34499]: pgmap v726: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:45.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:44 smithi059 ceph-mon[30025]: pgmap v726: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:46 smithi189 ceph-mon[34772]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:46 smithi089 ceph-mon[34499]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:46 smithi059 ceph-mon[30025]: pgmap v727: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:47.542 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:08:47.557 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:08:47.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:08:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:48.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 -- 172.21.15.59:0/2597959079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 msgr2=0x7f8c941089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 --2- 172.21.15.59:0/2597959079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c941089f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009860 tx=0x7f8c8002a340 comp rx=0 tx=0).stop 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 -- 172.21.15.59:0/2597959079 shutdown_connections 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 --2- 172.21.15.59:0/2597959079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94109120 0x7f8c94110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 --2- 172.21.15.59:0/2597959079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c941089f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 --2- 172.21.15.59:0/2597959079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94103ff0 0x7f8c941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 -- 172.21.15.59:0/2597959079 >> 172.21.15.59:0/2597959079 conn(0x7f8c940ffa90 msgr2=0x7f8c94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 -- 172.21.15.59:0/2597959079 shutdown_connections 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.508+0000 7f8c9a800640 1 -- 172.21.15.59:0/2597959079 wait complete. 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.509+0000 7f8c9a800640 1 Processor -- start 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.509+0000 7f8c9a800640 1 -- start start 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c9419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 0x7f8c9419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94114140 con 0x7f8c94109120 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94113fc0 con 0x7f8c941049a0 2024-09-05T12:08:48.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c9a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941142c0 con 0x7f8c94103ff0 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 0x7f8c9419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52422/0 (socket says 172.21.15.59:52422) 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93400640 1 -- 172.21.15.59:0/1391718711 learned_addr learned my addr 172.21.15.59:0/1391718711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c92a00640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c9419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 -- 172.21.15.59:0/1391718711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 msgr2=0x7f8c9419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 -- 172.21.15.59:0/1391718711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 msgr2=0x7f8c9419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c9419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.510+0000 7f8c93e00640 1 -- 172.21.15.59:0/1391718711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9419f650 con 0x7f8c94109120 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c93400640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c93e00640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 0x7f8c9419ef70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e940 tx=0x7f8c8c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:48.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c004d20 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8c005020 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9419f940 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0116b0 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.511+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c941a7220 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.512+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c941043d0 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.514+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c8c0054d0 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.515+0000 7f8c8be00640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 0x7f8c780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.516+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8c8c0cf7f0 con 0x7f8c94109120 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.516+0000 7f8c93400640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 0x7f8c780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.516+0000 7f8c93400640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 0x7f8c780a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c840058c0 tx=0x7f8c84009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:48.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.520+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c8c016030 con 0x7f8c94109120 2024-09-05T12:08:48.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.748+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c9410c3c0 con 0x7f8c94109120 2024-09-05T12:08:48.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.749+0000 7f8c8be00640 1 -- 172.21.15.59:0/1391718711 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f8c8c09bc90 con 0x7f8c94109120 2024-09-05T12:08:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:08:48.751 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:08:48.751 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:08:48.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 msgr2=0x7f8c780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:48.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 0x7f8c780a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c840058c0 tx=0x7f8c84009260 comp rx=0 tx=0).stop 2024-09-05T12:08:48.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 msgr2=0x7f8c9419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 0x7f8c9419ef70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e940 tx=0x7f8c8c00ee00 comp rx=0 tx=0).stop 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 shutdown_connections 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8c780a2b80 0x7f8c780a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c94109120 0x7f8c9419ef70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c941049a0 0x7f8c9419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 --2- 172.21.15.59:0/1391718711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94103ff0 0x7f8c9419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 >> 172.21.15.59:0/1391718711 conn(0x7f8c940ffa90 msgr2=0x7f8c94100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 shutdown_connections 2024-09-05T12:08:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:48.752+0000 7f8c9a800640 1 -- 172.21.15.59:0/1391718711 wait complete. 2024-09-05T12:08:48.991 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:49.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:48 smithi189 ceph-mon[34772]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:49.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:48 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1391718711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:49.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:48 smithi059 ceph-mon[30025]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:49.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:48 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1391718711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:48 smithi089 ceph-mon[34499]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:08:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:48 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1391718711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:49.557 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:08:49.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:08:50.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.088+0000 7fd15ae00640 1 -- 172.21.15.59:0/4064996320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 msgr2=0x7fd1541043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.088+0000 7fd15ae00640 1 --2- 172.21.15.59:0/4064996320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd1541043d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd144009900 tx=0x7fd14402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.088+0000 7fd15ae00640 1 -- 172.21.15.59:0/4064996320 shutdown_connections 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 --2- 172.21.15.59:0/4064996320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 0x7fd154110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 --2- 172.21.15.59:0/4064996320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd1541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 --2- 172.21.15.59:0/4064996320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd1541043d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 -- 172.21.15.59:0/4064996320 >> 172.21.15.59:0/4064996320 conn(0x7fd1540ffa90 msgr2=0x7fd154101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 -- 172.21.15.59:0/4064996320 shutdown_connections 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.089+0000 7fd15ae00640 1 -- 172.21.15.59:0/4064996320 wait complete. 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 Processor -- start 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 -- start start 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd154076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 0x7fd154077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd154114150 con 0x7fd1541049a0 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd154113fd0 con 0x7fd154103ff0 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd15ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541142d0 con 0x7fd154109120 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.090+0000 7fd153e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd154076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42916/0 (socket says 172.21.15.59:42916) 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd158e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 0x7fd154077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 -- 172.21.15.59:0/156343477 learned_addr learned my addr 172.21.15.59:0/156343477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 -- 172.21.15.59:0/156343477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 msgr2=0x7fd154077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 0x7fd154077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 -- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 msgr2=0x7fd154076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd154076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 -- 172.21.15.59:0/156343477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1541aacb0 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153400640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd154076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.091+0000 7fd153e00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a20 tx=0x7fd144004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.092+0000 7fd150c00640 1 -- 172.21.15.59:0/156343477 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14403f030 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.092+0000 7fd150c00640 1 -- 172.21.15.59:0/156343477 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd144003ad0 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.092+0000 7fd150c00640 1 -- 172.21.15.59:0/156343477 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144035070 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.092+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1541aaf40 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.092+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541ab340 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.095+0000 7fd150c00640 1 -- 172.21.15.59:0/156343477 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd144042030 con 0x7fd154103ff0 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.095+0000 7fd15d866740 1 Processor -- start 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.095+0000 7fd15d866740 1 -- start start 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c94b0 0x55f1b56d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1b562f3e0 con 0x55f1b56d3d70 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1b55d6d80 con 0x55f1b56c94b0 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd15d866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1b562b3a0 con 0x55f1b56c8b90 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd158e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd158e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52450/0 (socket says 172.21.15.59:52450) 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd158e00640 1 -- 172.21.15.59:0/377662763 learned_addr learned my addr 172.21.15.59:0/377662763 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd153e00640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c94b0 0x55f1b56d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.096+0000 7fd153400640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd153400640 1 -- 172.21.15.59:0/377662763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 msgr2=0x55f1b56c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd153400640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd153400640 1 -- 172.21.15.59:0/377662763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c94b0 msgr2=0x55f1b56d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd153400640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c94b0 0x55f1b56d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd150c00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 0x7fd1380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd153400640 1 -- 172.21.15.59:0/377662763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b55f5ae0 con 0x55f1b56d3d70 2024-09-05T12:08:50.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd158e00640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:50.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.097+0000 7fd150c00640 1 -- 172.21.15.59:0/156343477 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd1440ef6d0 con 0x7fd154103ff0 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.098+0000 7fd158e00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 0x7fd1380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.098+0000 7fd153400640 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d6160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd154077f50 tx=0x7fd1400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66037eb5471295f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.098+0000 7fd149600640 1 -- 172.21.15.59:0/377662763 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004030 con 0x55f1b56d3d70 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.098+0000 7fd149600640 1 -- 172.21.15.59:0/377662763 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd140005500 con 0x55f1b56d3d70 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.098+0000 7fd149600640 1 -- 172.21.15.59:0/377662763 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004d80 con 0x55f1b56d3d70 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd158e00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 0x7fd1380a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009fd0 tx=0x7fd14c009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 -- 172.21.15.59:0/377662763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 msgr2=0x55f1b56d6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d6160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd154077f50 tx=0x7fd1400118d0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 -- 172.21.15.59:0/377662763 shutdown_connections 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d6160 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c94b0 0x55f1b56d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 --2- 172.21.15.59:0/377662763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c8b90 0x55f1b56c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 -- 172.21.15.59:0/377662763 >> 172.21.15.59:0/377662763 conn(0x55f1b56c5120 msgr2=0x55f1b56c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 -- 172.21.15.59:0/377662763 shutdown_connections 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.099+0000 7fd15d866740 1 -- 172.21.15.59:0/377662763 wait complete. 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.100+0000 7fd15d866740 1 -- start start 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.100+0000 7fd15d866740 1 Processor -- start 2024-09-05T12:08:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 0x55f1b56d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 0x55f1b56d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1b56d69e0 con 0x55f1b56c8b90 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1b55ea0f0 con 0x55f1b56d3d70 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1b56d6b60 con 0x55f1b56c94b0 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd15d866740 10 client.0 _ll_register_callbacks cb 0x55f1b55db9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd153400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd153e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 0x55f1b56d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd158e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 0x55f1b56d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd153400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:52456/0 (socket says 172.21.15.59:52456) 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.101+0000 7fd153400640 1 -- 172.21.15.59:0/2066042817 learned_addr learned my addr 172.21.15.59:0/2066042817 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 -- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 msgr2=0x55f1b56d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 0x55f1b56d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 -- 172.21.15.59:0/2066042817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 msgr2=0x55f1b56d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 0x55f1b56d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 -- 172.21.15.59:0/2066042817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b53a03c0 con 0x55f1b56c94b0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153e00640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 0x55f1b56d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd158e00640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 0x55f1b56d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd153400640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd140003820 tx=0x7fd140003850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae2e3639ff3ad34 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.102+0000 7fd13e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140028020 con 0x55f1b56c94b0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd140016040 con 0x55f1b56c94b0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd140005500 con 0x55f1b56c94b0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd140004470 con 0x55f1b56c94b0 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd15d866740 10 client.27028 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1b56e26e0 con 0x55f1b56c94b0 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.103+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd140005500 con 0x55f1b56c94b0 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd140005100 con 0x55f1b56c94b0 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 10 client.27028 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 20 client.27028 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b56e3b40 0x55f1b56e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 4 client.27028 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b563c950 con 0x55f1b56e3b40 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.104+0000 7fd153e00640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b56e3b40 0x55f1b56e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.105+0000 7fd153e00640 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b56e3b40 0x55f1b56e5f30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb50e96b2cd89af8 server_cookie=9942a8bb699f2173 in_seq=0 out_seq=0 2024-09-05T12:08:50.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.105+0000 7fd13e000640 10 client.27028 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.105+0000 7fd13e000640 1 -- 172.21.15.59:0/2066042817 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1b563c950 con 0x55f1b56e3b40 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.105+0000 7fd13e000640 10 client.27028 handle_command_reply: tid=0 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.105+0000 7fd15d866740 1 client.27028 shutdown 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 msgr2=0x55f1b56d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd140003820 tx=0x7fd140003850 comp rx=0 tx=0).stop 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 shutdown_connections 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56d3d70 0x55f1b56d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d11f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b56e3b40 0x55f1b56e5f30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 --2- 172.21.15.59:0/2066042817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c8b90 0x55f1b56d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd13e000640 0 client.27028 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 >> 172.21.15.59:0/2066042817 conn(0x55f1b56c5120 msgr2=0x55f1b56cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 shutdown_connections 2024-09-05T12:08:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 1 -- 172.21.15.59:0/2066042817 wait complete. 2024-09-05T12:08:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.106+0000 7fd15d866740 20 client.27028 trim_cache size 0 max 16384 2024-09-05T12:08:50.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.144+0000 7fd15d866740 1 Processor -- start 2024-09-05T12:08:50.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 -- start start 2024-09-05T12:08:50.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56d3d70 0x55f1b56ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b3490 con 0x55f1b56c94b0 2024-09-05T12:08:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b5d10 con 0x55f1b56c8b90 2024-09-05T12:08:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd15d866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b3b00 con 0x55f1b56d3d70 2024-09-05T12:08:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.145+0000 7fd153400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49438/0 (socket says 172.21.15.59:49438) 2024-09-05T12:08:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 -- 172.21.15.59:0/158783313 learned_addr learned my addr 172.21.15.59:0/158783313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153e00640 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd158e00640 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56d3d70 0x55f1b56ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 -- 172.21.15.59:0/158783313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56d3d70 msgr2=0x55f1b56ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56d3d70 0x55f1b56ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 -- 172.21.15.59:0/158783313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 msgr2=0x55f1b56cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.146+0000 7fd153400640 1 -- 172.21.15.59:0/158783313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b56c92e0 con 0x55f1b56c94b0 2024-09-05T12:08:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.147+0000 7fd153400640 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd140000c00 tx=0x7fd140005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c36277addddc655 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.148+0000 7fd148c00640 1 -- 172.21.15.59:0/158783313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140028040 con 0x55f1b56c94b0 2024-09-05T12:08:50.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.148+0000 7fd148c00640 1 -- 172.21.15.59:0/158783313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400054a0 con 0x55f1b56c94b0 2024-09-05T12:08:50.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.148+0000 7fd148c00640 1 -- 172.21.15.59:0/158783313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14000f9e0 con 0x55f1b56c94b0 2024-09-05T12:08:50.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 -- 172.21.15.59:0/158783313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 msgr2=0x55f1b56eb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd140000c00 tx=0x7fd140005b90 comp rx=0 tx=0).stop 2024-09-05T12:08:50.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 -- 172.21.15.59:0/158783313 shutdown_connections 2024-09-05T12:08:50.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56d3d70 0x55f1b56ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56c94b0 0x55f1b56eb5e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 --2- 172.21.15.59:0/158783313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.149+0000 7fd15d866740 1 -- 172.21.15.59:0/158783313 >> 172.21.15.59:0/158783313 conn(0x55f1b56c5120 msgr2=0x55f1b56ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.150+0000 7fd15d866740 1 -- 172.21.15.59:0/158783313 shutdown_connections 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.150+0000 7fd15d866740 1 -- 172.21.15.59:0/158783313 wait complete. 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.150+0000 7fd15d866740 1 -- start start 2024-09-05T12:08:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.150+0000 7fd15d866740 1 Processor -- start 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b41d0 con 0x55f1b56d3d70 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b5640 con 0x55f1b56c8b90 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1541b48a0 con 0x55f1b56c94b0 2024-09-05T12:08:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.151+0000 7fd15d866740 10 client.0 _ll_register_callbacks cb 0x7fd140009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:42956/0 (socket says 172.21.15.59:42956) 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 -- 172.21.15.59:0/1814468800 learned_addr learned my addr 172.21.15.59:0/1814468800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd158e00640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 -- 172.21.15.59:0/1814468800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 msgr2=0x55f1b56d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 -- 172.21.15.59:0/1814468800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 msgr2=0x55f1b56d7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 -- 172.21.15.59:0/1814468800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b56d8730 con 0x55f1b56c8b90 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd158e00640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd153e00640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b8540 tx=0x7fd14402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb2b8926a1fb555 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.152+0000 7fd13e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:50.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144049040 con 0x55f1b56c8b90 2024-09-05T12:08:50.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd1440b81e0 con 0x55f1b56c8b90 2024-09-05T12:08:50.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1400054a0 con 0x55f1b56c8b90 2024-09-05T12:08:50.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd1440bbb70 con 0x55f1b56c8b90 2024-09-05T12:08:50.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd15d866740 10 client.27101 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.153+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1b574e720 con 0x55f1b56c8b90 2024-09-05T12:08:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1400054a0 con 0x55f1b56c8b90 2024-09-05T12:08:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd144003c80 con 0x55f1b56c8b90 2024-09-05T12:08:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 10 client.27101 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 20 client.27101 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b574eda0 0x55f1b57511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 4 client.27101 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.154+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1b5636e60 con 0x55f1b574eda0 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd153400640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b574eda0 0x55f1b57511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd153400640 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b574eda0 0x55f1b57511f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de6be34d809d34c server_cookie=94b02add20dddef1 in_seq=0 out_seq=0 2024-09-05T12:08:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd13e000640 10 client.27101 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd13e000640 1 -- 172.21.15.59:0/1814468800 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f1b5636e60 con 0x55f1b574eda0 2024-09-05T12:08:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd13e000640 10 client.27101 handle_command_reply: tid=0 2024-09-05T12:08:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.155+0000 7fd15d866740 1 client.27101 shutdown 2024-09-05T12:08:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 msgr2=0x55f1b56dc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b8540 tx=0x7fd14402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 shutdown_connections 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1b574eda0 0x55f1b57511f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1b56d3d70 0x55f1b56d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1b56c94b0 0x55f1b56d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 --2- 172.21.15.59:0/1814468800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1b56c8b90 0x55f1b56dc600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd13e000640 0 client.27101 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 >> 172.21.15.59:0/1814468800 conn(0x55f1b56c5120 msgr2=0x55f1b56d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 shutdown_connections 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 1 -- 172.21.15.59:0/1814468800 wait complete. 2024-09-05T12:08:50.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.156+0000 7fd15d866740 20 client.27101 trim_cache size 0 max 16384 2024-09-05T12:08:50.160 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:50.160 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:08:50.160 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:08:50.160 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 msgr2=0x7fd1380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 0x7fd1380a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009fd0 tx=0x7fd14c009500 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 msgr2=0x7fd154076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a20 tx=0x7fd144004040 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 shutdown_connections 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1380a2c00 0x7fd1380a50c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd154109120 0x7fd154077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1541049a0 0x7fd154076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.157+0000 7fd15ae00640 1 --2- 172.21.15.59:0/156343477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154103ff0 0x7fd154076700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.158+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 >> 172.21.15.59:0/156343477 conn(0x7fd1540ffa90 msgr2=0x7fd154100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.158+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 shutdown_connections 2024-09-05T12:08:50.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:50.158+0000 7fd15ae00640 1 -- 172.21.15.59:0/156343477 wait complete. 2024-09-05T12:08:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:50.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:08:50.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:08:50.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:50.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:50.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:50.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:50.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:08:50.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:08:50.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:50.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:08:50.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:08:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:08:51.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:08:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:08:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:08:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:08:51.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 -- 172.21.15.59:0/452849218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020102790 msgr2=0x7f30201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 --2- 172.21.15.59:0/452849218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020102790 0x7f30201067e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3008009900 tx=0x7f300802a410 comp rx=0 tx=0).stop 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 -- 172.21.15.59:0/452849218 shutdown_connections 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 --2- 172.21.15.59:0/452849218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020106f10 0x7f302010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 --2- 172.21.15.59:0/452849218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020102790 0x7f30201067e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 --2- 172.21.15.59:0/452849218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f30201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 -- 172.21.15.59:0/452849218 >> 172.21.15.59:0/452849218 conn(0x7f30200fd8c0 msgr2=0x7f30200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 -- 172.21.15.59:0/452849218 shutdown_connections 2024-09-05T12:08:51.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.449+0000 7f3024a00640 1 -- 172.21.15.59:0/452849218 wait complete. 2024-09-05T12:08:51.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.450+0000 7f3024a00640 1 Processor -- start 2024-09-05T12:08:51.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.450+0000 7f3024a00640 1 -- start start 2024-09-05T12:08:51.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 0x7f302019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 0x7f302019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3020111f80 con 0x7f3020106f10 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3020111e00 con 0x7f3020101de0 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3024a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3020112100 con 0x7f3020102790 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53458/0 (socket says 172.21.15.59:53458) 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 -- 172.21.15.59:0/1096180467 learned_addr learned my addr 172.21.15.59:0/1096180467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3015400640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 0x7f302019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301e800640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 0x7f302019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 -- 172.21.15.59:0/1096180467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 msgr2=0x7f302019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 0x7f302019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 -- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 msgr2=0x7f302019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 0x7f302019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301de00640 1 -- 172.21.15.59:0/1096180467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302019f690 con 0x7f3020101de0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f3015400640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 0x7f302019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.451+0000 7f301e800640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 0x7f302019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f301de00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f301000ea60 tx=0x7f301000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f3017400640 1 -- 172.21.15.59:0/1096180467 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301000cde0 con 0x7f3020101de0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f3017400640 1 -- 172.21.15.59:0/1096180467 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100035f0 con 0x7f3020101de0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302019f980 con 0x7f3020101de0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f3017400640 1 -- 172.21.15.59:0/1096180467 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010010670 con 0x7f3020101de0 2024-09-05T12:08:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.452+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30201a7260 con 0x7f3020101de0 2024-09-05T12:08:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.455+0000 7f3027439740 1 Processor -- start 2024-09-05T12:08:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.455+0000 7f3017400640 1 -- 172.21.15.59:0/1096180467 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f30100037a0 con 0x7f3020101de0 2024-09-05T12:08:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.455+0000 7f3027439740 1 -- start start 2024-09-05T12:08:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd1cd0 0x55d34eddc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34eddc590 0x55d34edde980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d34ed37530 con 0x55d34eddc590 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d34ecde860 con 0x55d34edd13b0 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3027439740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d34ed334f0 con 0x55d34edd1cd0 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53470/0 (socket says 172.21.15.59:53470) 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 -- 172.21.15.59:0/1732169297 learned_addr learned my addr 172.21.15.59:0/1732169297 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301de00640 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd1cd0 0x55d34eddc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f3015400640 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34eddc590 0x55d34edde980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 -- 172.21.15.59:0/1732169297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd1cd0 msgr2=0x55d34eddc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd1cd0 0x55d34eddc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 -- 172.21.15.59:0/1732169297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34eddc590 msgr2=0x55d34edde980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34eddc590 0x55d34edde980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.456+0000 7f301e800640 1 -- 172.21.15.59:0/1732169297 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34eaa8b90 con 0x55d34edd13b0 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f301e800640 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3018010710 tx=0x7f3018010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cac4a33d3c86adf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f3017400640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 0x7f30040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f300f000640 1 -- 172.21.15.59:0/1732169297 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800e8d0 con 0x55d34edd13b0 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f300f000640 1 -- 172.21.15.59:0/1732169297 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3018012450 con 0x55d34edd13b0 2024-09-05T12:08:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f300f000640 1 -- 172.21.15.59:0/1732169297 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301801c650 con 0x55d34edd13b0 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f3017400640 1 -- 172.21.15.59:0/1096180467 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f30100cf720 con 0x7f3020101de0 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.457+0000 7f301de00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 0x7f30040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- 172.21.15.59:0/1732169297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 msgr2=0x55d34edd1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3018010710 tx=0x7f3018010bd0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- 172.21.15.59:0/1732169297 shutdown_connections 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34eddc590 0x55d34edde980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd1cd0 0x55d34eddc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 --2- 172.21.15.59:0/1732169297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34edd13b0 0x55d34edd1790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f301de00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 0x7f30040a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30100cf9c0 tx=0x7f30100d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- 172.21.15.59:0/1732169297 >> 172.21.15.59:0/1732169297 conn(0x55d34edcd940 msgr2=0x55d34edcdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- 172.21.15.59:0/1732169297 shutdown_connections 2024-09-05T12:08:51.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- 172.21.15.59:0/1732169297 wait complete. 2024-09-05T12:08:51.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.458+0000 7f3027439740 1 -- start start 2024-09-05T12:08:51.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 Processor -- start 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edd9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34edda1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d34eddefb0 con 0x55d34edd1cd0 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d34ecf1e60 con 0x55d34eddc590 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d34eddf130 con 0x55d34edd13b0 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3027439740 10 client.0 _ll_register_callbacks cb 0x55d34ecf6cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:51.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52498/0 (socket says 172.21.15.59:52498) 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f3015400640 1 -- 172.21.15.59:0/525743860 learned_addr learned my addr 172.21.15.59:0/525743860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.459+0000 7f301e800640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34edda1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 -- 172.21.15.59:0/525743860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 msgr2=0x55d34edd9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edd9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 -- 172.21.15.59:0/525743860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 msgr2=0x55d34edd9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 -- 172.21.15.59:0/525743860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34ecfd990 con 0x55d34eddc590 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f3015400640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f301e800640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34edda1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3018000f80 tx=0x7f3018005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be7771a62dc519e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.460+0000 7f3003400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800eae0 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3018012450 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f301800ec80 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3018031050 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3027439740 10 client.27113 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d34edeaee0 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3018012450 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f301800f040 con 0x55d34eddc590 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3027439740 10 client.27113 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.461+0000 7f3027439740 10 client.27113 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3027439740 20 client.27113 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34edec230 0x55d34edee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3027439740 4 client.27113 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34ecda310 con 0x55d34edec230 2024-09-05T12:08:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3015400640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34edec230 0x55d34edee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3015400640 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34edec230 0x55d34edee620 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5e4ea50982d3bd server_cookie=6cf82b187dcbb69a in_seq=0 out_seq=0 2024-09-05T12:08:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.462+0000 7f3003400640 10 client.27113 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3003400640 1 -- 172.21.15.59:0/525743860 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d34ecda310 con 0x55d34edec230 2024-09-05T12:08:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3003400640 10 client.27113 handle_command_reply: tid=0 2024-09-05T12:08:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 client.27113 shutdown 2024-09-05T12:08:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 msgr2=0x55d34edda1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34edda1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3018000f80 tx=0x7f3018005c10 comp rx=0 tx=0).stop 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 shutdown_connections 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34edda1d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34edec230 0x55d34edee620 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edd9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 --2- 172.21.15.59:0/525743860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edd9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.463+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 >> 172.21.15.59:0/525743860 conn(0x55d34edcd940 msgr2=0x55d34edccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.464+0000 7f3003400640 0 client.27113 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.464+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 shutdown_connections 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.464+0000 7f3027439740 1 -- 172.21.15.59:0/525743860 wait complete. 2024-09-05T12:08:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.464+0000 7f3027439740 20 client.27113 trim_cache size 0 max 16384 2024-09-05T12:08:51.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 Processor -- start 2024-09-05T12:08:51.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 -- start start 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edf3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30201af1c0 con 0x55d34edd1cd0 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30201b1a40 con 0x55d34eddc590 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.483+0000 7f3027439740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30201af830 con 0x55d34edd13b0 2024-09-05T12:08:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52500/0 (socket says 172.21.15.59:52500) 2024-09-05T12:08:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 -- 172.21.15.59:0/3040196689 learned_addr learned my addr 172.21.15.59:0/3040196689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f301e800640 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edf3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f301e800640 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 -- 172.21.15.59:0/3040196689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 msgr2=0x55d34edf3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edf3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 -- 172.21.15.59:0/3040196689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 msgr2=0x55d34ede43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.484+0000 7f3015400640 1 -- 172.21.15.59:0/3040196689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34ede8290 con 0x55d34edd1cd0 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.485+0000 7f3015400640 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3008004650 tx=0x7f3008031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33ba69a15825fad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.485+0000 7f300e600640 1 -- 172.21.15.59:0/3040196689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300804b070 con 0x55d34edd1cd0 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.485+0000 7f300e600640 1 -- 172.21.15.59:0/3040196689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3008005560 con 0x55d34edd1cd0 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.485+0000 7f300e600640 1 -- 172.21.15.59:0/3040196689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008035040 con 0x55d34edd1cd0 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 -- 172.21.15.59:0/3040196689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 msgr2=0x55d34edf4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3008004650 tx=0x7f3008031510 comp rx=0 tx=0).stop 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 -- 172.21.15.59:0/3040196689 shutdown_connections 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34edf4100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 --2- 172.21.15.59:0/3040196689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34edf3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 -- 172.21.15.59:0/3040196689 >> 172.21.15.59:0/3040196689 conn(0x55d34edcd940 msgr2=0x55d34edccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 -- 172.21.15.59:0/3040196689 shutdown_connections 2024-09-05T12:08:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.486+0000 7f3027439740 1 -- 172.21.15.59:0/3040196689 wait complete. 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 -- start start 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 Processor -- start 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34ede0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30201aff00 con 0x55d34edd1cd0 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30201b1370 con 0x55d34eddc590 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30201b05d0 con 0x55d34edd13b0 2024-09-05T12:08:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.487+0000 7f3027439740 10 client.0 _ll_register_callbacks cb 0x7f3018016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f301e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34ede0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52512/0 (socket says 172.21.15.59:52512) 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 -- 172.21.15.59:0/846191015 learned_addr learned my addr 172.21.15.59:0/846191015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f301e800640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 -- 172.21.15.59:0/846191015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 msgr2=0x55d34ede0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34ede0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 -- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 msgr2=0x55d34ede0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 -- 172.21.15.59:0/846191015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34ee55c40 con 0x55d34edd1cd0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.488+0000 7f3015400640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3008004580 tx=0x7f3008009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73abf5e7ee8c8bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3003400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008035070 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3008031930 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3008005560 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f300805d050 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3027439740 10 client.17118 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.489+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d34ee56f00 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3008005560 con 0x55d34edd1cd0 2024-09-05T12:08:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f300804e070 con 0x55d34edd1cd0 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 10 client.17118 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 10 client.17118 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 20 client.17118 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34ee57630 0x55d34ee59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 4 client.17118 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55d34ed44c60 con 0x55d34ee57630 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.490+0000 7f301e800640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34ee57630 0x55d34ee59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.491+0000 7f301e800640 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34ee57630 0x55d34ee59a50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36967689c6279d2f server_cookie=5df62c28a1c325d5 in_seq=0 out_seq=0 2024-09-05T12:08:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.491+0000 7f3003400640 10 client.17118 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:51.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3003400640 1 -- 172.21.15.59:0/846191015 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d34ed44c60 con 0x55d34ee57630 2024-09-05T12:08:51.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3003400640 10 client.17118 handle_command_reply: tid=0 2024-09-05T12:08:51.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 client.17118 shutdown 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 msgr2=0x55d34ede07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3008004580 tx=0x7f3008009520 comp rx=0 tx=0).stop 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 shutdown_connections 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d34eddc590 0x55d34ede0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55d34ee57630 0x55d34ee59a50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d34edd1cd0 0x55d34ede07a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 --2- 172.21.15.59:0/846191015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d34edd13b0 0x55d34ede0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 >> 172.21.15.59:0/846191015 conn(0x55d34edcd940 msgr2=0x55d34edd8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.498+0000 7f3003400640 0 client.17118 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.499+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 shutdown_connections 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.499+0000 7f3027439740 1 -- 172.21.15.59:0/846191015 wait complete. 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.499+0000 7f3027439740 20 client.17118 trim_cache size 0 max 16384 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 38, 2024-09-05T12:08:51.501 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "b331832a": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 38, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:08:51.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 msgr2=0x7f30040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 0x7f30040a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30100cf9c0 tx=0x7f30100d5530 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 msgr2=0x7f302019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f301000ea60 tx=0x7f301000ef20 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 shutdown_connections 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30040a2b80 0x7f30040a5040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3020106f10 0x7f302019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3020102790 0x7f302019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 --2- 172.21.15.59:0/1096180467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3020101de0 0x7f302019a6e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 >> 172.21.15.59:0/1096180467 conn(0x7f30200fd8c0 msgr2=0x7f30200fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 shutdown_connections 2024-09-05T12:08:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:51.500+0000 7f3024a00640 1 -- 172.21.15.59:0/1096180467 wait complete. 2024-09-05T12:08:52.344 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b331832a', quiesce took 4.8 seconds. Will release after: 25.2 2024-09-05T12:08:52.416 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:52.859 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:52 smithi059 ceph-mon[30025]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 -- 172.21.15.59:0/2252269212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1079c0 msgr2=0x7f680c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 --2- 172.21.15.59:0/2252269212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1079c0 0x7f680c107da0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009930 tx=0x7f67fc02a420 comp rx=0 tx=0).stop 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 -- 172.21.15.59:0/2252269212 shutdown_connections 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 --2- 172.21.15.59:0/2252269212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 --2- 172.21.15.59:0/2252269212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1019c0 0x7f680c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:52.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 --2- 172.21.15.59:0/2252269212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1079c0 0x7f680c107da0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:52.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 -- 172.21.15.59:0/2252269212 >> 172.21.15.59:0/2252269212 conn(0x7f680c0fd740 msgr2=0x7f680c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:52.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.970+0000 7f680be00640 1 -- 172.21.15.59:0/2252269212 shutdown_connections 2024-09-05T12:08:52.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.971+0000 7f680be00640 1 -- 172.21.15.59:0/2252269212 wait complete. 2024-09-05T12:08:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.971+0000 7f680be00640 1 Processor -- start 2024-09-05T12:08:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 -- start start 2024-09-05T12:08:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 0x7f680c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:52.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 0x7f680c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f680c111dc0 con 0x7f680c1019c0 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f680c111c40 con 0x7f680c102360 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f680c111f40 con 0x7f680c1079c0 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 0x7f680c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 0x7f680c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52534/0 (socket says 172.21.15.59:52534) 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f6802000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680aa00640 1 -- 172.21.15.59:0/674259074 learned_addr learned my addr 172.21.15.59:0/674259074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.972+0000 7f680b400640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 0x7f680c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 -- 172.21.15.59:0/674259074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 msgr2=0x7f680c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 0x7f680c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 -- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 msgr2=0x7f680c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 0x7f680c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 -- 172.21.15.59:0/674259074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c19f490 con 0x7f680c102360 2024-09-05T12:08:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f680b400640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 0x7f680c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6802000640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c19aa20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f400ccc0 con 0x7f680c102360 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c19f780 con 0x7f680c102360 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4010910 con 0x7f680c102360 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.973+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018960 con 0x7f680c102360 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.974+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c1a7060 con 0x7f680c102360 2024-09-05T12:08:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.975+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c101e20 con 0x7f680c102360 2024-09-05T12:08:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.977+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67f4010ac0 con 0x7f680c102360 2024-09-05T12:08:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.978+0000 7f6810a00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 0x7f67f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.978+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f67f40cfdb0 con 0x7f680c102360 2024-09-05T12:08:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.978+0000 7f680aa00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 0x7f67f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.979+0000 7f680aa00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 0x7f67f00a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000c00 tx=0x7f67fc009560 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:52.980+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67f409c8a0 con 0x7f680c102360 2024-09-05T12:08:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:52 smithi189 ceph-mon[34772]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:52 smithi089 ceph-mon[34499]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:53.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.203+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f680c19b840 con 0x7f680c102360 2024-09-05T12:08:53.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.204+0000 7f6810a00640 1 -- 172.21.15.59:0/674259074 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f680c19b840 con 0x7f680c102360 2024-09-05T12:08:53.206 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:08:53.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:08:53.206 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:08:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.207+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 msgr2=0x7f67f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.207+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 0x7f67f00a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000c00 tx=0x7f67fc009560 comp rx=0 tx=0).stop 2024-09-05T12:08:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.207+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 msgr2=0x7f680c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.207+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c19aa20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 shutdown_connections 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f67f00a2b80 0x7f67f00a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f680c1079c0 0x7f680c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f680c102360 0x7f680c19aa20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 --2- 172.21.15.59:0/674259074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f680c1019c0 0x7f680c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 >> 172.21.15.59:0/674259074 conn(0x7f680c0fd740 msgr2=0x7f680c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 shutdown_connections 2024-09-05T12:08:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:53.208+0000 7f680be00640 1 -- 172.21.15.59:0/674259074 wait complete. 2024-09-05T12:08:53.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:53 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/674259074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:54.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:08:54.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:53 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/674259074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:53 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/674259074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:08:54.408 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:54.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:54 smithi059 ceph-mon[30025]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 -- 172.21.15.59:0/1717744187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31981079c0 msgr2=0x7f3198107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 --2- 172.21.15.59:0/1717744187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31981079c0 0x7f3198107da0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3188009930 tx=0x7f318802a420 comp rx=0 tx=0).stop 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 -- 172.21.15.59:0/1717744187 shutdown_connections 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 --2- 172.21.15.59:0/1717744187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3198102360 0x7f319810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 --2- 172.21.15.59:0/1717744187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981019c0 0x7f3198101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 --2- 172.21.15.59:0/1717744187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31981079c0 0x7f3198107da0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 -- 172.21.15.59:0/1717744187 >> 172.21.15.59:0/1717744187 conn(0x7f31980fd740 msgr2=0x7f31980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 -- 172.21.15.59:0/1717744187 shutdown_connections 2024-09-05T12:08:54.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.985+0000 7f3197200640 1 -- 172.21.15.59:0/1717744187 wait complete. 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.986+0000 7f3197200640 1 Processor -- start 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.986+0000 7f3197200640 1 -- start start 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31981019c0 0x7f319819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981079c0 0x7f319819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3198111e10 con 0x7f3198102360 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3198111c90 con 0x7f31981019c0 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3197200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3198111f90 con 0x7f31981079c0 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52552/0 (socket says 172.21.15.59:52552) 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 -- 172.21.15.59:0/2356421015 learned_addr learned my addr 172.21.15.59:0/2356421015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195e00640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31981019c0 0x7f319819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3196800640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981079c0 0x7f319819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 -- 172.21.15.59:0/2356421015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981079c0 msgr2=0x7f319819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981079c0 0x7f319819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 -- 172.21.15.59:0/2356421015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31981019c0 msgr2=0x7f319819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31981019c0 0x7f319819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.987+0000 7f3195400640 1 -- 172.21.15.59:0/2356421015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319819f480 con 0x7f3198102360 2024-09-05T12:08:54.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f3195400640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:54.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f318ea00640 1 -- 172.21.15.59:0/2356421015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400cd80 con 0x7f3198102360 2024-09-05T12:08:54.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319819f770 con 0x7f3198102360 2024-09-05T12:08:54.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f318ea00640 1 -- 172.21.15.59:0/2356421015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3184005500 con 0x7f3198102360 2024-09-05T12:08:54.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f318ea00640 1 -- 172.21.15.59:0/2356421015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184018630 con 0x7f3198102360 2024-09-05T12:08:54.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.988+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981a7050 con 0x7f3198102360 2024-09-05T12:08:54.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.992+0000 7f319fc7e740 1 Processor -- start 2024-09-05T12:08:54.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 -- start start 2024-09-05T12:08:54.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f318ea00640 1 -- 172.21.15.59:0/2356421015 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f31840046c0 con 0x7f3198102360 2024-09-05T12:08:54.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf024220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf031410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561baef8a3e0 con 0x561baf02f020 2024-09-05T12:08:54.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561baef31d80 con 0x561baf024760 2024-09-05T12:08:54.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f319fc7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561baef863a0 con 0x561baf023e40 2024-09-05T12:08:54.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f3195e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f3196800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf024220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f3195e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53540/0 (socket says 172.21.15.59:53540) 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f3195e00640 1 -- 172.21.15.59:0/798487188 learned_addr learned my addr 172.21.15.59:0/798487188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.993+0000 7f3195400640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf031410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 -- 172.21.15.59:0/798487188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 msgr2=0x561baf024220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf024220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 -- 172.21.15.59:0/798487188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 msgr2=0x561baf031410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf031410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 -- 172.21.15.59:0/798487188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561baef50ae0 con 0x561baf024760 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3196800640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf024220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195400640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf031410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3195e00640 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3188009930 tx=0x7f3188005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b179a1ae0ae78988 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.994+0000 7f3183400640 1 -- 172.21.15.59:0/798487188 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188035040 con 0x561baf024760 2024-09-05T12:08:54.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f3183400640 1 -- 172.21.15.59:0/798487188 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3188004480 con 0x561baf024760 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f3183400640 1 -- 172.21.15.59:0/798487188 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31880035f0 con 0x561baf024760 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f318ea00640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f318ea00640 1 -- 172.21.15.59:0/2356421015 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f31840cf930 con 0x7f3198102360 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f319fc7e740 1 -- 172.21.15.59:0/798487188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 msgr2=0x561baf02eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f319fc7e740 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3188009930 tx=0x7f3188005bf0 comp rx=0 tx=0).stop 2024-09-05T12:08:54.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.995+0000 7f3196800640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 -- 172.21.15.59:0/798487188 shutdown_connections 2024-09-05T12:08:55.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf031410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02eae0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 --2- 172.21.15.59:0/798487188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf024220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 -- 172.21.15.59:0/798487188 >> 172.21.15.59:0/798487188 conn(0x561baf0203d0 msgr2=0x561baf0207b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 -- 172.21.15.59:0/798487188 shutdown_connections 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 -- 172.21.15.59:0/798487188 wait complete. 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f3196800640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 0x7f317c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f31900096a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.996+0000 7f319fc7e740 1 -- start start 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 Processor -- start 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf02bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf02ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561baf031b90 con 0x561baf02f020 2024-09-05T12:08:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561baef450f0 con 0x561baf024760 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561baf031d10 con 0x561baf023e40 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f319fc7e740 10 client.0 _ll_register_callbacks cb 0x561baef2cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f3196800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf02ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f3195400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f3196800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf02ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52568/0 (socket says 172.21.15.59:52568) 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f3196800640 1 -- 172.21.15.59:0/909919188 learned_addr learned my addr 172.21.15.59:0/909919188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.997+0000 7f3195e00640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf02bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 -- 172.21.15.59:0/909919188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 msgr2=0x561baf02bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf02bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 -- 172.21.15.59:0/909919188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 msgr2=0x561baf02ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf02ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 -- 172.21.15.59:0/909919188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561baecfb3c0 con 0x561baf024760 2024-09-05T12:08:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3195400640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02c4c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31840ccff0 tx=0x7f3184004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b301b4969de9fbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3177e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318409b3d0 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3188004480 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.998+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3184097ad0 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f318409c480 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 10 client.27137 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561baf03dd00 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3188004480 con 0x561baf024760 2024-09-05T12:08:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f31840a12c0 con 0x561baf024760 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 10 client.27137 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 10 client.27137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 20 client.27137 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf03f050 0x561baf041440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 4 client.27137 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:54.999+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561baef97950 con 0x561baf03f050 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.000+0000 7f3195e00640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf03f050 0x561baf041440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.000+0000 7f3195e00640 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf03f050 0x561baf041440 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c0037417491748 server_cookie=8514ca649b96429a in_seq=0 out_seq=0 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.000+0000 7f3177e00640 10 client.27137 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f3177e00640 1 -- 172.21.15.59:0/909919188 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561baef97950 con 0x561baf03f050 2024-09-05T12:08:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f3177e00640 10 client.27137 handle_command_reply: tid=0 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 client.27137 shutdown 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 msgr2=0x561baf02c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02c4c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31840ccff0 tx=0x7f3184004bc0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 shutdown_connections 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf03f050 0x561baf041440 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf02ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf02c4c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 --2- 172.21.15.59:0/909919188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf02bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f3177e00640 0 client.27137 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.001+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 >> 172.21.15.59:0/909919188 conn(0x561baf0203d0 msgr2=0x561baf0260b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.002+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 shutdown_connections 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.002+0000 7f319fc7e740 1 -- 172.21.15.59:0/909919188 wait complete. 2024-09-05T12:08:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.002+0000 7f319fc7e740 20 client.27137 trim_cache size 0 max 16384 2024-09-05T12:08:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 Processor -- start 2024-09-05T12:08:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 -- start start 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf047250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31981aefa0 con 0x561baf02f020 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b1820 con 0x561baf024760 2024-09-05T12:08:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.016+0000 7f319fc7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31981af610 con 0x561baf023e40 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3196800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf047250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53546/0 (socket says 172.21.15.59:53546) 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 -- 172.21.15.59:0/1570695028 learned_addr learned my addr 172.21.15.59:0/1570695028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56902/0 (socket says 172.21.15.59:56902) 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 -- 172.21.15.59:0/1570695028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 msgr2=0x561baf01b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 -- 172.21.15.59:0/1570695028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 msgr2=0x561baf047250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf047250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 -- 172.21.15.59:0/1570695028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561baf02af30 con 0x561baf024760 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195e00640 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.017+0000 7f3195400640 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31840981e0 tx=0x7f3184004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f19f3c7c566256 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f3182a00640 1 -- 172.21.15.59:0/1570695028 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318409b8c0 con 0x561baf024760 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f3182a00640 1 -- 172.21.15.59:0/1570695028 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840cfc30 con 0x561baf024760 2024-09-05T12:08:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f3182a00640 1 -- 172.21.15.59:0/1570695028 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318409aea0 con 0x561baf024760 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 -- 172.21.15.59:0/1570695028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 msgr2=0x561baf046d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31840981e0 tx=0x7f3184004ae0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 -- 172.21.15.59:0/1570695028 shutdown_connections 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf02f020 0x561baf047250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf024760 0x561baf046d10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 --2- 172.21.15.59:0/1570695028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf023e40 0x561baf01b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.018+0000 7f319fc7e740 1 -- 172.21.15.59:0/1570695028 >> 172.21.15.59:0/1570695028 conn(0x561baf0203d0 msgr2=0x561baf02e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.019+0000 7f319fc7e740 1 -- 172.21.15.59:0/1570695028 shutdown_connections 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.019+0000 7f319fc7e740 1 -- 172.21.15.59:0/1570695028 wait complete. 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.019+0000 7f319fc7e740 1 -- start start 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.019+0000 7f319fc7e740 1 Processor -- start 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf024760 0x561baf037680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf02f020 0x561baf033f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31981afce0 con 0x561baf024760 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b1150 con 0x561baf023e40 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b03b0 con 0x561baf02f020 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f319fc7e740 10 client.0 _ll_register_callbacks cb 0x7f318400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf024760 0x561baf037680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53550/0 (socket says 172.21.15.59:53550) 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 -- 172.21.15.59:0/313583111 learned_addr learned my addr 172.21.15.59:0/313583111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3196800640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf02f020 0x561baf033f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 -- 172.21.15.59:0/313583111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf02f020 msgr2=0x561baf033f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf02f020 0x561baf033f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 -- 172.21.15.59:0/313583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf024760 msgr2=0x561baf037680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.020+0000 7f3195e00640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf024760 0x561baf037680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3195e00640 1 -- 172.21.15.59:0/313583111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561baf0345c0 con 0x561baf023e40 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3195e00640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3188004c40 tx=0x7f3188005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215ae9fb2a03b911 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3177e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188035040 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f31880046c0 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31840cfc30 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3188061050 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.021+0000 7f319fc7e740 10 client.27143 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561baf0aa670 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31840cfc30 con 0x561baf023e40 2024-09-05T12:08:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3188031850 con 0x561baf023e40 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 10 client.27143 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 10 client.27143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 20 client.27143 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf0aa9d0 0x561baf0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 4 client.27143 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561baef91e60 con 0x561baf0aa9d0 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.022+0000 7f3195400640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf0aa9d0 0x561baf0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f3195400640 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf0aa9d0 0x561baf0acdf0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b5cc6d502c36b8 server_cookie=1b84016500df7412 in_seq=0 out_seq=0 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f3177e00640 10 client.27143 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f3177e00640 1 -- 172.21.15.59:0/313583111 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561baef91e60 con 0x561baf0aa9d0 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f3177e00640 10 client.27143 handle_command_reply: tid=0 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 client.27143 shutdown 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 msgr2=0x561baf037140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3188004c40 tx=0x7f3188005b50 comp rx=0 tx=0).stop 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 shutdown_connections 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561baf02f020 0x561baf033f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561baf0aa9d0 0x561baf0acdf0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561baf024760 0x561baf037680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 --2- 172.21.15.59:0/313583111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561baf023e40 0x561baf037140 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f3177e00640 0 client.27143 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.023+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 >> 172.21.15.59:0/313583111 conn(0x561baf0203d0 msgr2=0x561baf01ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 shutdown_connections 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f319fc7e740 1 -- 172.21.15.59:0/313583111 wait complete. 2024-09-05T12:08:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f319fc7e740 20 client.27143 trim_cache size 0 max 16384 2024-09-05T12:08:55.029 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:08:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 msgr2=0x7f317c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 0x7f317c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f31900096a0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 msgr2=0x7f319819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9f0 tx=0x7f318400edf0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 shutdown_connections 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31981079c0 0x7f319819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3198102360 0x7f319819aa10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 --2- 172.21.15.59:0/2356421015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31981019c0 0x7f319819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.024+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 >> 172.21.15.59:0/2356421015 conn(0x7f31980fd740 msgr2=0x7f319810cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.025+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 shutdown_connections 2024-09-05T12:08:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:55.025+0000 7f3197200640 1 -- 172.21.15.59:0/2356421015 wait complete. 2024-09-05T12:08:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:54 smithi189 ceph-mon[34772]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:54 smithi089 ceph-mon[34499]: pgmap v731: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:08:57.017 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:08:57.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:08:57.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:56 smithi059 ceph-mon[30025]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:56 smithi189 ceph-mon[34772]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:56 smithi089 ceph-mon[34499]: pgmap v732: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:57.408 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 -- 172.21.15.59:0/754781614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 msgr2=0x7f665410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 --2- 172.21.15.59:0/754781614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665410dd40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f663c009900 tx=0x7f663c02a480 comp rx=0 tx=0).stop 2024-09-05T12:08:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 -- 172.21.15.59:0/754781614 shutdown_connections 2024-09-05T12:08:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 --2- 172.21.15.59:0/754781614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f665410e280 0x7f6654110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 --2- 172.21.15.59:0/754781614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665410dd40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 --2- 172.21.15.59:0/754781614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6654074510 0x7f66540748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 -- 172.21.15.59:0/754781614 >> 172.21.15.59:0/754781614 conn(0x7f66540fd740 msgr2=0x7f66540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 -- 172.21.15.59:0/754781614 shutdown_connections 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.961+0000 7f665a000640 1 -- 172.21.15.59:0/754781614 wait complete. 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.962+0000 7f665a000640 1 Processor -- start 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.962+0000 7f665a000640 1 -- start start 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f665410e280 0x7f665419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6654114000 con 0x7f6654074e30 2024-09-05T12:08:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6654113e80 con 0x7f6654074510 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f665a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6654114180 con 0x7f665410e280 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f6658c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f6658c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53556/0 (socket says 172.21.15.59:53556) 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f6658c00640 1 -- 172.21.15.59:0/1740509765 learned_addr learned my addr 172.21.15.59:0/1740509765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f664be00640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52620/0 (socket says 172.21.15.59:52620) 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.963+0000 7f6659600640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f665410e280 0x7f665419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 -- 172.21.15.59:0/1740509765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f665410e280 msgr2=0x7f665419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f665410e280 0x7f665419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 -- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 msgr2=0x7f665419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 -- 172.21.15.59:0/1740509765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665419f490 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6658c00640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f664400eb90 tx=0x7f664400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6652000640 1 -- 172.21.15.59:0/1740509765 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644004d20 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6652000640 1 -- 172.21.15.59:0/1740509765 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664400ee90 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665419f780 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f6652000640 1 -- 172.21.15.59:0/1740509765 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644010830 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.964+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541a7060 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.967+0000 7f6652000640 1 -- 172.21.15.59:0/1740509765 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6644005020 con 0x7f6654074510 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.967+0000 7f665e0c3740 1 Processor -- start 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 -- start start 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685fd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556168556290 con 0x5561685f05d0 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5561684fdb40 con 0x5561685fae90 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f665e0c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556168552250 con 0x5561685efcb0 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f6658c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.968+0000 7f6659600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f6659600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:56940/0 (socket says 172.21.15.59:56940) 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f6659600640 1 -- 172.21.15.59:0/1648618621 learned_addr learned my addr 172.21.15.59:0/1648618621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 -- 172.21.15.59:0/1648618621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 msgr2=0x5561685f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 -- 172.21.15.59:0/1648618621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 msgr2=0x5561685fa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685fa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 -- 172.21.15.59:0/1648618621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561682c76a0 con 0x5561685fae90 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f6658c00640 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f6652000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f6652000640 1 -- 172.21.15.59:0/1740509765 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f66440cfaf0 con 0x7f6654074510 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.969+0000 7f664be00640 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685fd280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f663c02aa70 tx=0x7f663c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bcc6dc0c555ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f6659600640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f6649c00640 1 -- 172.21.15.59:0/1648618621 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c035040 con 0x5561685fae90 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f6649c00640 1 -- 172.21.15.59:0/1648618621 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c004400 con 0x5561685fae90 2024-09-05T12:08:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f6649c00640 1 -- 172.21.15.59:0/1648618621 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0035f0 con 0x5561685fae90 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f665e0c3740 1 -- 172.21.15.59:0/1648618621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 msgr2=0x5561685fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f6659600640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 0x7f66380a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f664c009de0 tx=0x7f664c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.970+0000 7f665e0c3740 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685fd280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f663c02aa70 tx=0x7f663c005de0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 -- 172.21.15.59:0/1648618621 shutdown_connections 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685fd280 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 --2- 172.21.15.59:0/1648618621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 -- 172.21.15.59:0/1648618621 >> 172.21.15.59:0/1648618621 conn(0x5561685ec240 msgr2=0x5561685ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 -- 172.21.15.59:0/1648618621 shutdown_connections 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 -- 172.21.15.59:0/1648618621 wait complete. 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 -- start start 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.971+0000 7f665e0c3740 1 Processor -- start 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5561685fdaa0 con 0x5561685f05d0 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556168510fa0 con 0x5561685fae90 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561685fdc20 con 0x5561685efcb0 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f665e0c3740 10 client.0 _ll_register_callbacks cb 0x556168264ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52628/0 (socket says 172.21.15.59:52628) 2024-09-05T12:08:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f664be00640 1 -- 172.21.15.59:0/2116941768 learned_addr learned my addr 172.21.15.59:0/2116941768 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f6658c00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.972+0000 7f6659600640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 -- 172.21.15.59:0/2116941768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 msgr2=0x5561685fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685fa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 -- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 msgr2=0x5561685f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 -- 172.21.15.59:0/2116941768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561685f8820 con 0x5561685f05d0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f664be00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f663c0048f0 tx=0x7f663c004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5857eb1e86bbb1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f6642000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c052060 con 0x5561685f05d0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.973+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f663c004200 con 0x5561685f05d0 2024-09-05T12:08:57.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c004400 con 0x5561685f05d0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f663c035070 con 0x5561685f05d0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 10 client.17154 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5561686099e0 con 0x5561685f05d0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c004400 con 0x5561685f05d0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f663c0317e0 con 0x5561685f05d0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 10 client.17154 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 20 client.17154 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55616860ad30 0x55616860d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 4 client.17154 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.974+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556168563800 con 0x55616860ad30 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.975+0000 7f6658c00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55616860ad30 0x55616860d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.975+0000 7f6658c00640 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55616860ad30 0x55616860d120 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7914994d8a68bd58 server_cookie=682e5df4d77cf948 in_seq=0 out_seq=0 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.975+0000 7f6642000640 10 client.17154 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f6642000640 1 -- 172.21.15.59:0/2116941768 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556168563800 con 0x55616860ad30 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f6642000640 10 client.17154 handle_command_reply: tid=0 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 client.17154 shutdown 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 msgr2=0x5561685f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f663c0048f0 tx=0x7f663c004920 comp rx=0 tx=0).stop 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 shutdown_connections 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685fae90 0x5561685f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55616860ad30 0x55616860d120 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685f7a90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2116941768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685efcb0 0x5561685fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 >> 172.21.15.59:0/2116941768 conn(0x5561685ec240 msgr2=0x5561685f1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.976+0000 7f6642000640 0 client.17154 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.977+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 shutdown_connections 2024-09-05T12:08:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.977+0000 7f665e0c3740 1 -- 172.21.15.59:0/2116941768 wait complete. 2024-09-05T12:08:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.977+0000 7f665e0c3740 20 client.17154 trim_cache size 0 max 16384 2024-09-05T12:08:57.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 Processor -- start 2024-09-05T12:08:57.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 -- start start 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x556168612c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66541aeeb0 con 0x5561685f05d0 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b1730 con 0x5561685efcb0 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.995+0000 7f665e0c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66541af520 con 0x5561685fae90 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52644/0 (socket says 172.21.15.59:52644) 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 -- 172.21.15.59:0/2256883021 learned_addr learned my addr 172.21.15.59:0/2256883021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:57.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f6658c00640 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f6659600640 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x556168612c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 -- 172.21.15.59:0/2256883021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 msgr2=0x556168612c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x556168612c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 -- 172.21.15.59:0/2256883021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 msgr2=0x5561685f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.996+0000 7f664be00640 1 -- 172.21.15.59:0/2256883021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561685f5fe0 con 0x5561685f05d0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f664be00640 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f663c004bf0 tx=0x7f663c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e188f9ca4be66c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f6649200640 1 -- 172.21.15.59:0/2256883021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c052070 con 0x5561685f05d0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f6649200640 1 -- 172.21.15.59:0/2256883021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c004990 con 0x5561685f05d0 2024-09-05T12:08:57.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f6649200640 1 -- 172.21.15.59:0/2256883021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c003de0 con 0x5561685f05d0 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f665e0c3740 1 -- 172.21.15.59:0/2256883021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 msgr2=0x5561686126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.997+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f663c004bf0 tx=0x7f663c02a480 comp rx=0 tx=0).stop 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 -- 172.21.15.59:0/2256883021 shutdown_connections 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x556168612c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561686126c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2256883021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 -- 172.21.15.59:0/2256883021 >> 172.21.15.59:0/2256883021 conn(0x5561685ec240 msgr2=0x5561685f92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 -- 172.21.15.59:0/2256883021 shutdown_connections 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 -- 172.21.15.59:0/2256883021 wait complete. 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 -- start start 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.998+0000 7f665e0c3740 1 Processor -- start 2024-09-05T12:08:58.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x5561685ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66541afbf0 con 0x5561685f05d0 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b1060 con 0x5561685efcb0 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66541b02c0 con 0x5561685fae90 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f665e0c3740 10 client.0 _ll_register_callbacks cb 0x5561685f89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f6658c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f664be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52654/0 (socket says 172.21.15.59:52654) 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f664be00640 1 -- 172.21.15.59:0/2725354613 learned_addr learned my addr 172.21.15.59:0/2725354613 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:58.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:57.999+0000 7f6659600640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x5561685ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 -- 172.21.15.59:0/2725354613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 msgr2=0x5561685ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x5561685ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 -- 172.21.15.59:0/2725354613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 msgr2=0x5561685ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 -- 172.21.15.59:0/2725354613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561686748e0 con 0x5561685efcb0 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6659600640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x5561685ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f664be00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6658c00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685fee40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f664409bbe0 tx=0x7f6644098240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b9b0bae5209124 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6642000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440d6030 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f664409a1e0 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.000+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c004990 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f66440d9040 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 10 client.27161 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556168675c80 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c004990 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6644098550 con 0x5561685efcb0 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 10 client.27161 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 10 client.27161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 20 client.27161 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556168676350 0x556168678740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 4 client.27161 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55616855dd10 con 0x556168676350 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.001+0000 7f664be00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556168676350 0x556168678740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.002+0000 7f664be00640 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556168676350 0x556168678740 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce84fd3240052ac7 server_cookie=90b8dfcd6ff939e5 in_seq=0 out_seq=0 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.002+0000 7f6642000640 10 client.27161 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f6642000640 1 -- 172.21.15.59:0/2725354613 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55616855dd10 con 0x556168676350 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f6642000640 10 client.27161 handle_command_reply: tid=0 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 client.27161 shutdown 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 msgr2=0x5561685fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685fee40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f664409bbe0 tx=0x7f6644098240 comp rx=0 tx=0).stop 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 shutdown_connections 2024-09-05T12:08:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561685fae90 0x5561685ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556168676350 0x556168678740 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5561685f05d0 0x5561685ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 --2- 172.21.15.59:0/2725354613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561685efcb0 0x5561685fee40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 >> 172.21.15.59:0/2725354613 conn(0x5561685ec240 msgr2=0x5561685f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f6642000640 0 client.27161 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 shutdown_connections 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.003+0000 7f665e0c3740 1 -- 172.21.15.59:0/2725354613 wait complete. 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.004+0000 7f665e0c3740 20 client.27161 trim_cache size 0 max 16384 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:58.008 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:08:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "8bad5ff0-ceef-48eb-945b-8dd7aab79069", 2024-09-05T12:08:58.009 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:08:58.009 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:58.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.004+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 msgr2=0x7f66380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.004+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 0x7f66380a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f664c009de0 tx=0x7f664c005f90 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.004+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 msgr2=0x7f665419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.004+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f664400eb90 tx=0x7f664400c6a0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 shutdown_connections 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66380a2b80 0x7f66380a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f665410e280 0x7f665419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654074e30 0x7f665419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 --2- 172.21.15.59:0/1740509765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6654074510 0x7f665419a4e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 >> 172.21.15.59:0/1740509765 conn(0x7f66540fd740 msgr2=0x7f66540fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 shutdown_connections 2024-09-05T12:08:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:58.005+0000 7f665a000640 1 -- 172.21.15.59:0/1740509765 wait complete. 2024-09-05T12:08:58.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:08:58.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:58 smithi059 ceph-mon[30025]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:58 smithi189 ceph-mon[34772]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:08:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:58 smithi089 ceph-mon[34499]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 -- 172.21.15.59:0/1382652705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 msgr2=0x7f0f3c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 --2- 172.21.15.59:0/1382652705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c110ca0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009ca0 tx=0x7f0f3402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 -- 172.21.15.59:0/1382652705 shutdown_connections 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 --2- 172.21.15.59:0/1382652705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c110ca0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 --2- 172.21.15.59:0/1382652705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 --2- 172.21.15.59:0/1382652705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c103ff0 0x7f0f3c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 -- 172.21.15.59:0/1382652705 >> 172.21.15.59:0/1382652705 conn(0x7f0f3c0ffa90 msgr2=0x7f0f3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 -- 172.21.15.59:0/1382652705 shutdown_connections 2024-09-05T12:08:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.661+0000 7f0f41c00640 1 -- 172.21.15.59:0/1382652705 wait complete. 2024-09-05T12:08:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.662+0000 7f0f41c00640 1 Processor -- start 2024-09-05T12:08:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.662+0000 7f0f41c00640 1 -- start start 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c103ff0 0x7f0f3c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1141a0 con 0x7f0f3c109120 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c114020 con 0x7f0f3c103ff0 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f41c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c114320 con 0x7f0f3c1049a0 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f3a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52664/0 (socket says 172.21.15.59:52664) 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f3b400640 1 -- 172.21.15.59:0/668467418 learned_addr learned my addr 172.21.15.59:0/668467418 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.663+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c103ff0 0x7f0f3c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 -- 172.21.15.59:0/668467418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c103ff0 msgr2=0x7f0f3c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c103ff0 0x7f0f3c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 -- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 msgr2=0x7f0f3c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 -- 172.21.15.59:0/668467418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c19f570 con 0x7f0f3c1049a0 2024-09-05T12:08:59.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3b400640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f3a000640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c19ab00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800e9f0 tx=0x7f0f2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f33400640 1 -- 172.21.15.59:0/668467418 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800ccc0 con 0x7f0f3c1049a0 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f33400640 1 -- 172.21.15.59:0/668467418 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f28010910 con 0x7f0f3c1049a0 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c19f860 con 0x7f0f3c1049a0 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f33400640 1 -- 172.21.15.59:0/668467418 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28018960 con 0x7f0f3c1049a0 2024-09-05T12:08:59.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.664+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a7140 con 0x7f0f3c1049a0 2024-09-05T12:08:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.667+0000 7f0f446a3740 1 Processor -- start 2024-09-05T12:08:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 -- start start 2024-09-05T12:08:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc52faf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5308160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bc52613e0 con 0x556bc5305d70 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556bc5208d80 con 0x556bc52fb4b0 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f446a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556bc525d3a0 con 0x556bc52fab90 2024-09-05T12:08:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f33400640 1 -- 172.21.15.59:0/668467418 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f28010ac0 con 0x7f0f3c1049a0 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53616/0 (socket says 172.21.15.59:53616) 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f3aa00640 1 -- 172.21.15.59:0/4066772770 learned_addr learned my addr 172.21.15.59:0/4066772770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.668+0000 7f0f3b400640 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc52faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3a000640 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5308160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 -- 172.21.15.59:0/4066772770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 msgr2=0x556bc52faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc52faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 -- 172.21.15.59:0/4066772770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 msgr2=0x556bc5308160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5308160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 -- 172.21.15.59:0/4066772770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc5227ae0 con 0x556bc52fb4b0 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.669+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d20 tx=0x7f0f2c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776acc12ab4e793b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f27e00640 1 -- 172.21.15.59:0/4066772770 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01e070 con 0x556bc52fb4b0 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f27e00640 1 -- 172.21.15.59:0/4066772770 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c019b10 con 0x556bc52fb4b0 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f27e00640 1 -- 172.21.15.59:0/4066772770 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00eb60 con 0x556bc52fb4b0 2024-09-05T12:08:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f33400640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 0x7f0f200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f33400640 1 -- 172.21.15.59:0/668467418 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0f280cf7d0 con 0x7f0f3c1049a0 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f3b400640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 0x7f0f200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f446a3740 1 -- 172.21.15.59:0/4066772770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 msgr2=0x556bc5305830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.670+0000 7f0f446a3740 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d20 tx=0x7f0f2c011810 comp rx=0 tx=0).stop 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 -- 172.21.15.59:0/4066772770 shutdown_connections 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5308160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc5305830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 --2- 172.21.15.59:0/4066772770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc52faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 -- 172.21.15.59:0/4066772770 >> 172.21.15.59:0/4066772770 conn(0x556bc52f7120 msgr2=0x556bc52f7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 -- 172.21.15.59:0/4066772770 shutdown_connections 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 -- 172.21.15.59:0/4066772770 wait complete. 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f3b400640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 0x7f0f200a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0f34004af0 tx=0x7f0f34005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 -- start start 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.671+0000 7f0f446a3740 1 Processor -- start 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc5302c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc5305d70 0x556bc53036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bc53089e0 con 0x556bc52fb4b0 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556bc521c0f0 con 0x556bc5305d70 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556bc5308b60 con 0x556bc52fab90 2024-09-05T12:08:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f446a3740 10 client.0 _ll_register_callbacks cb 0x556bc520d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f3a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f3a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52680/0 (socket says 172.21.15.59:52680) 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f3a000640 1 -- 172.21.15.59:0/3990409485 learned_addr learned my addr 172.21.15.59:0/3990409485 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc5302c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.672+0000 7f0f3b400640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc5305d70 0x556bc53036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 -- 172.21.15.59:0/3990409485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 msgr2=0x556bc5302c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc5302c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 -- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc5305d70 msgr2=0x556bc53036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc5305d70 0x556bc53036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 -- 172.21.15.59:0/3990409485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc4fd23c0 con 0x556bc52fb4b0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc5302c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f2809bc00 tx=0x7f0f2809b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8972a18be3cfaf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f24c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:08:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f280cfbc0 con 0x556bc52fb4b0 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0f280cfd60 con 0x556bc52fb4b0 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.673+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2c019b10 con 0x556bc52fb4b0 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0f2809b0c0 con 0x556bc52fb4b0 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 10 client.17172 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bc5314660 con 0x556bc52fb4b0 2024-09-05T12:08:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c019b10 con 0x556bc52fb4b0 2024-09-05T12:08:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0f28018c20 con 0x556bc52fb4b0 2024-09-05T12:08:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 10 client.17172 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 20 client.17172 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5315b20 0x556bc5317f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 4 client.17172 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.674+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc526e950 con 0x556bc5315b20 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.675+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5315b20 0x556bc5317f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.675+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5315b20 0x556bc5317f70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e0818d1d319a12 server_cookie=aed777becc921abf in_seq=0 out_seq=0 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.675+0000 7f0f24c00640 10 client.17172 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f24c00640 1 -- 172.21.15.59:0/3990409485 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bc526e950 con 0x556bc5315b20 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f24c00640 10 client.17172 handle_command_reply: tid=0 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 client.17172 shutdown 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 msgr2=0x556bc5303160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f2809bc00 tx=0x7f0f2809b690 comp rx=0 tx=0).stop 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 shutdown_connections 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc5305d70 0x556bc53036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5315b20 0x556bc5317f70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc52fb4b0 0x556bc5303160 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f24c00640 0 client.17172 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3990409485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc5302c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 >> 172.21.15.59:0/3990409485 conn(0x556bc52f7120 msgr2=0x556bc52fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 shutdown_connections 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.676+0000 7f0f446a3740 1 -- 172.21.15.59:0/3990409485 wait complete. 2024-09-05T12:08:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.677+0000 7f0f446a3740 20 client.17172 trim_cache size 0 max 16384 2024-09-05T12:08:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 Processor -- start 2024-09-05T12:08:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 -- start start 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc531d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc531dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1af0f0 con 0x556bc5305d70 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b1970 con 0x556bc52fb4b0 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.695+0000 7f0f446a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1af760 con 0x556bc52fab90 2024-09-05T12:08:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc531dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52690/0 (socket says 172.21.15.59:52690) 2024-09-05T12:08:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 -- 172.21.15.59:0/3449561626 learned_addr learned my addr 172.21.15.59:0/3449561626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc531d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 -- 172.21.15.59:0/3449561626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 msgr2=0x556bc531d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc531d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 -- 172.21.15.59:0/3449561626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 msgr2=0x556bc531dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc531dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 -- 172.21.15.59:0/3449561626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc52fb2e0 con 0x556bc5305d70 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc531dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.696+0000 7f0f3b400640 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f3402f540 tx=0x7f0f34042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4141351b3cad8e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f27400640 1 -- 172.21.15.59:0/3449561626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34031450 con 0x556bc5305d70 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f27400640 1 -- 172.21.15.59:0/3449561626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f34036040 con 0x556bc5305d70 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f27400640 1 -- 172.21.15.59:0/3449561626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3402f610 con 0x556bc5305d70 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 -- 172.21.15.59:0/3449561626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 msgr2=0x556bc52ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f3402f540 tx=0x7f0f34042000 comp rx=0 tx=0).stop 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 -- 172.21.15.59:0/3449561626 shutdown_connections 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc52ffe80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fb4b0 0x556bc531dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3449561626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fab90 0x556bc531d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.697+0000 7f0f446a3740 1 -- 172.21.15.59:0/3449561626 >> 172.21.15.59:0/3449561626 conn(0x556bc52f7120 msgr2=0x556bc5304ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 -- 172.21.15.59:0/3449561626 shutdown_connections 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 -- 172.21.15.59:0/3449561626 wait complete. 2024-09-05T12:08:59.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 -- start start 2024-09-05T12:08:59.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 Processor -- start 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.698+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 0x556bc530a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f446a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5388020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f446a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1afe30 con 0x556bc5305d70 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f446a3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b12a0 con 0x556bc52fab90 2024-09-05T12:08:59.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f446a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1b0500 con 0x556bc52fb4b0 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f446a3740 10 client.0 _ll_register_callbacks cb 0x7f0f2809b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53650/0 (socket says 172.21.15.59:53650) 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 -- 172.21.15.59:0/3770267368 learned_addr learned my addr 172.21.15.59:0/3770267368 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 0x556bc530a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3b400640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5388020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 -- 172.21.15.59:0/3770267368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 msgr2=0x556bc530a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 0x556bc530a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 -- 172.21.15.59:0/3770267368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 msgr2=0x556bc5388020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5388020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3aa00640 1 -- 172.21.15.59:0/3770267368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc5388610 con 0x556bc52fab90 2024-09-05T12:08:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.699+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 0x556bc530a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.700+0000 7f0f3aa00640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01c040 tx=0x7f0f2c0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92ab1c53f51eca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.700+0000 7f0f24c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.700+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c033060 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.700+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f34036040 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.700+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0f2c019940 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0f2c03a050 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 10 client.27179 fetch_fsmap learned FSMap version 21 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bc53898e0 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f34036040 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0f2c00e7f0 con 0x556bc52fab90 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 10 client.27179 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:08:59.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 10 client.27179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 20 client.27179 populate_metadata read hostname 'smithi059' 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5389e40 0x556bc538c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 4 client.27179 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.701+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bc5268e60 con 0x556bc5389e40 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5389e40 0x556bc538c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f3a000640 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5389e40 0x556bc538c260 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a884698a7867929 server_cookie=799d064201995a11 in_seq=0 out_seq=0 2024-09-05T12:08:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f24c00640 10 client.27179 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f24c00640 1 -- 172.21.15.59:0/3770267368 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556bc5268e60 con 0x556bc5389e40 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f24c00640 10 client.27179 handle_command_reply: tid=0 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f446a3740 1 client.27179 shutdown 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 msgr2=0x556bc530a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.702+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01c040 tx=0x7f0f2c0093f0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 shutdown_connections 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x556bc5389e40 0x556bc538c260 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bc5305d70 0x556bc5388020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc52fb4b0 0x556bc530a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 --2- 172.21.15.59:0/3770267368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556bc52fab90 0x556bc530a2c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f24c00640 0 client.27179 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 >> 172.21.15.59:0/3770267368 conn(0x556bc52f7120 msgr2=0x556bc5302be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 shutdown_connections 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 1 -- 172.21.15.59:0/3770267368 wait complete. 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f446a3740 20 client.27179 trim_cache size 0 max 16384 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:08:59.708 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: "8bad5ff0-ceef-48eb-945b-8dd7aab79069": { 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "8bad5ff0-ceef-48eb-945b-8dd7aab79069", 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:08:59.709 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 msgr2=0x7f0f200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 0x7f0f200a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0f34004af0 tx=0x7f0f34005d80 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 msgr2=0x7f0f3c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.703+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c19ab00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800e9f0 tx=0x7f0f2800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 shutdown_connections 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0f200a2b80 0x7f0f200a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c109120 0x7f0f3c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c1049a0 0x7f0f3c19ab00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 --2- 172.21.15.59:0/668467418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f3c103ff0 0x7f0f3c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 >> 172.21.15.59:0/668467418 conn(0x7f0f3c0ffa90 msgr2=0x7f0f3c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:08:59.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 shutdown_connections 2024-09-05T12:08:59.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:08:59.704+0000 7f0f41c00640 1 -- 172.21.15.59:0/668467418 wait complete. 2024-09-05T12:09:00.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:59 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:09:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:59 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:59 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:09:00.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:08:59 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:59 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:09:00.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:59 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:59 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:09:00.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:08:59 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:59 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:09:00.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:59 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:59 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:09:00.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:08:59 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:00.539 INFO:tasks.cephfs.filesystem:scrub status for tag:8bad5ff0-ceef-48eb-945b-8dd7aab79069 - {'path': '/', 'tag': '8bad5ff0-ceef-48eb-945b-8dd7aab79069', 'options': 'recursive,force'} 2024-09-05T12:09:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:00 smithi189 ceph-mon[34772]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 4 op/s 2024-09-05T12:09:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:00 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:09:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:00 smithi089 ceph-mon[34499]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 4 op/s 2024-09-05T12:09:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:00 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:09:01.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:00 smithi059 ceph-mon[30025]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 4 op/s 2024-09-05T12:09:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:00 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:09:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:02 smithi189 ceph-mon[34772]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T12:09:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:02 smithi089 ceph-mon[34499]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T12:09:03.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:02 smithi059 ceph-mon[30025]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-09-05T12:09:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:04 smithi189 ceph-mon[34772]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 5 op/s 2024-09-05T12:09:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:04 smithi089 ceph-mon[34499]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 5 op/s 2024-09-05T12:09:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:04 smithi059 ceph-mon[30025]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 5 op/s 2024-09-05T12:09:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:06 smithi189 ceph-mon[34772]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:06 smithi089 ceph-mon[34499]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:06 smithi059 ceph-mon[30025]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:09.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:08 smithi189 ceph-mon[34772]: pgmap v738: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:08 smithi089 ceph-mon[34499]: pgmap v738: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:09.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:08 smithi059 ceph-mon[30025]: pgmap v738: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:10 smithi189 ceph-mon[34772]: pgmap v739: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:10 smithi089 ceph-mon[34499]: pgmap v739: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:10 smithi059 ceph-mon[30025]: pgmap v739: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:12 smithi189 ceph-mon[34772]: pgmap v740: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:12 smithi089 ceph-mon[34499]: pgmap v740: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:12 smithi059 ceph-mon[30025]: pgmap v740: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-05T12:09:15.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:14 smithi189 ceph-mon[34772]: pgmap v741: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-05T12:09:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:14 smithi089 ceph-mon[34499]: pgmap v741: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-05T12:09:15.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:14 smithi059 ceph-mon[30025]: pgmap v741: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-05T12:09:17.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:16 smithi189 ceph-mon[34772]: pgmap v742: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:16 smithi089 ceph-mon[34499]: pgmap v742: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:17.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:16 smithi059 ceph-mon[30025]: pgmap v742: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:17.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:09:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:18.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 -- 172.21.15.59:0/2790064819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 msgr2=0x7f2530100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 --2- 172.21.15.59:0/2790064819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f2530100b60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2520009900 tx=0x7f252002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 -- 172.21.15.59:0/2790064819 shutdown_connections 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 --2- 172.21.15.59:0/2790064819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f2530110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 --2- 172.21.15.59:0/2790064819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 0x7f253010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 --2- 172.21.15.59:0/2790064819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f2530100b60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 -- 172.21.15.59:0/2790064819 >> 172.21.15.59:0/2790064819 conn(0x7f25300fc610 msgr2=0x7f25300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 -- 172.21.15.59:0/2790064819 shutdown_connections 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.534+0000 7f252fe00640 1 -- 172.21.15.59:0/2790064819 wait complete. 2024-09-05T12:09:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.535+0000 7f252fe00640 1 Processor -- start 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.535+0000 7f252fe00640 1 -- start start 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f253019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 0x7f253019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2530114030 con 0x7f2530100780 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2530113eb0 con 0x7f253010e330 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25301141b0 con 0x7f2530101130 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f253019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56354/0 (socket says 172.21.15.59:56354) 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252f400640 1 -- 172.21.15.59:0/2038968822 learned_addr learned my addr 172.21.15.59:0/2038968822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f253019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57646/0 (socket says 172.21.15.59:57646) 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.536+0000 7f252e000640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 0x7f253019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 -- 172.21.15.59:0/2038968822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 msgr2=0x7f253019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 0x7f253019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 -- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 msgr2=0x7f253019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f253019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 -- 172.21.15.59:0/2038968822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253019f4d0 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252f400640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2528009c90 tx=0x7f252800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528003b70 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2528003d10 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253019f7c0 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528010960 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.537+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25301a70a0 con 0x7f253010e330 2024-09-05T12:09:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.538+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530100b60 con 0x7f253010e330 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.541+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2528004b40 con 0x7f253010e330 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.542+0000 7f2527400640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 0x7f25140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.542+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f25280cfab0 con 0x7f253010e330 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.542+0000 7f252ea00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 0x7f25140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.543+0000 7f252ea00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 0x7f25140a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2520000c00 tx=0x7f2520005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.544+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f252809c630 con 0x7f253010e330 2024-09-05T12:09:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.770+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f253019b880 con 0x7f253010e330 2024-09-05T12:09:18.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.771+0000 7f2527400640 1 -- 172.21.15.59:0/2038968822 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f253019b880 con 0x7f253010e330 2024-09-05T12:09:18.773 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:09:18.773 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:09:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 msgr2=0x7f25140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 0x7f25140a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2520000c00 tx=0x7f2520005b50 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 msgr2=0x7f253019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2528009c90 tx=0x7f252800c6a0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 shutdown_connections 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f25140a2b80 0x7f25140a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f253010e330 0x7f253019edf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2530101130 0x7f253019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 --2- 172.21.15.59:0/2038968822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530100780 0x7f253019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 >> 172.21.15.59:0/2038968822 conn(0x7f25300fc610 msgr2=0x7f25300fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 shutdown_connections 2024-09-05T12:09:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:18.774+0000 7f252fe00640 1 -- 172.21.15.59:0/2038968822 wait complete. 2024-09-05T12:09:19.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:18 smithi059 ceph-mon[30025]: pgmap v743: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:19.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:18 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2038968822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:18 smithi089 ceph-mon[34499]: pgmap v743: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:18 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2038968822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:19.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:18 smithi189 ceph-mon[34772]: pgmap v743: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:19.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:18 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2038968822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:19.641 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id b331832a --release --await' 2024-09-05T12:09:19.641 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id b331832a --release --await 2024-09-05T12:09:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 -- 172.21.15.59:0/2141190006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c01047e0 msgr2=0x7f47c0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 --2- 172.21.15.59:0/2141190006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c01047e0 0x7f47c0108830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac02a410 comp rx=0 tx=0).stop 2024-09-05T12:09:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 -- 172.21.15.59:0/2141190006 shutdown_connections 2024-09-05T12:09:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 --2- 172.21.15.59:0/2141190006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c0108f60 0x7f47c0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 --2- 172.21.15.59:0/2141190006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c01047e0 0x7f47c0108830 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 --2- 172.21.15.59:0/2141190006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 0x7f47c0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 -- 172.21.15.59:0/2141190006 >> 172.21.15.59:0/2141190006 conn(0x7f47c00ff8d0 msgr2=0x7f47c0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 -- 172.21.15.59:0/2141190006 shutdown_connections 2024-09-05T12:09:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.552+0000 7f47c5200640 1 -- 172.21.15.59:0/2141190006 wait complete. 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.553+0000 7f47c5200640 1 Processor -- start 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.553+0000 7f47c5200640 1 -- start start 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 0x7f47c019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 0x7f47c019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47c0113fd0 con 0x7f47c0108f60 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47c0113e50 con 0x7f47c01047e0 2024-09-05T12:09:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47c5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47c0114150 con 0x7f47c0103e30 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 0x7f47c019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 0x7f47c019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56380/0 (socket says 172.21.15.59:56380) 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 0x7f47c019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57658/0 (socket says 172.21.15.59:57658) 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.554+0000 7f47bea00640 1 -- 172.21.15.59:0/1735943982 learned_addr learned my addr 172.21.15.59:0/1735943982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 -- 172.21.15.59:0/1735943982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 msgr2=0x7f47c019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 0x7f47c019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 -- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 msgr2=0x7f47c019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 0x7f47c019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 -- 172.21.15.59:0/1735943982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c019f420 con 0x7f47c01047e0 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47bea00640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47ac004770 tx=0x7f47ac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47b7e00640 1 -- 172.21.15.59:0/1735943982 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac03f070 con 0x7f47c01047e0 2024-09-05T12:09:20.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47b7e00640 1 -- 172.21.15.59:0/1735943982 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac031ca0 con 0x7f47c01047e0 2024-09-05T12:09:20.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c019f6b0 con 0x7f47c01047e0 2024-09-05T12:09:20.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.555+0000 7f47b7e00640 1 -- 172.21.15.59:0/1735943982 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac035030 con 0x7f47c01047e0 2024-09-05T12:09:20.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.556+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01a6ff0 con 0x7f47c01047e0 2024-09-05T12:09:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 Processor -- start 2024-09-05T12:09:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 -- start start 2024-09-05T12:09:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47b7e00640 1 -- 172.21.15.59:0/1735943982 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f47ac042070 con 0x7f47c01047e0 2024-09-05T12:09:20.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 0x559e184f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e1844b650 con 0x559e184e5e40 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e183f2f10 con 0x559e184f0700 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.559+0000 7f47c927f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e18447610 con 0x559e184e5520 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:59092/0 (socket says 172.21.15.59:59092) 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 -- 172.21.15.59:0/668939256 learned_addr learned my addr 172.21.15.59:0/668939256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bf400640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 0x559e184f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bea00640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 -- 172.21.15.59:0/668939256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 msgr2=0x559e184f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 -- 172.21.15.59:0/668939256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 msgr2=0x559e184f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 0x559e184f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bfe00640 1 -- 172.21.15.59:0/668939256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1815a580 con 0x559e184e5520 2024-09-05T12:09:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.560+0000 7f47bf400640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 0x559e184f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:09:20.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47bfe00640 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009f50 tx=0x7f47b80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93550a0d1abbf16d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47b7e00640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 0x7f47a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47b4c00640 1 -- 172.21.15.59:0/668939256 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004d20 con 0x559e184e5520 2024-09-05T12:09:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47b4c00640 1 -- 172.21.15.59:0/668939256 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8004050 con 0x559e184e5520 2024-09-05T12:09:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47b4c00640 1 -- 172.21.15.59:0/668939256 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8005280 con 0x559e184e5520 2024-09-05T12:09:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47b7e00640 1 -- 172.21.15.59:0/1735943982 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47ac0efab0 con 0x7f47c01047e0 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.561+0000 7f47bf400640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 0x7f47a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 -- 172.21.15.59:0/668939256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 msgr2=0x559e184e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009f50 tx=0x7f47b80118d0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 -- 172.21.15.59:0/668939256 shutdown_connections 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5e40 0x559e184f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 --2- 172.21.15.59:0/668939256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5520 0x559e184e5900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 -- 172.21.15.59:0/668939256 >> 172.21.15.59:0/668939256 conn(0x559e184e1ab0 msgr2=0x559e184e1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47bf400640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 0x7f47a40a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47b000b7c0 tx=0x7f47b000b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 -- 172.21.15.59:0/668939256 shutdown_connections 2024-09-05T12:09:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.562+0000 7f47c927f740 1 -- 172.21.15.59:0/668939256 wait complete. 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 -- start start 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 Processor -- start 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e184f32d0 con 0x559e184e5520 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559e18406350 con 0x559e184f0700 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e184f3450 con 0x559e184e5e40 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.563+0000 7f47c927f740 10 client.0 _ll_register_callbacks cb 0x559e18429290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184ed6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57674/0 (socket says 172.21.15.59:57674) 2024-09-05T12:09:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bf400640 1 -- 172.21.15.59:0/1127759381 learned_addr learned my addr 172.21.15.59:0/1127759381 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:20.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bea00640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 -- 172.21.15.59:0/1127759381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 msgr2=0x559e184edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 -- 172.21.15.59:0/1127759381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 msgr2=0x559e184ed6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184ed6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 -- 172.21.15.59:0/1127759381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e18224960 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47bfe00640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184ee130 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000f80 tx=0x7f47b80044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b81d54463ab17c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.564+0000 7f47a9600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8029070 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47b800f480 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b8004050 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f47b8035050 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47c927f740 10 client.27197 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e184fefe0 con 0x559e184f0700 2024-09-05T12:09:20.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8004050 con 0x559e184f0700 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.565+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f47b802e440 con 0x559e184f0700 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 10 client.27197 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 10 client.27197 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 20 client.27197 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e18500420 0x559e18502810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 4 client.27197 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e18458bc0 con 0x559e18500420 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.566+0000 7f47bf400640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e18500420 0x559e18502810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.567+0000 7f47bf400640 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e18500420 0x559e18502810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6395ab6e5ffe8630 server_cookie=9d2c1d7801d75973 in_seq=0 out_seq=0 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.567+0000 7f47a9600640 10 client.27197 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.567+0000 7f47a9600640 1 -- 172.21.15.59:0/1127759381 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e18458bc0 con 0x559e18500420 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.567+0000 7f47a9600640 10 client.27197 handle_command_reply: tid=0 2024-09-05T12:09:20.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 client.27197 shutdown 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 msgr2=0x559e184ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184ee130 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f47b8000f80 tx=0x7f47b80044b0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 shutdown_connections 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184ee130 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e18500420 0x559e18502810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 --2- 172.21.15.59:0/1127759381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184ed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47a9600640 0 client.27197 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 >> 172.21.15.59:0/1127759381 conn(0x559e184e1ab0 msgr2=0x559e184e0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 shutdown_connections 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 1 -- 172.21.15.59:0/1127759381 wait complete. 2024-09-05T12:09:20.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.568+0000 7f47c927f740 20 client.27197 trim_cache size 0 max 16384 2024-09-05T12:09:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.586+0000 7f47c927f740 1 Processor -- start 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.586+0000 7f47c927f740 1 -- start start 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e18507e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e18508350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01aef80 con 0x559e184e5520 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01b1800 con 0x559e184f0700 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47c927f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01af5f0 con 0x559e184e5e40 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e18507e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57688/0 (socket says 172.21.15.59:57688) 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 -- 172.21.15.59:0/1896062731 learned_addr learned my addr 172.21.15.59:0/1896062731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bfe00640 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e18508350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 -- 172.21.15.59:0/1896062731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 msgr2=0x559e18507e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e18507e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 -- 172.21.15.59:0/1896062731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 msgr2=0x559e18508350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e18508350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bf400640 1 -- 172.21.15.59:0/1896062731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e184e5c70 con 0x559e184e5520 2024-09-05T12:09:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.587+0000 7f47bea00640 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e18507e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:20.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47bf400640 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x559e18502d50 tx=0x7f47b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc44ed83b1854b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47abe00640 1 -- 172.21.15.59:0/1896062731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0005020 con 0x559e184e5520 2024-09-05T12:09:20.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47abe00640 1 -- 172.21.15.59:0/1896062731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b00047a0 con 0x559e184e5520 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47abe00640 1 -- 172.21.15.59:0/1896062731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00285f0 con 0x559e184e5520 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47c927f740 1 -- 172.21.15.59:0/1896062731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 msgr2=0x559e184e0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.588+0000 7f47c927f740 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x559e18502d50 tx=0x7f47b0015000 comp rx=0 tx=0).stop 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 -- 172.21.15.59:0/1896062731 shutdown_connections 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e18508350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e18507e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 --2- 172.21.15.59:0/1896062731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184e5520 0x559e184e0cf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 -- 172.21.15.59:0/1896062731 >> 172.21.15.59:0/1896062731 conn(0x559e184e1ab0 msgr2=0x559e184ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 -- 172.21.15.59:0/1896062731 shutdown_connections 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 -- 172.21.15.59:0/1896062731 wait complete. 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 -- start start 2024-09-05T12:09:20.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.589+0000 7f47c927f740 1 Processor -- start 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184f5010 0x559e18569de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01afcc0 con 0x559e184f5010 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01b1130 con 0x559e184f0700 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47c01b0390 con 0x559e184e5e40 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47c927f740 10 client.0 _ll_register_callbacks cb 0x7f47ac0f33c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47bea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47bf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47bea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56424/0 (socket says 172.21.15.59:56424) 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47bea00640 1 -- 172.21.15.59:0/868434257 learned_addr learned my addr 172.21.15.59:0/868434257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:20.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.590+0000 7f47bfe00640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184f5010 0x559e18569de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 -- 172.21.15.59:0/868434257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 msgr2=0x559e184f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 -- 172.21.15.59:0/868434257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184f5010 msgr2=0x559e18569de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184f5010 0x559e18569de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 -- 172.21.15.59:0/868434257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1856a3f0 con 0x559e184f0700 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47bea00640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f47ac031550 tx=0x7f47ac0f3110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db06649f155a25c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47a9600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac049040 con 0x559e184f0700 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47ac0b85c0 con 0x559e184f0700 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b00047a0 con 0x559e184f0700 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f47ac005d50 con 0x559e184f0700 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47c927f740 10 client.27203 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.591+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559e1856b6d0 con 0x559e184f0700 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b00047a0 con 0x559e184f0700 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f47ac0bc1b0 con 0x559e184f0700 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 10 client.27203 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 10 client.27203 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 20 client.27203 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e1856bed0 0x559e1856e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 4 client.27203 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b331832a", "release": true, "await": true}] 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b331832a", "release": true, "await": true}) -- 0x559e184530d0 con 0x559e1856bed0 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.592+0000 7f47bf400640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e1856bed0 0x559e1856e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.593+0000 7f47bf400640 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e1856bed0 0x559e1856e2f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb1f580bc151b19 server_cookie=f83f17158b107d7d in_seq=0 out_seq=0 2024-09-05T12:09:20.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.593+0000 7f47a9600640 10 client.27203 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:20.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.594+0000 7f47a9600640 1 -- 172.21.15.59:0/868434257 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e184530d0 con 0x559e1856bed0 2024-09-05T12:09:20.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.594+0000 7f47a9600640 10 client.27203 handle_command_reply: tid=0 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.594+0000 7f47c927f740 1 client.27203 shutdown 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.594+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 msgr2=0x559e184f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.594+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f47ac031550 tx=0x7f47ac0f3110 comp rx=0 tx=0).stop 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 shutdown_connections 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559e1856bed0 0x559e1856e2f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e184f5010 0x559e18569de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559e184f0700 0x559e184f4ad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 --2- 172.21.15.59:0/868434257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e184e5e40 0x559e184f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47a9600640 0 client.27203 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 >> 172.21.15.59:0/868434257 conn(0x559e184e1ab0 msgr2=0x559e184ede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 shutdown_connections 2024-09-05T12:09:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 1 -- 172.21.15.59:0/868434257 wait complete. 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.595+0000 7f47c927f740 20 client.27203 trim_cache size 0 max 16384 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 40, 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "b331832a": { 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 40, 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:09:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:20.603 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 msgr2=0x7f47a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 0x7f47a40a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f47b000b7c0 tx=0x7f47b000b390 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 msgr2=0x7f47c019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47ac004770 tx=0x7f47ac003c10 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 shutdown_connections 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f47a40a2c00 0x7f47a40a50c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47c0108f60 0x7f47c019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47c01047e0 0x7f47c019a9b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 --2- 172.21.15.59:0/1735943982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47c0103e30 0x7f47c019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 >> 172.21.15.59:0/1735943982 conn(0x7f47c00ff8d0 msgr2=0x7f47c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 shutdown_connections 2024-09-05T12:09:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:20.596+0000 7f47c5200640 1 -- 172.21.15.59:0/1735943982 wait complete. 2024-09-05T12:09:21.100 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:20 smithi059 ceph-mon[30025]: pgmap v744: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:09:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:20 smithi089 ceph-mon[34499]: pgmap v744: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:09:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:20 smithi189 ceph-mon[34772]: pgmap v744: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-09-05T12:09:21.622 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b331832a', total seconds elapsed: 34.1 2024-09-05T12:09:21.622 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:09:21.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:09:22.000 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 -- 172.21.15.59:0/1484628885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 msgr2=0x7fc97810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 --2- 172.21.15.59:0/1484628885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc97810e890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc970009cd0 tx=0x7fc97002a8e0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 -- 172.21.15.59:0/1484628885 shutdown_connections 2024-09-05T12:09:22.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 --2- 172.21.15.59:0/1484628885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc97810e890 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 --2- 172.21.15.59:0/1484628885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc978101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 --2- 172.21.15.59:0/1484628885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 0x7fc978107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 -- 172.21.15.59:0/1484628885 >> 172.21.15.59:0/1484628885 conn(0x7fc9780fd740 msgr2=0x7fc9780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 -- 172.21.15.59:0/1484628885 shutdown_connections 2024-09-05T12:09:22.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.595+0000 7fc97e800640 1 -- 172.21.15.59:0/1484628885 wait complete. 2024-09-05T12:09:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.596+0000 7fc97e800640 1 Processor -- start 2024-09-05T12:09:22.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.596+0000 7fc97e800640 1 -- start start 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.596+0000 7fc97e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.596+0000 7fc97e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc978074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 0x7fc978074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc978111dc0 con 0x7fc978102360 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc978111c40 con 0x7fc9781019c0 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc978111f40 con 0x7fc9781079c0 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:60760/0 (socket says 172.21.15.59:60760) 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 -- 172.21.15.59:0/2555570049 learned_addr learned my addr 172.21.15.59:0/2555570049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc978074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97de00640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 0x7fc978074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 -- 172.21.15.59:0/2555570049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 msgr2=0x7fc978074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 0x7fc978074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 -- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 msgr2=0x7fc978074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc978074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.597+0000 7fc97d400640 1 -- 172.21.15.59:0/2555570049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9781a66e0 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc97d400640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e9f0 tx=0x7fc96c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00cd80 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781a69d0 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c005500 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c010690 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.598+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781a6f10 con 0x7fc9781019c0 2024-09-05T12:09:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.599+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978101e20 con 0x7fc9781019c0 2024-09-05T12:09:22.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.601+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc96c0046c0 con 0x7fc9781019c0 2024-09-05T12:09:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.602+0000 7fc976000640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 0x7fc9600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.603+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fc96c0cf900 con 0x7fc9781019c0 2024-09-05T12:09:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.603+0000 7fc97ca00640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 0x7fc9600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.603+0000 7fc97ca00640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 0x7fc9600a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc9640095f0 tx=0x7fc964005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.605+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc96c09c3f0 con 0x7fc9781019c0 2024-09-05T12:09:22.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.825+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc97802d7c0 con 0x7fc9781019c0 2024-09-05T12:09:22.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.826+0000 7fc976000640 1 -- 172.21.15.59:0/2555570049 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fc96c09bcf0 con 0x7fc9781019c0 2024-09-05T12:09:22.828 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:09:22.828 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:09:22.828 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:09:22.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 msgr2=0x7fc9600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:22.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 0x7fc9600a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc9640095f0 tx=0x7fc964005f90 comp rx=0 tx=0).stop 2024-09-05T12:09:22.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 msgr2=0x7fc9780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e9f0 tx=0x7fc96c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 shutdown_connections 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fc9600a2b80 0x7fc9600a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9781079c0 0x7fc978074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc978102360 0x7fc978074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 --2- 172.21.15.59:0/2555570049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9781019c0 0x7fc9780744f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 >> 172.21.15.59:0/2555570049 conn(0x7fc9780fd740 msgr2=0x7fc9780fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 shutdown_connections 2024-09-05T12:09:22.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:22.829+0000 7fc97e800640 1 -- 172.21.15.59:0/2555570049 wait complete. 2024-09-05T12:09:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:22 smithi089 ceph-mon[34499]: pgmap v745: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:22 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2555570049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:23.185 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:22 smithi059 ceph-mon[30025]: pgmap v745: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:23.185 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:22 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2555570049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:23.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:22 smithi189 ceph-mon[34772]: pgmap v745: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:23.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:22 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2555570049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:23.688 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:09:23.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:09:24.063 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.725+0000 7f23bb200640 1 -- 172.21.15.59:0/3376277076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc1010d0 msgr2=0x7f23bc10dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 --2- 172.21.15.59:0/3376277076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc1010d0 0x7f23bc10dd90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009900 tx=0x7f23a802a410 comp rx=0 tx=0).stop 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 -- 172.21.15.59:0/3376277076 shutdown_connections 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 --2- 172.21.15.59:0/3376277076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc10e2d0 0x7f23bc1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 --2- 172.21.15.59:0/3376277076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc1010d0 0x7f23bc10dd90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 --2- 172.21.15.59:0/3376277076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 -- 172.21.15.59:0/3376277076 >> 172.21.15.59:0/3376277076 conn(0x7f23bc0fc610 msgr2=0x7f23bc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 -- 172.21.15.59:0/3376277076 shutdown_connections 2024-09-05T12:09:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.726+0000 7f23bb200640 1 -- 172.21.15.59:0/3376277076 wait complete. 2024-09-05T12:09:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 Processor -- start 2024-09-05T12:09:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 -- start start 2024-09-05T12:09:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc1010d0 0x7f23bc10ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc10e2d0 0x7f23bc10b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114020 con 0x7f23bc10e2d0 2024-09-05T12:09:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc113ea0 con 0x7f23bc100720 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.727+0000 7f23bb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1141a0 con 0x7f23bc1010d0 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:60792/0 (socket says 172.21.15.59:60792) 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 -- 172.21.15.59:0/1114600154 learned_addr learned my addr 172.21.15.59:0/1114600154 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23ba800640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc10e2d0 0x7f23bc10b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9400640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc1010d0 0x7f23bc10ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 -- 172.21.15.59:0/1114600154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc1010d0 msgr2=0x7f23bc10ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc1010d0 0x7f23bc10ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 -- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc10e2d0 msgr2=0x7f23bc10b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc10e2d0 0x7f23bc10b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.728+0000 7f23b9e00640 1 -- 172.21.15.59:0/1114600154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc10bb30 con 0x7f23bc100720 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23b9e00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00d950 tx=0x7f23ac00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23b2a00640 1 -- 172.21.15.59:0/1114600154 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac017030 con 0x7f23bc100720 2024-09-05T12:09:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23b2a00640 1 -- 172.21.15.59:0/1114600154 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ac004020 con 0x7f23bc100720 2024-09-05T12:09:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23b2a00640 1 -- 172.21.15.59:0/1114600154 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0047a0 con 0x7f23bc100720 2024-09-05T12:09:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc1aae80 con 0x7f23bc100720 2024-09-05T12:09:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.729+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc1ab340 con 0x7f23bc100720 2024-09-05T12:09:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.732+0000 7f23c3683740 1 Processor -- start 2024-09-05T12:09:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 -- start start 2024-09-05T12:09:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23b2a00640 1 -- 172.21.15.59:0/1114600154 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23ac0041d0 con 0x7f23bc100720 2024-09-05T12:09:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 0x55c8c17b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 0x55c8c17be200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1719530 con 0x55c8c17be740 2024-09-05T12:09:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8c16c0860 con 0x55c8c17b3560 2024-09-05T12:09:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23c3683740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8c17154f0 con 0x55c8c17b3e80 2024-09-05T12:09:24.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.733+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 0x55c8c17be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 0x55c8c17be200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:35908/0 (socket says 172.21.15.59:35908) 2024-09-05T12:09:24.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23b9e00640 1 -- 172.21.15.59:0/1671874145 learned_addr learned my addr 172.21.15.59:0/1671874145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 0x55c8c17b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23b9400640 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 -- 172.21.15.59:0/1671874145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 msgr2=0x55c8c17be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 0x55c8c17be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 -- 172.21.15.59:0/1671874145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 msgr2=0x55c8c17c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17c0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.734+0000 7f23ba800640 1 -- 172.21.15.59:0/1671874145 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c148ab90 con 0x55c8c17b3560 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23ba800640 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 0x55c8c17b3940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f23bc069750 tx=0x7f23b4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f0c8b7a652af41 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23b2a00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 0x7f23a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23a7000640 1 -- 172.21.15.59:0/1671874145 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4016070 con 0x55c8c17b3560 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23a7000640 1 -- 172.21.15.59:0/1671874145 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4004d20 con 0x55c8c17b3560 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23a7000640 1 -- 172.21.15.59:0/1671874145 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40127b0 con 0x55c8c17b3560 2024-09-05T12:09:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23b2a00640 1 -- 172.21.15.59:0/1114600154 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f23ac0cece0 con 0x7f23bc100720 2024-09-05T12:09:24.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.735+0000 7f23b9e00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 0x7f23a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 -- 172.21.15.59:0/1671874145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 msgr2=0x55c8c17b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 0x55c8c17b3940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f23bc069750 tx=0x7f23b4010740 comp rx=0 tx=0).stop 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23b9e00640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 0x7f23a00a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005fd0 tx=0x7f23ac0d27b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 -- 172.21.15.59:0/1671874145 shutdown_connections 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3e80 0x55c8c17be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 --2- 172.21.15.59:0/1671874145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3560 0x55c8c17b3940 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 -- 172.21.15.59:0/1671874145 >> 172.21.15.59:0/1671874145 conn(0x55c8c17afaf0 msgr2=0x55c8c17afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 -- 172.21.15.59:0/1671874145 shutdown_connections 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.736+0000 7f23c3683740 1 -- 172.21.15.59:0/1671874145 wait complete. 2024-09-05T12:09:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 -- start start 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 Processor -- start 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c8c17c1310 con 0x55c8c17be740 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c8c16d3e60 con 0x55c8c17b3e80 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8c17c1490 con 0x55c8c17b3560 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.737+0000 7f23c3683740 10 client.0 _ll_register_callbacks cb 0x55c8c16d8cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:60816/0 (socket says 172.21.15.59:60816) 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 -- 172.21.15.59:0/1006394095 learned_addr learned my addr 172.21.15.59:0/1006394095 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23ba800640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 -- 172.21.15.59:0/1006394095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 msgr2=0x55c8c17bb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17bb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 -- 172.21.15.59:0/1006394095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 msgr2=0x55c8c17bc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17bc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.738+0000 7f23b9400640 1 -- 172.21.15.59:0/1006394095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c16df990 con 0x55c8c17b3e80 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f23b9400640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23a8031550 tx=0x7f23a8004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c311f40c933b78c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f239b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8035040 con 0x55c8c17b3e80 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b4004d20 con 0x55c8c17b3e80 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f23a8005450 con 0x55c8c17b3e80 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f23a8005650 con 0x55c8c17b3e80 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f23c3683740 10 client.27221 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8c17ccfd0 con 0x55c8c17b3e80 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.739+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4004d20 con 0x55c8c17b3e80 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f23a80039f0 con 0x55c8c17b3e80 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 10 client.27221 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 10 client.27221 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 20 client.27221 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c17ce3e0 0x55c8c17d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 4 client.27221 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c16bc310 con 0x55c8c17ce3e0 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.740+0000 7f23ba800640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c17ce3e0 0x55c8c17d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.741+0000 7f23ba800640 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c17ce3e0 0x55c8c17d0800 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b94d41c41921fb server_cookie=b0e1449fe73e78e0 in_seq=0 out_seq=0 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.741+0000 7f239b400640 10 client.27221 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f239b400640 1 -- 172.21.15.59:0/1006394095 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8c16bc310 con 0x55c8c17ce3e0 2024-09-05T12:09:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f239b400640 10 client.27221 handle_command_reply: tid=0 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 client.27221 shutdown 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 msgr2=0x55c8c17bbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23a8031550 tx=0x7f23a8004440 comp rx=0 tx=0).stop 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 shutdown_connections 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c17ce3e0 0x55c8c17d0800 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c17bc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbbb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 --2- 172.21.15.59:0/1006394095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f239b400640 0 client.27221 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 >> 172.21.15.59:0/1006394095 conn(0x55c8c17afaf0 msgr2=0x55c8c17aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 shutdown_connections 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.742+0000 7f23c3683740 1 -- 172.21.15.59:0/1006394095 wait complete. 2024-09-05T12:09:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.743+0000 7f23c3683740 20 client.27221 trim_cache size 0 max 16384 2024-09-05T12:09:24.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.760+0000 7f23c3683740 1 Processor -- start 2024-09-05T12:09:24.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.760+0000 7f23c3683740 1 -- start start 2024-09-05T12:09:24.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17be740 0x55c8c17d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b3440 con 0x55c8c17b3e80 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b5cc0 con 0x55c8c17be740 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23c3683740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b3ab0 con 0x55c8c17b3560 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23ba800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49142/0 (socket says 172.21.15.59:49142) 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23b9400640 1 -- 172.21.15.59:0/254297767 learned_addr learned my addr 172.21.15.59:0/254297767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:24.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.761+0000 7f23ba800640 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17be740 0x55c8c17d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 -- 172.21.15.59:0/254297767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 msgr2=0x55c8c17aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 -- 172.21.15.59:0/254297767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17be740 msgr2=0x55c8c17d60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17be740 0x55c8c17d60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 -- 172.21.15.59:0/254297767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c17b3cb0 con 0x55c8c17b3e80 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23ba800640 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:24.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23b9400640 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f23a802a930 tx=0x7f23a8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cede98b3b87287 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23a6600640 1 -- 172.21.15.59:0/254297767 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8057070 con 0x55c8c17b3e80 2024-09-05T12:09:24.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23a6600640 1 -- 172.21.15.59:0/254297767 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a8040070 con 0x55c8c17b3e80 2024-09-05T12:09:24.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.762+0000 7f23a6600640 1 -- 172.21.15.59:0/254297767 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8035030 con 0x55c8c17b3e80 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 -- 172.21.15.59:0/254297767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 msgr2=0x55c8c17bbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f23a802a930 tx=0x7f23a8001e50 comp rx=0 tx=0).stop 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 -- 172.21.15.59:0/254297767 shutdown_connections 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17be740 0x55c8c17d60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17b3e80 0x55c8c17bbf40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 --2- 172.21.15.59:0/254297767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.763+0000 7f23c3683740 1 -- 172.21.15.59:0/254297767 >> 172.21.15.59:0/254297767 conn(0x55c8c17afaf0 msgr2=0x55c8c17ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.764+0000 7f23c3683740 1 -- 172.21.15.59:0/254297767 shutdown_connections 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.764+0000 7f23c3683740 1 -- 172.21.15.59:0/254297767 wait complete. 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.764+0000 7f23c3683740 1 -- start start 2024-09-05T12:09:24.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.764+0000 7f23c3683740 1 Processor -- start 2024-09-05T12:09:24.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b4180 con 0x55c8c17be740 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b55f0 con 0x55c8c17b3e80 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1b4850 con 0x55c8c17b3560 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23c3683740 10 client.0 _ll_register_callbacks cb 0x7f23a8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.765+0000 7f23ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49150/0 (socket says 172.21.15.59:49150) 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 -- 172.21.15.59:0/3508731775 learned_addr learned my addr 172.21.15.59:0/3508731775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23b9400640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 -- 172.21.15.59:0/3508731775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 msgr2=0x55c8c17c27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17c27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 -- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 msgr2=0x55c8c17c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 -- 172.21.15.59:0/3508731775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c17c3570 con 0x55c8c17be740 2024-09-05T12:09:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f23ba800640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009fd0 tx=0x7f23a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2ce314db2ebae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.766+0000 7f239b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4016040 con 0x55c8c17be740 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f23b4003b70 con 0x55c8c17be740 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a8040070 con 0x55c8c17be740 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f23b4023040 con 0x55c8c17be740 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f23c3683740 10 client.17226 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8c1839690 con 0x55c8c17be740 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a8040070 con 0x55c8c17be740 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.767+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f23b4033070 con 0x55c8c17be740 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 10 client.17226 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 10 client.17226 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 20 client.17226 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c1839be0 0x55c8c183bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 4 client.17226 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c8c1726c60 con 0x55c8c1839be0 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.768+0000 7f23b9400640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c1839be0 0x55c8c183bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.769+0000 7f23b9400640 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c1839be0 0x55c8c183bfd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef25d0bb7afe11e server_cookie=1ff42369aab1dd12 in_seq=0 out_seq=0 2024-09-05T12:09:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.769+0000 7f239b400640 10 client.17226 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:24.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f239b400640 1 -- 172.21.15.59:0/3508731775 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c8c1726c60 con 0x55c8c1839be0 2024-09-05T12:09:24.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f239b400640 10 client.17226 handle_command_reply: tid=0 2024-09-05T12:09:24.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 client.17226 shutdown 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 msgr2=0x55c8c18380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009fd0 tx=0x7f23a8004040 comp rx=0 tx=0).stop 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 shutdown_connections 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8c1839be0 0x55c8c183bfd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c8c17be740 0x55c8c18380a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8c17b3e80 0x55c8c17c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 --2- 172.21.15.59:0/3508731775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8c17b3560 0x55c8c17c27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 >> 172.21.15.59:0/3508731775 conn(0x55c8c17afaf0 msgr2=0x55c8c17aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f239b400640 0 client.17226 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 shutdown_connections 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.776+0000 7f23c3683740 1 -- 172.21.15.59:0/3508731775 wait complete. 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.777+0000 7f23c3683740 20 client.17226 trim_cache size 0 max 16384 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:09:24.779 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 42, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "b68fb54b": { 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 42, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:09:24.780 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:24.781 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.777+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 msgr2=0x7f23a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.777+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 0x7f23a00a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005fd0 tx=0x7f23ac0d27b0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.777+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 msgr2=0x7f23bc10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.777+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00d950 tx=0x7f23ac00de10 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 shutdown_connections 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f23a00a2c00 0x7f23a00a50c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23bc10e2d0 0x7f23bc10b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23bc1010d0 0x7f23bc10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 --2- 172.21.15.59:0/1114600154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23bc100720 0x7f23bc10dc30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 >> 172.21.15.59:0/1114600154 conn(0x7f23bc0fc610 msgr2=0x7f23bc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 shutdown_connections 2024-09-05T12:09:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:24.778+0000 7f23bb200640 1 -- 172.21.15.59:0/1114600154 wait complete. 2024-09-05T12:09:25.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:24 smithi059 ceph-mon[30025]: pgmap v746: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-09-05T12:09:25.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:24 smithi089 ceph-mon[34499]: pgmap v746: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-09-05T12:09:25.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:24 smithi189 ceph-mon[34772]: pgmap v746: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-09-05T12:09:25.601 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b68fb54b', quiesce took 4.0 seconds. Will release after: 26.0 2024-09-05T12:09:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:26 smithi089 ceph-mon[34499]: pgmap v747: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:09:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:26 smithi059 ceph-mon[30025]: pgmap v747: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:09:27.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:26 smithi189 ceph-mon[34772]: pgmap v747: 129 pgs: 129 active+clean; 5.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-05T12:09:29.140 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:28 smithi189 ceph-mon[34772]: pgmap v748: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:28 smithi089 ceph-mon[34499]: pgmap v748: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:28 smithi059 ceph-mon[30025]: pgmap v748: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:30.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:09:30.935 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:30 smithi089 ceph-mon[34499]: pgmap v749: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 3 op/s 2024-09-05T12:09:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:30 smithi059 ceph-mon[30025]: pgmap v749: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 3 op/s 2024-09-05T12:09:31.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:30 smithi189 ceph-mon[34772]: pgmap v749: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 3 op/s 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 -- 172.21.15.59:0/4239621499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8112a50 msgr2=0x7f45d8114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 --2- 172.21.15.59:0/4239621499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8112a50 0x7f45d8114e40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009ca0 tx=0x7f45d002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 -- 172.21.15.59:0/4239621499 shutdown_connections 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 --2- 172.21.15.59:0/4239621499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8112a50 0x7f45d8114e40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 --2- 172.21.15.59:0/4239621499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8077040 0x7f45d8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 --2- 172.21.15.59:0/4239621499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8076720 0x7f45d8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 -- 172.21.15.59:0/4239621499 >> 172.21.15.59:0/4239621499 conn(0x7f45d80ffa50 msgr2=0x7f45d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 -- 172.21.15.59:0/4239621499 shutdown_connections 2024-09-05T12:09:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.743+0000 7f45de800640 1 -- 172.21.15.59:0/4239621499 wait complete. 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.744+0000 7f45de800640 1 Processor -- start 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.744+0000 7f45de800640 1 -- start start 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8077040 0x7f45d819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 0x7f45d81a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8118600 con 0x7f45d8077040 2024-09-05T12:09:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8118480 con 0x7f45d8112a50 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45de800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8118780 con 0x7f45d8076720 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8077040 0x7f45d819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41218/0 (socket says 172.21.15.59:41218) 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 -- 172.21.15.59:0/3012500608 learned_addr learned my addr 172.21.15.59:0/3012500608 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7e00640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 0x7f45d81a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 -- 172.21.15.59:0/3012500608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 msgr2=0x7f45d81a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 0x7f45d81a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 -- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8077040 msgr2=0x7f45d819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.745+0000 7f45d7400640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8077040 0x7f45d819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d7400640 1 -- 172.21.15.59:0/3012500608 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81a3bb0 con 0x7f45d8076720 2024-09-05T12:09:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d7e00640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 0x7f45d81a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d7400640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e9f0 tx=0x7f45c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d4c00640 1 -- 172.21.15.59:0/3012500608 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800cd80 con 0x7f45d8076720 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d81a3ea0 con 0x7f45d8076720 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d4c00640 1 -- 172.21.15.59:0/3012500608 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8010910 con 0x7f45d8076720 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45d4c00640 1 -- 172.21.15.59:0/3012500608 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80188f0 con 0x7f45d8076720 2024-09-05T12:09:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.746+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81ab730 con 0x7f45d8076720 2024-09-05T12:09:31.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.750+0000 7f45e128d740 1 Processor -- start 2024-09-05T12:09:31.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.750+0000 7f45e128d740 1 -- start start 2024-09-05T12:09:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2dff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e02160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fec2d5b3e0 con 0x55fec2df4b90 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fec2d02d80 con 0x55fec2dffd70 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45e128d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fec2d573a0 con 0x55fec2df54b0 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45d4c00640 1 -- 172.21.15.59:0/3012500608 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45c8010ac0 con 0x7f45d8076720 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46868/0 (socket says 172.21.15.59:46868) 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45d7e00640 1 -- 172.21.15.59:0/3893612417 learned_addr learned my addr 172.21.15.59:0/3893612417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:31.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.751+0000 7f45cfe00640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e02160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 -- 172.21.15.59:0/3893612417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 msgr2=0x55fec2dff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7400640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2dff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2dff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 -- 172.21.15.59:0/3893612417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 msgr2=0x55fec2e02160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e02160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 -- 172.21.15.59:0/3893612417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec2d21ae0 con 0x55fec2df4b90 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7400640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2dff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:09:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45cfe00640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e02160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:31.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45d7e00640 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f45d807aa10 tx=0x7f45d0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61742c051e1cf4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45ccc00640 1 -- 172.21.15.59:0/3893612417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0030720 con 0x55fec2df4b90 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45ccc00640 1 -- 172.21.15.59:0/3893612417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0036040 con 0x55fec2df4b90 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.752+0000 7f45ccc00640 1 -- 172.21.15.59:0/3893612417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004460 con 0x55fec2df4b90 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45d4c00640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 -- 172.21.15.59:0/3893612417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 msgr2=0x55fec2df4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f45d807aa10 tx=0x7f45d0009a20 comp rx=0 tx=0).stop 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45d4c00640 1 -- 172.21.15.59:0/3012500608 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45c80cf8f0 con 0x7f45d8076720 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 -- 172.21.15.59:0/3893612417 shutdown_connections 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e02160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2dff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 --2- 172.21.15.59:0/3893612417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df4f70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 -- 172.21.15.59:0/3893612417 >> 172.21.15.59:0/3893612417 conn(0x55fec2df1120 msgr2=0x55fec2df1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45d7400640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 -- 172.21.15.59:0/3893612417 shutdown_connections 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.753+0000 7f45e128d740 1 -- 172.21.15.59:0/3893612417 wait complete. 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.754+0000 7f45e128d740 1 -- start start 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.754+0000 7f45e128d740 1 Processor -- start 2024-09-05T12:09:31.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.754+0000 7f45d7400640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c6a0 tx=0x7f45c801f000 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2dfd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2dffd70 0x55fec2dfd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fec2e02980 con 0x55fec2df4b90 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fec2d160f0 con 0x55fec2df54b0 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fec2e02b00 con 0x55fec2dffd70 2024-09-05T12:09:31.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45e128d740 10 client.0 _ll_register_callbacks cb 0x55fec2d079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46880/0 (socket says 172.21.15.59:46880) 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45d7e00640 1 -- 172.21.15.59:0/1252092722 learned_addr learned my addr 172.21.15.59:0/1252092722 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2dfd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45d7e00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2dffd70 0x55fec2dfd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 -- 172.21.15.59:0/1252092722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2dffd70 msgr2=0x55fec2dfd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2dffd70 0x55fec2dfd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 -- 172.21.15.59:0/1252092722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 msgr2=0x55fec2dfccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45cfe00640 1 -- 172.21.15.59:0/1252092722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec2acc3c0 con 0x55fec2df54b0 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.755+0000 7f45d7e00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:31.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.756+0000 7f45cfe00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2dfd1f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45d80792f0 tx=0x7f45c0012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e63750e2da7a91c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.756+0000 7f45c5600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.756+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0005020 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.756+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d0036040 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.756+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45c00051c0 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45c00053c0 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 10 client.27239 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fec2e0e650 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0036040 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f45c000f820 con 0x55fec2df54b0 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 10 client.27239 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 20 client.27239 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:31.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e0fa60 0x55fec2e11e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 4 client.27239 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:31.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.757+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fec2d68950 con 0x55fec2e0fa60 2024-09-05T12:09:31.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.758+0000 7f45d7e00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e0fa60 0x55fec2e11e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.758+0000 7f45d7e00640 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e0fa60 0x55fec2e11e70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc46a7a71474f66 server_cookie=f70407fff7461f83 in_seq=0 out_seq=0 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.758+0000 7f45c5600640 10 client.27239 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.759+0000 7f45c5600640 1 -- 172.21.15.59:0/1252092722 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fec2d68950 con 0x55fec2e0fa60 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.759+0000 7f45c5600640 10 client.27239 handle_command_reply: tid=0 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.759+0000 7f45e128d740 1 client.27239 shutdown 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 msgr2=0x55fec2dfd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2dfd1f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45d80792f0 tx=0x7f45c0012a80 comp rx=0 tx=0).stop 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 shutdown_connections 2024-09-05T12:09:31.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2dffd70 0x55fec2dfd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2dfd1f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e0fa60 0x55fec2e11e70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 --2- 172.21.15.59:0/1252092722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2dfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 >> 172.21.15.59:0/1252092722 conn(0x55fec2df1120 msgr2=0x55fec2df6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45c5600640 0 client.27239 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 shutdown_connections 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 1 -- 172.21.15.59:0/1252092722 wait complete. 2024-09-05T12:09:31.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.760+0000 7f45e128d740 20 client.27239 trim_cache size 0 max 16384 2024-09-05T12:09:31.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.778+0000 7f45e128d740 1 Processor -- start 2024-09-05T12:09:31.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.778+0000 7f45e128d740 1 -- start start 2024-09-05T12:09:31.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2e17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e17c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b35b0 con 0x55fec2df4b90 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b5e30 con 0x55fec2dffd70 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45e128d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b3c20 con 0x55fec2df54b0 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46888/0 (socket says 172.21.15.59:46888) 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 -- 172.21.15.59:0/53862919 learned_addr learned my addr 172.21.15.59:0/53862919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:31.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45cfe00640 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2e17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e17c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 -- 172.21.15.59:0/53862919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 msgr2=0x55fec2e17740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2e17740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 -- 172.21.15.59:0/53862919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 msgr2=0x55fec2e17c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e17c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.779+0000 7f45d7e00640 1 -- 172.21.15.59:0/53862919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec2df52e0 con 0x55fec2df4b90 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.780+0000 7f45d7e00640 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009ca0 tx=0x7f45d0030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec1cbe80f5f5919 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.780+0000 7f45c7e00640 1 -- 172.21.15.59:0/53862919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0030ad0 con 0x55fec2df4b90 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.780+0000 7f45c7e00640 1 -- 172.21.15.59:0/53862919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0036030 con 0x55fec2df4b90 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.780+0000 7f45c7e00640 1 -- 172.21.15.59:0/53862919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004600 con 0x55fec2df4b90 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- 172.21.15.59:0/53862919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 msgr2=0x55fec2df75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009ca0 tx=0x7f45d0030f90 comp rx=0 tx=0).stop 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- 172.21.15.59:0/53862919 shutdown_connections 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2dffd70 0x55fec2e17c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df54b0 0x55fec2e17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 --2- 172.21.15.59:0/53862919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2df4b90 0x55fec2df75f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- 172.21.15.59:0/53862919 >> 172.21.15.59:0/53862919 conn(0x55fec2df1120 msgr2=0x55fec2df01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- 172.21.15.59:0/53862919 shutdown_connections 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- 172.21.15.59:0/53862919 wait complete. 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 -- start start 2024-09-05T12:09:31.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.781+0000 7f45e128d740 1 Processor -- start 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df4b90 0x55fec2e03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2e04410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b42f0 con 0x55fec2dffd70 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b5760 con 0x55fec2df54b0 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b49c0 con 0x55fec2df4b90 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45e128d740 10 client.0 _ll_register_callbacks cb 0x7f45c0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46892/0 (socket says 172.21.15.59:46892) 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45d7e00640 1 -- 172.21.15.59:0/487292101 learned_addr learned my addr 172.21.15.59:0/487292101 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45cfe00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2e04410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.782+0000 7f45d7e00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df4b90 0x55fec2e03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 -- 172.21.15.59:0/487292101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df4b90 msgr2=0x55fec2e03ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df4b90 0x55fec2e03ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 -- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 msgr2=0x55fec2e04410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2e04410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 -- 172.21.15.59:0/487292101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec2e79c50 con 0x55fec2dffd70 2024-09-05T12:09:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45cfe00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2e04410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45d7e00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009960 tx=0x7f45d00041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fa09bd4f43d2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45c5600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0030cf0 con 0x55fec2dffd70 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d0036030 con 0x55fec2dffd70 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.783+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45d0004b40 con 0x55fec2dffd70 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45d0004280 con 0x55fec2dffd70 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 10 client.17244 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fec2e7ae20 con 0x55fec2dffd70 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0036030 con 0x55fec2dffd70 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f45d005c030 con 0x55fec2dffd70 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 10 client.17244 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 20 client.17244 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e7b050 0x55fec2e7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 4 client.17244 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.784+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fec2d62e60 con 0x55fec2e7b050 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45cfe00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e7b050 0x55fec2e7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45cfe00640 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e7b050 0x55fec2e7d440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04d511c66de7e4e server_cookie=9e6279fdace6ab6e in_seq=0 out_seq=0 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45c5600640 10 client.17244 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45c5600640 1 -- 172.21.15.59:0/487292101 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fec2d62e60 con 0x55fec2e7b050 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45c5600640 10 client.17244 handle_command_reply: tid=0 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.785+0000 7f45e128d740 1 client.17244 shutdown 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 msgr2=0x55fec2e79710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009960 tx=0x7f45d00041e0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 shutdown_connections 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fec2e7b050 0x55fec2e7d440 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec2dffd70 0x55fec2e79710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fec2df54b0 0x55fec2e04410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 --2- 172.21.15.59:0/487292101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fec2df4b90 0x55fec2e03ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45c5600640 0 client.17244 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 >> 172.21.15.59:0/487292101 conn(0x55fec2df1120 msgr2=0x55fec2dfcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 shutdown_connections 2024-09-05T12:09:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 1 -- 172.21.15.59:0/487292101 wait complete. 2024-09-05T12:09:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.786+0000 7f45e128d740 20 client.17244 trim_cache size 0 max 16384 2024-09-05T12:09:31.792 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:31.792 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:09:31.792 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:09:31.792 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 msgr2=0x7f45bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f45c800c6a0 tx=0x7f45c801f000 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 msgr2=0x7f45d819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e9f0 tx=0x7f45c800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 shutdown_connections 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d8112a50 0x7f45d81a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45d8077040 0x7f45d819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 --2- 172.21.15.59:0/3012500608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8076720 0x7f45d819eb80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 >> 172.21.15.59:0/3012500608 conn(0x7f45d80ffa50 msgr2=0x7f45d8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 shutdown_connections 2024-09-05T12:09:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:31.787+0000 7f45de800640 1 -- 172.21.15.59:0/3012500608 wait complete. 2024-09-05T12:09:32.674 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:09:32.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:09:32.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:32 smithi059 ceph-mon[30025]: pgmap v750: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:32 smithi089 ceph-mon[34499]: pgmap v750: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:33.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:32 smithi189 ceph-mon[34772]: pgmap v750: 129 pgs: 129 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 -- 172.21.15.59:0/3533299972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 msgr2=0x7fa060101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 --2- 172.21.15.59:0/3533299972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa060101e20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa048009930 tx=0x7fa04802a420 comp rx=0 tx=0).stop 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 -- 172.21.15.59:0/3533299972 shutdown_connections 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 --2- 172.21.15.59:0/3533299972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa060102360 0x7fa06010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 --2- 172.21.15.59:0/3533299972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa060101e20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 --2- 172.21.15.59:0/3533299972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0601079c0 0x7fa060107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 -- 172.21.15.59:0/3533299972 >> 172.21.15.59:0/3533299972 conn(0x7fa0600fd740 msgr2=0x7fa0600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:33.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 -- 172.21.15.59:0/3533299972 shutdown_connections 2024-09-05T12:09:33.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.707+0000 7fa065200640 1 -- 172.21.15.59:0/3533299972 wait complete. 2024-09-05T12:09:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.708+0000 7fa065200640 1 Processor -- start 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.708+0000 7fa065200640 1 -- start start 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa060102360 0x7fa060198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0601079c0 0x7fa06019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa060111e10 con 0x7fa0601019c0 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa060111c90 con 0x7fa0601079c0 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa065200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa060111f90 con 0x7fa060102360 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46910/0 (socket says 172.21.15.59:46910) 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 -- 172.21.15.59:0/999706533 learned_addr learned my addr 172.21.15.59:0/999706533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05fe00640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0601079c0 0x7fa06019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa056a00640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa060102360 0x7fa060198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 -- 172.21.15.59:0/999706533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa060102360 msgr2=0x7fa060198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa060102360 0x7fa060198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 -- 172.21.15.59:0/999706533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0601079c0 msgr2=0x7fa06019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0601079c0 0x7fa06019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.709+0000 7fa05f400640 1 -- 172.21.15.59:0/999706533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019d2a0 con 0x7fa0601019c0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa05f400640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa05000eb90 tx=0x7fa05000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004d20 con 0x7fa0601019c0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05000ee90 con 0x7fa0601019c0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050010830 con 0x7fa0601019c0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06019d590 con 0x7fa0601019c0 2024-09-05T12:09:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.710+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a4e70 con 0x7fa0601019c0 2024-09-05T12:09:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.711+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa060101e20 con 0x7fa0601019c0 2024-09-05T12:09:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.713+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa050005020 con 0x7fa0601019c0 2024-09-05T12:09:33.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.714+0000 7fa05cc00640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:33.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.715+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa0500cfb20 con 0x7fa0601019c0 2024-09-05T12:09:33.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.715+0000 7fa056a00640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:33.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.715+0000 7fa056a00640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.718+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa05009c610 con 0x7fa0601019c0 2024-09-05T12:09:33.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.940+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa060109f50 con 0x7fa0601019c0 2024-09-05T12:09:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.941+0000 7fa05cc00640 1 -- 172.21.15.59:0/999706533 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fa05009bf10 con 0x7fa0601019c0 2024-09-05T12:09:33.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:09:33.943 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:09:33.943 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 msgr2=0x7fa0440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa048005a20 comp rx=0 tx=0).stop 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 msgr2=0x7fa0601982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa05000eb90 tx=0x7fa05000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 shutdown_connections 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0601079c0 0x7fa06019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa060102360 0x7fa060198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 --2- 172.21.15.59:0/999706533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0601019c0 0x7fa0601982f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.943+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 >> 172.21.15.59:0/999706533 conn(0x7fa0600fd740 msgr2=0x7fa0600fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.944+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 shutdown_connections 2024-09-05T12:09:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:33.944+0000 7fa065200640 1 -- 172.21.15.59:0/999706533 wait complete. 2024-09-05T12:09:34.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:09:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:34 smithi089 ceph-mon[34499]: pgmap v751: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:09:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/999706533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:34 smithi059 ceph-mon[30025]: pgmap v751: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:09:35.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/999706533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:34 smithi189 ceph-mon[34772]: pgmap v751: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:09:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/999706533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 -- 172.21.15.59:0/405748262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3181049a0 msgr2=0x7fb3181089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 --2- 172.21.15.59:0/405748262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3181049a0 0x7fb3181089f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb300009900 tx=0x7fb30002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 -- 172.21.15.59:0/405748262 shutdown_connections 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 --2- 172.21.15.59:0/405748262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318109120 0x7fb318110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 --2- 172.21.15.59:0/405748262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3181049a0 0x7fb3181089f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 --2- 172.21.15.59:0/405748262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb318103ff0 0x7fb3181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 -- 172.21.15.59:0/405748262 >> 172.21.15.59:0/405748262 conn(0x7fb3180ffa90 msgr2=0x7fb318101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 -- 172.21.15.59:0/405748262 shutdown_connections 2024-09-05T12:09:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.759+0000 7fb31f000640 1 -- 172.21.15.59:0/405748262 wait complete. 2024-09-05T12:09:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.760+0000 7fb31f000640 1 Processor -- start 2024-09-05T12:09:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.760+0000 7fb31f000640 1 -- start start 2024-09-05T12:09:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 0x7fb31819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 0x7fb31819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb318109120 0x7fb31819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb318114170 con 0x7fb318103ff0 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb318113ff0 con 0x7fb318109120 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181142f0 con 0x7fb3181049a0 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 0x7fb31819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 0x7fb31819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46928/0 (socket says 172.21.15.59:46928) 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb317e00640 1 -- 172.21.15.59:0/3709507673 learned_addr learned my addr 172.21.15.59:0/3709507673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb30f400640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 0x7fb31819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.761+0000 7fb31d000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb318109120 0x7fb31819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 -- 172.21.15.59:0/3709507673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb318109120 msgr2=0x7fb31819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb318109120 0x7fb31819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 -- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 msgr2=0x7fb31819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 0x7fb31819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 -- 172.21.15.59:0/3709507673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31819f600 con 0x7fb3181049a0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb30f400640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 0x7fb31819ab90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb30002a910 tx=0x7fb300003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb315600640 1 -- 172.21.15.59:0/3709507673 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30003f070 con 0x7fb3181049a0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31819f890 con 0x7fb3181049a0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb315600640 1 -- 172.21.15.59:0/3709507673 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb300031c90 con 0x7fb3181049a0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb315600640 1 -- 172.21.15.59:0/3709507673 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300035030 con 0x7fb3181049a0 2024-09-05T12:09:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.762+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3181a71d0 con 0x7fb3181049a0 2024-09-05T12:09:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.765+0000 7fb321a8d740 1 Processor -- start 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.765+0000 7fb321a8d740 1 -- start start 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e87910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8de33e0 con 0x55bcd8e87e50 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8d8ad80 con 0x55bcd8e7cc70 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb321a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8ddf3a0 con 0x55bcd8e7d590 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb315600640 1 -- 172.21.15.59:0/3709507673 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb300042070 con 0x7fb3181049a0 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb31d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb317e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e87910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb31d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45460/0 (socket says 172.21.15.59:45460) 2024-09-05T12:09:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb31d000640 1 -- 172.21.15.59:0/124061235 learned_addr learned my addr 172.21.15.59:0/124061235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.766+0000 7fb30f400640 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 -- 172.21.15.59:0/124061235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 msgr2=0x55bcd8e87910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e87910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 -- 172.21.15.59:0/124061235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 msgr2=0x55bcd8e8a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 -- 172.21.15.59:0/124061235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd8da9ae0 con 0x55bcd8e7cc70 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb30f400640 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.767+0000 7fb31d000640 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb310009f50 tx=0x7fb3100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dc15d4caca4d670 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb30d600640 1 -- 172.21.15.59:0/124061235 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310004d20 con 0x55bcd8e7cc70 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb30d600640 1 -- 172.21.15.59:0/124061235 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310004050 con 0x55bcd8e7cc70 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb30d600640 1 -- 172.21.15.59:0/124061235 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310005280 con 0x55bcd8e7cc70 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb315600640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 0x7fb2fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb315600640 1 -- 172.21.15.59:0/3709507673 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb3000efa30 con 0x7fb3181049a0 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb317e00640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 0x7fb2fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb321a8d740 1 -- 172.21.15.59:0/124061235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 msgr2=0x55bcd8e7d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.768+0000 7fb321a8d740 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb310009f50 tx=0x7fb3100118d0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 -- 172.21.15.59:0/124061235 shutdown_connections 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e87910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 --2- 172.21.15.59:0/124061235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e7d050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 -- 172.21.15.59:0/124061235 >> 172.21.15.59:0/124061235 conn(0x55bcd8e79200 msgr2=0x55bcd8e795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 -- 172.21.15.59:0/124061235 shutdown_connections 2024-09-05T12:09:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 -- 172.21.15.59:0/124061235 wait complete. 2024-09-05T12:09:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb317e00640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 0x7fb2fc0a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb308009620 tx=0x7fb308005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.769+0000 7fb321a8d740 1 -- start start 2024-09-05T12:09:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 Processor -- start 2024-09-05T12:09:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e84d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8e8aa20 con 0x55bcd8e7cc70 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8d9e0f0 con 0x55bcd8e7d590 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bcd8e8aba0 con 0x55bcd8e87e50 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb321a8d740 10 client.0 _ll_register_callbacks cb 0x55bcd8d85bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb30f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.770+0000 7fb317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e84d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45472/0 (socket says 172.21.15.59:45472) 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 -- 172.21.15.59:0/1387246054 learned_addr learned my addr 172.21.15.59:0/1387246054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb31d000640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 -- 172.21.15.59:0/1387246054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 msgr2=0x55bcd8e85800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e85800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 -- 172.21.15.59:0/1387246054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 msgr2=0x55bcd8e84d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e84d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 -- 172.21.15.59:0/1387246054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd8b543c0 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb30f400640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb3000bbe60 tx=0x7fb300031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7702e50ce261fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb306000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300049050 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb3000b8540 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.771+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb310004050 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb3000bb830 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 10 client.27263 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bcd8e96ab0 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310004050 con 0x55bcd8e7d590 2024-09-05T12:09:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb3000bc130 con 0x55bcd8e7d590 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 10 client.27263 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 20 client.27263 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8e97f10 0x55bcd8e9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 4 client.27263 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.772+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd8df0950 con 0x55bcd8e97f10 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.773+0000 7fb317e00640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8e97f10 0x55bcd8e9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.773+0000 7fb317e00640 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8e97f10 0x55bcd8e9a3d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84431eac944cb32 server_cookie=6a320f6fd06039e7 in_seq=0 out_seq=0 2024-09-05T12:09:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.773+0000 7fb306000640 10 client.27263 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:35.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb306000640 1 -- 172.21.15.59:0/1387246054 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcd8df0950 con 0x55bcd8e97f10 2024-09-05T12:09:35.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb306000640 10 client.27263 handle_command_reply: tid=0 2024-09-05T12:09:35.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 client.27263 shutdown 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 msgr2=0x55bcd8e852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb3000bbe60 tx=0x7fb300031490 comp rx=0 tx=0).stop 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 shutdown_connections 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e85800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8e852c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8e97f10 0x55bcd8e9a3d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1387246054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e84d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb306000640 0 client.27263 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 >> 172.21.15.59:0/1387246054 conn(0x55bcd8e79200 msgr2=0x55bcd8e7eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 shutdown_connections 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.774+0000 7fb321a8d740 1 -- 172.21.15.59:0/1387246054 wait complete. 2024-09-05T12:09:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.775+0000 7fb321a8d740 20 client.27263 trim_cache size 0 max 16384 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.788+0000 7fb321a8d740 1 Processor -- start 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.788+0000 7fb321a8d740 1 -- start start 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8ea01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e81fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181af060 con 0x55bcd8e7cc70 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b18e0 con 0x55bcd8e7d590 2024-09-05T12:09:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb321a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181af6d0 con 0x55bcd8e87e50 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46948/0 (socket says 172.21.15.59:46948) 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb317e00640 1 -- 172.21.15.59:0/134918947 learned_addr learned my addr 172.21.15.59:0/134918947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8ea01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb31d000640 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e81fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 -- 172.21.15.59:0/134918947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 msgr2=0x55bcd8e81fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e81fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 -- 172.21.15.59:0/134918947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 msgr2=0x55bcd8e9fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb30f400640 1 -- 172.21.15.59:0/134918947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd8e74770 con 0x55bcd8e7d590 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.789+0000 7fb317e00640 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb30f400640 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8ea01e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb3000bbbd0 tx=0x7fb30002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769e25bab7075af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb30cc00640 1 -- 172.21.15.59:0/134918947 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300049070 con 0x55bcd8e7d590 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb30cc00640 1 -- 172.21.15.59:0/134918947 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3000c4a90 con 0x55bcd8e7d590 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb30cc00640 1 -- 172.21.15.59:0/134918947 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000bc130 con 0x55bcd8e7d590 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb321a8d740 1 -- 172.21.15.59:0/134918947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 msgr2=0x55bcd8ea01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.790+0000 7fb321a8d740 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8ea01e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb3000bbbd0 tx=0x7fb30002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 -- 172.21.15.59:0/134918947 shutdown_connections 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e87e50 0x55bcd8e81fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7d590 0x55bcd8ea01e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 --2- 172.21.15.59:0/134918947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 -- 172.21.15.59:0/134918947 >> 172.21.15.59:0/134918947 conn(0x55bcd8e79200 msgr2=0x55bcd8e86e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 -- 172.21.15.59:0/134918947 shutdown_connections 2024-09-05T12:09:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 -- 172.21.15.59:0/134918947 wait complete. 2024-09-05T12:09:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 -- start start 2024-09-05T12:09:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.791+0000 7fb321a8d740 1 Processor -- start 2024-09-05T12:09:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e8c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181afda0 con 0x55bcd8e87e50 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b1210 con 0x55bcd8e7cc70 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b0470 con 0x55bcd8e7d590 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb321a8d740 10 client.0 _ll_register_callbacks cb 0x7fb300009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb31d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb317e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb30f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e8c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb317e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45490/0 (socket says 172.21.15.59:45490) 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.792+0000 7fb317e00640 1 -- 172.21.15.59:0/1947636616 learned_addr learned my addr 172.21.15.59:0/1947636616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 -- 172.21.15.59:0/1947636616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 msgr2=0x55bcd8e8c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e8c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 -- 172.21.15.59:0/1947636616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 msgr2=0x55bcd8e8c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 -- 172.21.15.59:0/1947636616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd8e7a2b0 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb317e00640 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb3080045c0 tx=0x7fb3080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1264d7246b3e3292 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb306000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308003cb0 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3000c4a90 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb308003e50 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb30803c050 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb321a8d740 10 client.27269 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.793+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bcd8f02e30 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3000c4a90 con 0x55bcd8e7cc70 2024-09-05T12:09:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb308004cc0 con 0x55bcd8e7cc70 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 10 client.27269 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 10 client.27269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 20 client.27269 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8f03560 0x55bcd8f05980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 4 client.27269 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.794+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcd8deae60 con 0x55bcd8f03560 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb30f400640 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8f03560 0x55bcd8f05980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb30f400640 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8f03560 0x55bcd8f05980 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ed55ae2775785a server_cookie=c18b9e2b644f1072 in_seq=0 out_seq=0 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb306000640 10 client.27269 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb306000640 1 -- 172.21.15.59:0/1947636616 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcd8deae60 con 0x55bcd8f03560 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb306000640 10 client.27269 handle_command_reply: tid=0 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb321a8d740 1 client.27269 shutdown 2024-09-05T12:09:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 msgr2=0x55bcd8e8bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb3080045c0 tx=0x7fb3080046c0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.795+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 shutdown_connections 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bcd8f03560 0x55bcd8f05980 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcd8e87e50 0x55bcd8e8c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bcd8e7d590 0x55bcd8e8c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 --2- 172.21.15.59:0/1947636616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcd8e7cc70 0x55bcd8e8bba0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb306000640 0 client.27269 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 >> 172.21.15.59:0/1947636616 conn(0x55bcd8e79200 msgr2=0x55bcd8e84d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 shutdown_connections 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 1 -- 172.21.15.59:0/1947636616 wait complete. 2024-09-05T12:09:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb321a8d740 20 client.27269 trim_cache size 0 max 16384 2024-09-05T12:09:35.802 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:09:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 msgr2=0x7fb2fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 0x7fb2fc0a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb308009620 tx=0x7fb308005f90 comp rx=0 tx=0).stop 2024-09-05T12:09:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 msgr2=0x7fb31819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.796+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 0x7fb31819ab90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb30002a910 tx=0x7fb300003b20 comp rx=0 tx=0).stop 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 shutdown_connections 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fb2fc0a2b80 0x7fb2fc0a5040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb318109120 0x7fb31819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3181049a0 0x7fb31819ab90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 --2- 172.21.15.59:0/3709507673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb318103ff0 0x7fb31819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 >> 172.21.15.59:0/3709507673 conn(0x7fb3180ffa90 msgr2=0x7fb318100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 shutdown_connections 2024-09-05T12:09:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:35.797+0000 7fb31f000640 1 -- 172.21.15.59:0/3709507673 wait complete. 2024-09-05T12:09:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:36 smithi089 ceph-mon[34499]: pgmap v752: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-05T12:09:37.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:36 smithi059 ceph-mon[30025]: pgmap v752: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-05T12:09:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:36 smithi189 ceph-mon[34772]: pgmap v752: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-09-05T12:09:37.601 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:09:37.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:09:37.988 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 -- 172.21.15.59:0/1278606412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc1079c0 msgr2=0x7ff9fc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/1278606412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc107da0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009930 tx=0x7ff9ec02a440 comp rx=0 tx=0).stop 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 -- 172.21.15.59:0/1278606412 shutdown_connections 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/1278606412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc102360 0x7ff9fc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/1278606412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/1278606412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc107da0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 -- 172.21.15.59:0/1278606412 >> 172.21.15.59:0/1278606412 conn(0x7ff9fc0fd740 msgr2=0x7ff9fc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 -- 172.21.15.59:0/1278606412 shutdown_connections 2024-09-05T12:09:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.516+0000 7ff9fbe00640 1 -- 172.21.15.59:0/1278606412 wait complete. 2024-09-05T12:09:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.517+0000 7ff9fbe00640 1 Processor -- start 2024-09-05T12:09:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.517+0000 7ff9fbe00640 1 -- start start 2024-09-05T12:09:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 0x7ff9fc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc111dc0 con 0x7ff9fc1079c0 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc111c40 con 0x7ff9fc1019c0 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc111f40 con 0x7ff9fc102360 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9faa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 0x7ff9fc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9faa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45508/0 (socket says 172.21.15.59:45508) 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.518+0000 7ff9faa00640 1 -- 172.21.15.59:0/786544818 learned_addr learned my addr 172.21.15.59:0/786544818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 -- 172.21.15.59:0/786544818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 msgr2=0x7ff9fc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 0x7ff9fc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 -- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 msgr2=0x7ff9fc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 -- 172.21.15.59:0/786544818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc19f410 con 0x7ff9fc1019c0 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 0x7ff9fc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9fb400640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.519+0000 7ff9faa00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009230 tx=0x7ff9ec004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.520+0000 7ff9f3400640 1 -- 172.21.15.59:0/786544818 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec03f030 con 0x7ff9fc1019c0 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.520+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc19f6a0 con 0x7ff9fc1019c0 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.520+0000 7ff9f3400640 1 -- 172.21.15.59:0/786544818 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec004440 con 0x7ff9fc1019c0 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.520+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1a6fe0 con 0x7ff9fc1019c0 2024-09-05T12:09:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.520+0000 7ff9f3400640 1 -- 172.21.15.59:0/786544818 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec049060 con 0x7ff9fc1019c0 2024-09-05T12:09:38.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 Processor -- start 2024-09-05T12:09:38.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 -- start start 2024-09-05T12:09:38.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ff9f3400640 1 -- 172.21.15.59:0/786544818 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9ec042030 con 0x7ff9fc1019c0 2024-09-05T12:09:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638c5430290 con 0x5638c54c9cb0 2024-09-05T12:09:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5638c53d7b40 con 0x5638c54d4e90 2024-09-05T12:09:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.523+0000 7ffa047dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638c542c250 con 0x5638c54ca5d0 2024-09-05T12:09:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46970/0 (socket says 172.21.15.59:46970) 2024-09-05T12:09:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 -- 172.21.15.59:0/3162197880 learned_addr learned my addr 172.21.15.59:0/3162197880 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9faa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 -- 172.21.15.59:0/3162197880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 msgr2=0x5638c54d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 -- 172.21.15.59:0/3162197880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 msgr2=0x5638c54d7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.524+0000 7ff9fb400640 1 -- 172.21.15.59:0/3162197880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638c51a16a0 con 0x5638c54c9cb0 2024-09-05T12:09:38.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9fb400640 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4011790 tx=0x7ff9f4011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227941688386085e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9f3400640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 0x7ff9e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9f3400640 1 -- 172.21.15.59:0/786544818 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff9ec0ef350 con 0x7ff9fc1019c0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9ebe00640 1 -- 172.21.15.59:0/3162197880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4003c30 con 0x5638c54c9cb0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9ebe00640 1 -- 172.21.15.59:0/3162197880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f400fe80 con 0x5638c54c9cb0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9ebe00640 1 -- 172.21.15.59:0/3162197880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40193e0 con 0x5638c54c9cb0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.525+0000 7ff9faa00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 0x7ff9e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 -- 172.21.15.59:0/3162197880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 msgr2=0x5638c54ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4011790 tx=0x7ff9f4011c50 comp rx=0 tx=0).stop 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ff9faa00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 0x7ff9e00a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0bb550 tx=0x7ff9ec0f2e30 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 -- 172.21.15.59:0/3162197880 shutdown_connections 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3162197880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54ca090 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 -- 172.21.15.59:0/3162197880 >> 172.21.15.59:0/3162197880 conn(0x5638c54c6240 msgr2=0x5638c54c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 -- 172.21.15.59:0/3162197880 shutdown_connections 2024-09-05T12:09:38.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.526+0000 7ffa047dd740 1 -- 172.21.15.59:0/3162197880 wait complete. 2024-09-05T12:09:38.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.527+0000 7ffa047dd740 1 -- start start 2024-09-05T12:09:38.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.527+0000 7ffa047dd740 1 Processor -- start 2024-09-05T12:09:38.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638c54d7b00 con 0x5638c54c9cb0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5638c53eafa0 con 0x5638c54d4e90 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638c54d7c80 con 0x5638c54ca5d0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ffa047dd740 10 client.0 _ll_register_callbacks cb 0x5638c513eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46974/0 (socket says 172.21.15.59:46974) 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ff9fb400640 1 -- 172.21.15.59:0/1872961941 learned_addr learned my addr 172.21.15.59:0/1872961941 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.528+0000 7ff9fb400640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 -- 172.21.15.59:0/1872961941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 msgr2=0x5638c54d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 -- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 msgr2=0x5638c54d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 -- 172.21.15.59:0/1872961941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638c54d2770 con 0x5638c54c9cb0 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9fb400640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4005c10 tx=0x7ff9f40045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549efb9bd0b6ec7e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401f040 con 0x5638c54c9cb0 2024-09-05T12:09:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638c543d800 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.529+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff9f4004a40 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.530+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4004c40 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.530+0000 7ffa047dd740 10 client.17280 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.530+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638c54e39d0 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.530+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5638c543d800 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.530+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff9f4018070 con 0x5638c54c9cb0 2024-09-05T12:09:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 10 client.17280 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 20 client.17280 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c54e4de0 0x5638c54e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 4 client.17280 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638c5437d10 con 0x5638c54e4de0 2024-09-05T12:09:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.531+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c54e4de0 0x5638c54e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.532+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c54e4de0 0x5638c54e71d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4faf1819a22d0f2 server_cookie=c42fc846a94f519b in_seq=0 out_seq=0 2024-09-05T12:09:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.532+0000 7ff9e8c00640 10 client.17280 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.532+0000 7ff9e8c00640 1 -- 172.21.15.59:0/1872961941 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638c5437d10 con 0x5638c54e4de0 2024-09-05T12:09:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.532+0000 7ff9e8c00640 10 client.17280 handle_command_reply: tid=0 2024-09-05T12:09:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.532+0000 7ffa047dd740 1 client.17280 shutdown 2024-09-05T12:09:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 msgr2=0x5638c54d4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4005c10 tx=0x7ff9f40045c0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 shutdown_connections 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54d4e90 0x5638c54d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c54e4de0 0x5638c54e71d0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 --2- 172.21.15.59:0/1872961941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54c9cb0 0x5638c54d4750 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ff9e8c00640 0 client.17280 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 >> 172.21.15.59:0/1872961941 conn(0x5638c54c6240 msgr2=0x5638c54cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 shutdown_connections 2024-09-05T12:09:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 1 -- 172.21.15.59:0/1872961941 wait complete. 2024-09-05T12:09:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.533+0000 7ffa047dd740 20 client.17280 trim_cache size 0 max 16384 2024-09-05T12:09:38.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 Processor -- start 2024-09-05T12:09:38.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 -- start start 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54eced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1aee90 con 0x5638c54d4e90 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b1710 con 0x5638c54c9cb0 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.552+0000 7ffa047dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1af500 con 0x5638c54ca5d0 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45548/0 (socket says 172.21.15.59:45548) 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9fb400640 1 -- 172.21.15.59:0/3491728870 learned_addr learned my addr 172.21.15.59:0/3491728870 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9fb400640 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 -- 172.21.15.59:0/3491728870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 msgr2=0x5638c54c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 -- 172.21.15.59:0/3491728870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 msgr2=0x5638c54eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 -- 172.21.15.59:0/3491728870 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638c54c6d80 con 0x5638c54ca5d0 2024-09-05T12:09:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9fb400640 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.553+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54ec990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5638c54d2fb0 tx=0x7ff9e4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46aeb80bd6b064ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.554+0000 7ff9eb400640 1 -- 172.21.15.59:0/3491728870 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4005220 con 0x5638c54ca5d0 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.554+0000 7ff9eb400640 1 -- 172.21.15.59:0/3491728870 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e4004c80 con 0x5638c54ca5d0 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.554+0000 7ff9eb400640 1 -- 172.21.15.59:0/3491728870 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4011a80 con 0x5638c54ca5d0 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.554+0000 7ffa047dd740 1 -- 172.21.15.59:0/3491728870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 msgr2=0x5638c54ec990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.554+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54ec990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5638c54d2fb0 tx=0x7ff9e4005f90 comp rx=0 tx=0).stop 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 -- 172.21.15.59:0/3491728870 shutdown_connections 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54ec990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 --2- 172.21.15.59:0/3491728870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 -- 172.21.15.59:0/3491728870 >> 172.21.15.59:0/3491728870 conn(0x5638c54c6240 msgr2=0x5638c54c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 -- 172.21.15.59:0/3491728870 shutdown_connections 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 -- 172.21.15.59:0/3491728870 wait complete. 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 -- start start 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.555+0000 7ffa047dd740 1 Processor -- start 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1afbd0 con 0x5638c54d4e90 2024-09-05T12:09:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b1040 con 0x5638c54c9cb0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b02a0 con 0x5638c54ca5d0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ffa047dd740 10 client.0 _ll_register_callbacks cb 0x7ff9e4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46990/0 (socket says 172.21.15.59:46990) 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 -- 172.21.15.59:0/596992011 learned_addr learned my addr 172.21.15.59:0/596992011 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 -- 172.21.15.59:0/596992011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 msgr2=0x5638c54db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.556+0000 7ff9fb400640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9fb400640 1 -- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 msgr2=0x5638c54dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9fb400640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9fb400640 1 -- 172.21.15.59:0/596992011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638c554eb70 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9fb400640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40044c0 tx=0x7ff9f40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0eaead1fb8fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400fcb0 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e4004c80 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff9f4011310 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4039030 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ffa047dd740 10 client.17286 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.557+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638c554fe50 con 0x5638c54d4e90 2024-09-05T12:09:38.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e4004c80 con 0x5638c54d4e90 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff9f4018070 con 0x5638c54d4e90 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 10 client.17286 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 20 client.17286 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c5550430 0x5638c5552850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 4 client.17286 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.558+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638c53b3fb0 con 0x5638c5550430 2024-09-05T12:09:38.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.559+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c5550430 0x5638c5552850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.559+0000 7ff9f3e00640 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c5550430 0x5638c5552850 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b064acf0819ea35 server_cookie=39495d751a08a0e1 in_seq=0 out_seq=0 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.559+0000 7ff9e8c00640 10 client.17286 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.560+0000 7ff9e8c00640 1 -- 172.21.15.59:0/596992011 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638c53b3fb0 con 0x5638c5550430 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.560+0000 7ff9e8c00640 10 client.17286 handle_command_reply: tid=0 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.560+0000 7ffa047dd740 1 client.17286 shutdown 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.560+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 msgr2=0x5638c54db8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.560+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40044c0 tx=0x7ff9f40047e0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 shutdown_connections 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638c5550430 0x5638c5552850 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638c54d4e90 0x5638c54db8d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638c54ca5d0 0x5638c54db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 --2- 172.21.15.59:0/596992011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638c54c9cb0 0x5638c54dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ff9e8c00640 0 client.17286 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:38.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 >> 172.21.15.59:0/596992011 conn(0x5638c54c6240 msgr2=0x5638c54c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 shutdown_connections 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 1 -- 172.21.15.59:0/596992011 wait complete. 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.561+0000 7ffa047dd740 20 client.17286 trim_cache size 0 max 16384 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "239d5e57-b1e3-455e-83aa-f1547e6fab64", 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:09:38.566 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 msgr2=0x7ff9e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 0x7ff9e00a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0bb550 tx=0x7ff9ec0f2e30 comp rx=0 tx=0).stop 2024-09-05T12:09:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 msgr2=0x7ff9fc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009230 tx=0x7ff9ec004040 comp rx=0 tx=0).stop 2024-09-05T12:09:38.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 shutdown_connections 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff9e00a2c00 0x7ff9e00a50c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9fc1079c0 0x7ff9fc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc102360 0x7ff9fc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 --2- 172.21.15.59:0/786544818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9fc1019c0 0x7ff9fc19a460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 >> 172.21.15.59:0/786544818 conn(0x7ff9fc0fd740 msgr2=0x7ff9fc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 shutdown_connections 2024-09-05T12:09:38.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:38.562+0000 7ff9fbe00640 1 -- 172.21.15.59:0/786544818 wait complete. 2024-09-05T12:09:39.150 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:38 smithi059 ceph-mon[30025]: pgmap v753: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T12:09:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:38 smithi089 ceph-mon[34499]: pgmap v753: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T12:09:39.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:09:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:38 smithi189 ceph-mon[34772]: pgmap v753: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-05T12:09:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:40.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:39 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:09:40.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:39 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:39 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:09:40.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:39 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:39 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:09:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:39 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:39 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:09:40.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:39 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 -- 172.21.15.59:0/2837288010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 msgr2=0x7fea54110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 --2- 172.21.15.59:0/2837288010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea54110810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009ca0 tx=0x7fea4c02a950 comp rx=0 tx=0).stop 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 -- 172.21.15.59:0/2837288010 shutdown_connections 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 --2- 172.21.15.59:0/2837288010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea54110810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 --2- 172.21.15.59:0/2837288010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074f40 0x7fea5410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 --2- 172.21.15.59:0/2837288010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074620 0x7fea54074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 -- 172.21.15.59:0/2837288010 >> 172.21.15.59:0/2837288010 conn(0x7fea540fd8c0 msgr2=0x7fea540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 -- 172.21.15.59:0/2837288010 shutdown_connections 2024-09-05T12:09:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.254+0000 7fea5a600640 1 -- 172.21.15.59:0/2837288010 wait complete. 2024-09-05T12:09:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.255+0000 7fea5a600640 1 Processor -- start 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.255+0000 7fea5a600640 1 -- start start 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 0x7fea5419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 0x7fea5419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea5419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea54114150 con 0x7fea54074620 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea54113fd0 con 0x7fea54074f40 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea5a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea541142d0 con 0x7fea5410e3c0 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea53400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 0x7fea5419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea53400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 0x7fea5419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47004/0 (socket says 172.21.15.59:47004) 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 0x7fea5419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea53400640 1 -- 172.21.15.59:0/2958545591 learned_addr learned my addr 172.21.15.59:0/2958545591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea53e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea5419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 -- 172.21.15.59:0/2958545591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 msgr2=0x7fea5419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea5419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 -- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 msgr2=0x7fea5419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 0x7fea5419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.256+0000 7fea52a00640 1 -- 172.21.15.59:0/2958545591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5419f5e0 con 0x7fea54074f40 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea52a00640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 0x7fea5419ab70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea4be00640 1 -- 172.21.15.59:0/2958545591 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000ccc0 con 0x7fea54074f40 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5419f8d0 con 0x7fea54074f40 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea4be00640 1 -- 172.21.15.59:0/2958545591 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40010910 con 0x7fea54074f40 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea4be00640 1 -- 172.21.15.59:0/2958545591 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40018960 con 0x7fea54074f40 2024-09-05T12:09:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.257+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a71b0 con 0x7fea54074f40 2024-09-05T12:09:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.260+0000 7fea5d081740 1 Processor -- start 2024-09-05T12:09:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.260+0000 7fea5d081740 1 -- start start 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585056f93e0 con 0x55850579dd70 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5585056a0d80 con 0x558505792b90 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea5d081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585056f53a0 con 0x5585057934b0 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea4be00640 1 -- 172.21.15.59:0/2958545591 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea40010ac0 con 0x7fea54074f40 2024-09-05T12:09:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45580/0 (socket says 172.21.15.59:45580) 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 -- 172.21.15.59:0/173184896 learned_addr learned my addr 172.21.15.59:0/173184896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53400640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea52a00640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 -- 172.21.15.59:0/173184896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 msgr2=0x55850579d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 -- 172.21.15.59:0/173184896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 msgr2=0x5585057a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53e00640 1 -- 172.21.15.59:0/173184896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585056bfae0 con 0x558505792b90 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.261+0000 7fea53400640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.262+0000 7fea53e00640 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea4c035000 tx=0x7fea4c0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfcfa5abf3d6021 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.262+0000 7fea48c00640 1 -- 172.21.15.59:0/173184896 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c034040 con 0x558505792b90 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.262+0000 7fea48c00640 1 -- 172.21.15.59:0/173184896 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c02f940 con 0x558505792b90 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.262+0000 7fea48c00640 1 -- 172.21.15.59:0/173184896 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0042a0 con 0x558505792b90 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea4be00640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 0x7fea380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 -- 172.21.15.59:0/173184896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 msgr2=0x558505792f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea4c035000 tx=0x7fea4c0093e0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea4be00640 1 -- 172.21.15.59:0/2958545591 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea400cf7d0 con 0x7fea54074f40 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 -- 172.21.15.59:0/173184896 shutdown_connections 2024-09-05T12:09:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 --2- 172.21.15.59:0/173184896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x558505792f70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea5d081740 1 -- 172.21.15.59:0/173184896 >> 172.21.15.59:0/173184896 conn(0x55850578f120 msgr2=0x55850578f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.263+0000 7fea53400640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 0x7fea380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.264+0000 7fea5d081740 1 -- 172.21.15.59:0/173184896 shutdown_connections 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.264+0000 7fea5d081740 1 -- 172.21.15.59:0/173184896 wait complete. 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.264+0000 7fea5d081740 1 -- start start 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.264+0000 7fea53400640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 0x7fea380a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fea44009620 tx=0x7fea44005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.264+0000 7fea5d081740 1 Processor -- start 2024-09-05T12:09:40.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x55850579b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585057a0980 con 0x55850579dd70 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5585056b40f0 con 0x558505792b90 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585057a0b00 con 0x5585057934b0 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea5d081740 10 client.0 _ll_register_callbacks cb 0x5585056a59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x55850579b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea53400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea53400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45590/0 (socket says 172.21.15.59:45590) 2024-09-05T12:09:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea53400640 1 -- 172.21.15.59:0/1353306999 learned_addr learned my addr 172.21.15.59:0/1353306999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x55850579b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47010/0 (socket says 172.21.15.59:47010) 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.265+0000 7fea52a00640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 -- 172.21.15.59:0/1353306999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 msgr2=0x55850579b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 -- 172.21.15.59:0/1353306999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 msgr2=0x55850579b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x55850579b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 -- 172.21.15.59:0/1353306999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850546a3c0 con 0x558505792b90 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea53400640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea440035f0 tx=0x7fea44004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b052cf575830a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea3d600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44020790 con 0x558505792b90 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c02f940 con 0x558505792b90 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.266+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea44020930 con 0x558505792b90 2024-09-05T12:09:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fea44020b70 con 0x558505792b90 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 10 client.27299 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5585057ac740 con 0x558505792b90 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c02f940 con 0x558505792b90 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fea44027070 con 0x558505792b90 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 10 client.27299 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 20 client.27299 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5585057adb50 0x5585057aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 4 client.27299 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.267+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558505706950 con 0x5585057adb50 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.268+0000 7fea52a00640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5585057adb50 0x5585057aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.268+0000 7fea52a00640 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5585057adb50 0x5585057aff60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebf3a9c7e46d0fe server_cookie=549ebb5b2fc35859 in_seq=0 out_seq=0 2024-09-05T12:09:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.268+0000 7fea3d600640 10 client.27299 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea3d600640 1 -- 172.21.15.59:0/1353306999 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558505706950 con 0x5585057adb50 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea3d600640 10 client.27299 handle_command_reply: tid=0 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 client.27299 shutdown 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 msgr2=0x55850579ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea440035f0 tx=0x7fea44004730 comp rx=0 tx=0).stop 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 shutdown_connections 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5585057adb50 0x5585057aff60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x55850579b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x55850579b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 --2- 172.21.15.59:0/1353306999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x55850579ac80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea3d600640 0 client.27299 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 >> 172.21.15.59:0/1353306999 conn(0x55850578f120 msgr2=0x55850578d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.269+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 shutdown_connections 2024-09-05T12:09:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.270+0000 7fea5d081740 1 -- 172.21.15.59:0/1353306999 wait complete. 2024-09-05T12:09:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.270+0000 7fea5d081740 20 client.27299 trim_cache size 0 max 16384 2024-09-05T12:09:40.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 Processor -- start 2024-09-05T12:09:40.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 -- start start 2024-09-05T12:09:40.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 0x5585057b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea541af130 con 0x558505792b90 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b19b0 con 0x55850579dd70 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.288+0000 7fea5d081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea541af7a0 con 0x5585057934b0 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47012/0 (socket says 172.21.15.59:47012) 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 -- 172.21.15.59:0/1019102493 learned_addr learned my addr 172.21.15.59:0/1019102493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53e00640 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 0x5585057b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 -- 172.21.15.59:0/1019102493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 msgr2=0x5585057b5800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057b5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 -- 172.21.15.59:0/1019102493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 msgr2=0x5585057b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 0x5585057b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 -- 172.21.15.59:0/1019102493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585057b05b0 con 0x558505792b90 2024-09-05T12:09:40.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53e00640 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 0x5585057b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea53400640 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea44019040 tx=0x7fea44012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b13d9bebc6006b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea3fe00640 1 -- 172.21.15.59:0/1019102493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44009c20 con 0x558505792b90 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.289+0000 7fea3fe00640 1 -- 172.21.15.59:0/1019102493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44020990 con 0x558505792b90 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea3fe00640 1 -- 172.21.15.59:0/1019102493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44029870 con 0x558505792b90 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 -- 172.21.15.59:0/1019102493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 msgr2=0x5585057955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea44019040 tx=0x7fea44012d90 comp rx=0 tx=0).stop 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 -- 172.21.15.59:0/1019102493 shutdown_connections 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55850579dd70 0x5585057b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 --2- 172.21.15.59:0/1019102493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558505792b90 0x5585057955f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 -- 172.21.15.59:0/1019102493 >> 172.21.15.59:0/1019102493 conn(0x55850578f120 msgr2=0x55850578e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 -- 172.21.15.59:0/1019102493 shutdown_connections 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.290+0000 7fea5d081740 1 -- 172.21.15.59:0/1019102493 wait complete. 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 -- start start 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 Processor -- start 2024-09-05T12:09:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x5585057a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea541afe70 con 0x55850579dd70 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b12e0 con 0x558505792b90 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b0540 con 0x5585057934b0 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.291+0000 7fea5d081740 10 client.0 _ll_register_callbacks cb 0x7fea44004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47020/0 (socket says 172.21.15.59:47020) 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53e00640 1 -- 172.21.15.59:0/2232083039 learned_addr learned my addr 172.21.15.59:0/2232083039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x5585057a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea52a00640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 -- 172.21.15.59:0/2232083039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 msgr2=0x5585057a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 -- 172.21.15.59:0/2232083039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 msgr2=0x5585057a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 -- 172.21.15.59:0/2232083039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585057a2ab0 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea53400640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x5585057a5e40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fea44009700 tx=0x7fea44003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a72b6e0f8bfb579 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.292+0000 7fea3d600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44005250 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea44020990 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea440053f0 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fea44004710 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea5d081740 10 client.27305 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558505819250 con 0x558505792b90 2024-09-05T12:09:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.293+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44020990 con 0x558505792b90 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fea44027070 con 0x558505792b90 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 10 client.27305 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 20 client.27305 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558505819860 0x55850581bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 4 client.27305 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558505700e60 con 0x558505819860 2024-09-05T12:09:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.294+0000 7fea52a00640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558505819860 0x55850581bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea52a00640 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558505819860 0x55850581bc50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ccbd0a3deaefd11 server_cookie=a0369df7c2a20192 in_seq=0 out_seq=0 2024-09-05T12:09:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea3d600640 10 client.27305 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea3d600640 1 -- 172.21.15.59:0/2232083039 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558505700e60 con 0x558505819860 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea3d600640 10 client.27305 handle_command_reply: tid=0 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 client.27305 shutdown 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 msgr2=0x5585057a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x5585057a5e40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fea44009700 tx=0x7fea44003e80 comp rx=0 tx=0).stop 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 shutdown_connections 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558505819860 0x55850581bc50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55850579dd70 0x5585057a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585057934b0 0x5585057a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea3d600640 0 client.27305 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 --2- 172.21.15.59:0/2232083039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558505792b90 0x5585057a5e40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 >> 172.21.15.59:0/2232083039 conn(0x55850578f120 msgr2=0x55850579c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 shutdown_connections 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 1 -- 172.21.15.59:0/2232083039 wait complete. 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.295+0000 7fea5d081740 20 client.27305 trim_cache size 0 max 16384 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:09:40.302 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 msgr2=0x7fea380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 0x7fea380a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fea44009620 tx=0x7fea44005f90 comp rx=0 tx=0).stop 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 msgr2=0x7fea5419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 0x7fea5419ab70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 shutdown_connections 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fea380a2b80 0x7fea380a5040 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5410e3c0 0x7fea5419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea54074f40 0x7fea5419ab70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 --2- 172.21.15.59:0/2958545591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea54074620 0x7fea5419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 >> 172.21.15.59:0/2958545591 conn(0x7fea540fd8c0 msgr2=0x7fea540fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.296+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 shutdown_connections 2024-09-05T12:09:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:40.297+0000 7fea5a600640 1 -- 172.21.15.59:0/2958545591 wait complete. 2024-09-05T12:09:40.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:39 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:09:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:39 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:39 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:09:40.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:39 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:40.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:40 smithi059 ceph-mon[30025]: pgmap v754: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-05T12:09:40.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:40 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:09:41.048 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:09:41.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:09:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:40 smithi089 ceph-mon[34499]: pgmap v754: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-05T12:09:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:40 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:09:41.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:40 smithi189 ceph-mon[34772]: pgmap v754: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-05T12:09:41.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:40 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:09:41.426 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 -- 172.21.15.59:0/2370140641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40103ff0 msgr2=0x7f6d401043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 --2- 172.21.15.59:0/2370140641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40103ff0 0x7f6d401043d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009900 tx=0x7f6d3002a410 comp rx=0 tx=0).stop 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 -- 172.21.15.59:0/2370140641 shutdown_connections 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 --2- 172.21.15.59:0/2370140641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 0x7f6d40110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 --2- 172.21.15.59:0/2370140641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d401049a0 0x7f6d401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 --2- 172.21.15.59:0/2370140641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d40103ff0 0x7f6d401043d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 -- 172.21.15.59:0/2370140641 >> 172.21.15.59:0/2370140641 conn(0x7f6d400ffa90 msgr2=0x7f6d40101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 -- 172.21.15.59:0/2370140641 shutdown_connections 2024-09-05T12:09:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.974+0000 7f6d45e00640 1 -- 172.21.15.59:0/2370140641 wait complete. 2024-09-05T12:09:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.975+0000 7f6d45e00640 1 Processor -- start 2024-09-05T12:09:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.975+0000 7f6d45e00640 1 -- start start 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 0x7f6d4019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 0x7f6d4019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114190 con 0x7f6d401049a0 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114010 con 0x7f6d40109120 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d45e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114310 con 0x7f6d40103ff0 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60166/0 (socket says 172.21.15.59:60166) 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 -- 172.21.15.59:0/1338062100 learned_addr learned my addr 172.21.15.59:0/1338062100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d3ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 0x7f6d4019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d3f400640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 0x7f6d4019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 -- 172.21.15.59:0/1338062100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 msgr2=0x7f6d4019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 0x7f6d4019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 -- 172.21.15.59:0/1338062100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 msgr2=0x7f6d4019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 0x7f6d4019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d36000640 1 -- 172.21.15.59:0/1338062100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4019f5c0 con 0x7f6d401049a0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.976+0000 7f6d3ea00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 0x7f6d4019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d36000640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9f0 tx=0x7f6d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800cd80 con 0x7f6d401049a0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d28005500 con 0x7f6d401049a0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28010690 con 0x7f6d401049a0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4019f8b0 con 0x7f6d401049a0 2024-09-05T12:09:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.977+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a7190 con 0x7f6d401049a0 2024-09-05T12:09:41.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.978+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d401043d0 con 0x7f6d401049a0 2024-09-05T12:09:41.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.980+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d280046c0 con 0x7f6d401049a0 2024-09-05T12:09:41.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.982+0000 7f6d44a00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 0x7f6d240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:41.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.982+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6d280cf830 con 0x7f6d401049a0 2024-09-05T12:09:41.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.982+0000 7f6d3ea00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 0x7f6d240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:41.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.983+0000 7f6d3ea00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 0x7f6d240a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:41.987+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d2809c320 con 0x7f6d401049a0 2024-09-05T12:09:42.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.209+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4002d7c0 con 0x7f6d401049a0 2024-09-05T12:09:42.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.209+0000 7f6d44a00640 1 -- 172.21.15.59:0/1338062100 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f6d28086020 con 0x7f6d401049a0 2024-09-05T12:09:42.211 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:09:42.211 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:09:42.212 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:09:42.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 msgr2=0x7f6d240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 0x7f6d240a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30031510 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 msgr2=0x7f6d4019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9f0 tx=0x7f6d2800edf0 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 shutdown_connections 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6d240a2c00 0x7f6d240a50c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d40109120 0x7f6d4019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d401049a0 0x7f6d4019ab50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 --2- 172.21.15.59:0/1338062100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d40103ff0 0x7f6d4019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 >> 172.21.15.59:0/1338062100 conn(0x7f6d400ffa90 msgr2=0x7f6d40100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 shutdown_connections 2024-09-05T12:09:42.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:42.212+0000 7f6d45e00640 1 -- 172.21.15.59:0/1338062100 wait complete. 2024-09-05T12:09:42.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:09:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:42 smithi089 ceph-mon[34499]: pgmap v755: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:42 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1338062100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:43.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:42 smithi059 ceph-mon[30025]: pgmap v755: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:43.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1338062100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:43.310 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:42 smithi189 ceph-mon[34772]: pgmap v755: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:09:43.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:42 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1338062100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 -- 172.21.15.59:0/1991353528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c108f60 msgr2=0x7fe27c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 --2- 172.21.15.59:0/1991353528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c108f60 0x7fe27c110ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe274009be0 tx=0x7fe27402a6c0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 -- 172.21.15.59:0/1991353528 shutdown_connections 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 --2- 172.21.15.59:0/1991353528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c108f60 0x7fe27c110ae0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 --2- 172.21.15.59:0/1991353528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c1047e0 0x7fe27c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 --2- 172.21.15.59:0/1991353528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c103e30 0x7fe27c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 -- 172.21.15.59:0/1991353528 >> 172.21.15.59:0/1991353528 conn(0x7fe27c0ff8d0 msgr2=0x7fe27c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 -- 172.21.15.59:0/1991353528 shutdown_connections 2024-09-05T12:09:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.900+0000 7fe282e00640 1 -- 172.21.15.59:0/1991353528 wait complete. 2024-09-05T12:09:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.901+0000 7fe282e00640 1 Processor -- start 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.901+0000 7fe282e00640 1 -- start start 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c103e30 0x7fe27c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c108f60 0x7fe27c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c113fc0 con 0x7fe27c103e30 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c113e40 con 0x7fe27c1047e0 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c114140 con 0x7fe27c108f60 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c103e30 0x7fe27c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36888/0 (socket says 172.21.15.59:36888) 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 -- 172.21.15.59:0/3212344339 learned_addr learned my addr 172.21.15.59:0/3212344339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe282400640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c108f60 0x7fe27c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 -- 172.21.15.59:0/3212344339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c108f60 msgr2=0x7fe27c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c108f60 0x7fe27c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 -- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c103e30 msgr2=0x7fe27c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c103e30 0x7fe27c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.902+0000 7fe281000640 1 -- 172.21.15.59:0/3212344339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c19f3d0 con 0x7fe27c1047e0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe281000640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe26800e9f0 tx=0x7fe26800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe27a000640 1 -- 172.21.15.59:0/3212344339 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26800ccc0 con 0x7fe27c1047e0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c19f6c0 con 0x7fe27c1047e0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe27a000640 1 -- 172.21.15.59:0/3212344339 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe268005500 con 0x7fe27c1047e0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe27a000640 1 -- 172.21.15.59:0/3212344339 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268010690 con 0x7fe27c1047e0 2024-09-05T12:09:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.903+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1a6fa0 con 0x7fe27c1047e0 2024-09-05T12:09:43.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.906+0000 7fe27a000640 1 -- 172.21.15.59:0/3212344339 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe2680046c0 con 0x7fe27c1047e0 2024-09-05T12:09:43.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.906+0000 7fe287063740 1 Processor -- start 2024-09-05T12:09:43.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.906+0000 7fe287063740 1 -- start start 2024-09-05T12:09:43.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c83151b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c831526020 0x55c831528410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c8314813e0 con 0x55c831526020 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c831428d80 con 0x55c83151b760 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe287063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c83147d3a0 con 0x55c83151ae40 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe282400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c83151b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36904/0 (socket says 172.21.15.59:36904) 2024-09-05T12:09:43.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 -- 172.21.15.59:0/746202467 learned_addr learned my addr 172.21.15.59:0/746202467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:43.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281000640 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c831526020 0x55c831528410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 -- 172.21.15.59:0/746202467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 msgr2=0x55c83151b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c83151b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 -- 172.21.15.59:0/746202467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c831526020 msgr2=0x55c831528410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c831526020 0x55c831528410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.907+0000 7fe281a00640 1 -- 172.21.15.59:0/746202467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c831447ae0 con 0x55c83151b760 2024-09-05T12:09:43.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe282400640 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c83151b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe281a00640 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe270009cf0 tx=0x7fe270011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3fc44e69e3ae5e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe27a000640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 0x7fe2640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe27a000640 1 -- 172.21.15.59:0/3212344339 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe2680cf5c0 con 0x7fe27c1047e0 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe26ea00640 1 -- 172.21.15.59:0/746202467 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270004230 con 0x55c83151b760 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe26ea00640 1 -- 172.21.15.59:0/746202467 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2700043d0 con 0x55c83151b760 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe282400640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 0x7fe2640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.908+0000 7fe26ea00640 1 -- 172.21.15.59:0/746202467 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270019960 con 0x55c83151b760 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 -- 172.21.15.59:0/746202467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 msgr2=0x55c831525ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe270009cf0 tx=0x7fe270011810 comp rx=0 tx=0).stop 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 -- 172.21.15.59:0/746202467 shutdown_connections 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c831526020 0x55c831528410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831525ae0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 --2- 172.21.15.59:0/746202467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c83151b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 -- 172.21.15.59:0/746202467 >> 172.21.15.59:0/746202467 conn(0x55c8315173d0 msgr2=0x55c8315177b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe282400640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 0x7fe2640a50c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe274004af0 tx=0x7fe274005fd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 -- 172.21.15.59:0/746202467 shutdown_connections 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.909+0000 7fe287063740 1 -- 172.21.15.59:0/746202467 wait complete. 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 -- start start 2024-09-05T12:09:43.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 Processor -- start 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c831522f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831523a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c831528bf0 con 0x55c83151b760 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c83143c0f0 con 0x55c831526020 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.910+0000 7fe287063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c831528d70 con 0x55c83151ae40 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe287063740 10 client.0 _ll_register_callbacks cb 0x55c831423bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c831522f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60198/0 (socket says 172.21.15.59:60198) 2024-09-05T12:09:43.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 -- 172.21.15.59:0/4223309461 learned_addr learned my addr 172.21.15.59:0/4223309461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe282400640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831523a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 -- 172.21.15.59:0/4223309461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 msgr2=0x55c831522f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c831522f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 -- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 msgr2=0x55c831523a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831523a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe281000640 1 -- 172.21.15.59:0/4223309461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8311f23c0 con 0x55c83151b760 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.911+0000 7fe282400640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831523a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe281000640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe2680983e0 tx=0x7fe268097d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1e62df39636d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ce070 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe2680a41f0 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2700043d0 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2680056b0 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe287063740 10 client.17322 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c831534d30 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2700043d0 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.912+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe2680ba2a0 con 0x55c83151b760 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 10 client.17322 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 20 client.17322 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c831536140 0x55c831538530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 4 client.17322 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:43.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83148e950 con 0x55c831536140 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.913+0000 7fe281a00640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c831536140 0x55c831538530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.914+0000 7fe281a00640 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c831536140 0x55c831538530 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b36725a1a4541f server_cookie=784841183e341e35 in_seq=0 out_seq=0 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.914+0000 7fe263400640 10 client.17322 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe263400640 1 -- 172.21.15.59:0/4223309461 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c83148e950 con 0x55c831536140 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe263400640 10 client.17322 handle_command_reply: tid=0 2024-09-05T12:09:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 client.17322 shutdown 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 msgr2=0x55c8315234d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe2680983e0 tx=0x7fe268097d70 comp rx=0 tx=0).stop 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 shutdown_connections 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831523a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c831536140 0x55c831538530 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151b760 0x55c8315234d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 --2- 172.21.15.59:0/4223309461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151ae40 0x55c831522f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 >> 172.21.15.59:0/4223309461 conn(0x55c8315173d0 msgr2=0x55c83151d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe263400640 0 client.17322 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 shutdown_connections 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 1 -- 172.21.15.59:0/4223309461 wait complete. 2024-09-05T12:09:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.915+0000 7fe287063740 20 client.17322 trim_cache size 0 max 16384 2024-09-05T12:09:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 Processor -- start 2024-09-05T12:09:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 -- start start 2024-09-05T12:09:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831516380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c831526020 0x55c83153dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1aef60 con 0x55c83151ae40 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b17e0 con 0x55c83151b760 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.929+0000 7fe287063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1af5d0 con 0x55c831526020 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60212/0 (socket says 172.21.15.59:60212) 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831516380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281a00640 1 -- 172.21.15.59:0/1495891097 learned_addr learned my addr 172.21.15.59:0/1495891097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe282400640 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c831526020 0x55c83153dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 -- 172.21.15.59:0/1495891097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c831526020 msgr2=0x55c83153dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c831526020 0x55c83153dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 -- 172.21.15.59:0/1495891097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 msgr2=0x55c831522850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 -- 172.21.15.59:0/1495891097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83151ddf0 con 0x55c83151b760 2024-09-05T12:09:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281a00640 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.930+0000 7fe281000640 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831516380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c831524540 tx=0x7fe268097de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21d86d5f88c35ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe26e000640 1 -- 172.21.15.59:0/1495891097 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ce030 con 0x55c83151b760 2024-09-05T12:09:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe26e000640 1 -- 172.21.15.59:0/1495891097 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26809c110 con 0x55c83151b760 2024-09-05T12:09:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe26e000640 1 -- 172.21.15.59:0/1495891097 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ba2a0 con 0x55c83151b760 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 -- 172.21.15.59:0/1495891097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 msgr2=0x55c831516380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831516380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c831524540 tx=0x7fe268097de0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 -- 172.21.15.59:0/1495891097 shutdown_connections 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c831526020 0x55c83153dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c83151b760 0x55c831516380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 --2- 172.21.15.59:0/1495891097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c831522850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 -- 172.21.15.59:0/1495891097 >> 172.21.15.59:0/1495891097 conn(0x55c8315173d0 msgr2=0x55c831515c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 -- 172.21.15.59:0/1495891097 shutdown_connections 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.931+0000 7fe287063740 1 -- 172.21.15.59:0/1495891097 wait complete. 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 -- start start 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 Processor -- start 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c83152e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151b760 0x55c831529a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1afca0 con 0x55c83151ae40 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b1110 con 0x55c831526020 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.932+0000 7fe287063740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b0370 con 0x55c83151b760 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe287063740 10 client.0 _ll_register_callbacks cb 0x7fe268004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c83152e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe282400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe282400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36922/0 (socket says 172.21.15.59:36922) 2024-09-05T12:09:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe282400640 1 -- 172.21.15.59:0/3830541258 learned_addr learned my addr 172.21.15.59:0/3830541258 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281000640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151b760 0x55c831529a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 -- 172.21.15.59:0/3830541258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151b760 msgr2=0x55c831529a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151b760 0x55c831529a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 -- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 msgr2=0x55c831529fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 -- 172.21.15.59:0/3830541258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83152a790 con 0x55c83151ae40 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe282400640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe281a00640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c83152e740 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe270014040 tx=0x7fe2700194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820bcaf3a04595d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.933+0000 7fe263400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270004390 con 0x55c83151ae40 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe270005040 con 0x55c83151ae40 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26809c110 con 0x55c83151ae40 2024-09-05T12:09:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe27003c050 con 0x55c83151ae40 2024-09-05T12:09:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe287063740 10 client.17328 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c8315a0dc0 con 0x55c83151ae40 2024-09-05T12:09:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.934+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26809c110 con 0x55c83151ae40 2024-09-05T12:09:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe270003ae0 con 0x55c83151ae40 2024-09-05T12:09:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 10 client.17328 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 20 client.17328 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8315a12f0 0x55c8315a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 4 client.17328 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c831488e60 con 0x55c8315a12f0 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.935+0000 7fe281000640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8315a12f0 0x55c8315a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe281000640 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8315a12f0 0x55c8315a36e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288fd7d955091ca1 server_cookie=62d1e71df9f294b in_seq=0 out_seq=0 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe263400640 10 client.17328 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe263400640 1 -- 172.21.15.59:0/3830541258 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c831488e60 con 0x55c8315a12f0 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe263400640 10 client.17328 handle_command_reply: tid=0 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 client.17328 shutdown 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 msgr2=0x55c83152e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c83152e740 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe270014040 tx=0x7fe2700194c0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 shutdown_connections 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c831526020 0x55c831529fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c83151b760 0x55c831529a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c8315a12f0 0x55c8315a36e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 --2- 172.21.15.59:0/3830541258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c83151ae40 0x55c83152e740 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe263400640 0 client.17328 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 >> 172.21.15.59:0/3830541258 conn(0x55c8315173d0 msgr2=0x55c831516490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 shutdown_connections 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 1 -- 172.21.15.59:0/3830541258 wait complete. 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.936+0000 7fe287063740 20 client.17328 trim_cache size 0 max 16384 2024-09-05T12:09:43.942 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:09:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 msgr2=0x7fe2640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 0x7fe2640a50c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe274004af0 tx=0x7fe274005fd0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 msgr2=0x7fe27c19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe26800e9f0 tx=0x7fe26800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 shutdown_connections 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe2640a2c00 0x7fe2640a50c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe27c108f60 0x7fe27c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe27c1047e0 0x7fe27c19a960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 --2- 172.21.15.59:0/3212344339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe27c103e30 0x7fe27c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 >> 172.21.15.59:0/3212344339 conn(0x7fe27c0ff8d0 msgr2=0x7fe27c100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 shutdown_connections 2024-09-05T12:09:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:43.937+0000 7fe282e00640 1 -- 172.21.15.59:0/3212344339 wait complete. 2024-09-05T12:09:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:44 smithi089 ceph-mon[34499]: pgmap v756: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-05T12:09:45.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:44 smithi059 ceph-mon[30025]: pgmap v756: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-05T12:09:45.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:44 smithi189 ceph-mon[34772]: pgmap v756: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-05T12:09:45.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:09:45.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:09:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:46.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 -- 172.21.15.59:0/133714100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09981049a0 msgr2=0x7f09981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 --2- 172.21.15.59:0/133714100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09981049a0 0x7f09981089f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0980009900 tx=0x7f098002a410 comp rx=0 tx=0).stop 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 -- 172.21.15.59:0/133714100 shutdown_connections 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 --2- 172.21.15.59:0/133714100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0998109120 0x7f0998110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 --2- 172.21.15.59:0/133714100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09981049a0 0x7f09981089f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 --2- 172.21.15.59:0/133714100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998103ff0 0x7f09981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 -- 172.21.15.59:0/133714100 >> 172.21.15.59:0/133714100 conn(0x7f09980ffa90 msgr2=0x7f0998101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 -- 172.21.15.59:0/133714100 shutdown_connections 2024-09-05T12:09:46.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.641+0000 7f099e200640 1 -- 172.21.15.59:0/133714100 wait complete. 2024-09-05T12:09:46.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.642+0000 7f099e200640 1 Processor -- start 2024-09-05T12:09:46.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.642+0000 7f099e200640 1 -- start start 2024-09-05T12:09:46.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09981049a0 0x7f099819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998109120 0x7f099819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0998114140 con 0x7f0998103ff0 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0998113fc0 con 0x7f0998109120 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f099e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09981142c0 con 0x7f09981049a0 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f0997400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f0997400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60238/0 (socket says 172.21.15.59:60238) 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f0997400640 1 -- 172.21.15.59:0/2268742754 learned_addr learned my addr 172.21.15.59:0/2268742754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f098ea00640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09981049a0 0x7f099819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f0997e00640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998109120 0x7f099819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.643+0000 7f0997400640 1 -- 172.21.15.59:0/2268742754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09981049a0 msgr2=0x7f099819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0997400640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09981049a0 0x7f099819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0997400640 1 -- 172.21.15.59:0/2268742754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998109120 msgr2=0x7f099819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0997400640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998109120 0x7f099819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0997400640 1 -- 172.21.15.59:0/2268742754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099819f650 con 0x7f0998103ff0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0997400640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f098800e9f0 tx=0x7f098800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0994c00640 1 -- 172.21.15.59:0/2268742754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098800cd80 con 0x7f0998103ff0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0994c00640 1 -- 172.21.15.59:0/2268742754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988010910 con 0x7f0998103ff0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099819f940 con 0x7f0998103ff0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f0994c00640 1 -- 172.21.15.59:0/2268742754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988018990 con 0x7f0998103ff0 2024-09-05T12:09:46.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.644+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09981a7220 con 0x7f0998103ff0 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.647+0000 7f0994c00640 1 -- 172.21.15.59:0/2268742754 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0988010ac0 con 0x7f0998103ff0 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.647+0000 7f09a0e31740 1 Processor -- start 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 -- start start 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ecd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ed0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558676e29290 con 0x558676ec2ac0 2024-09-05T12:09:46.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558676dd0b40 con 0x558676ec33e0 2024-09-05T12:09:46.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f09a0e31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558676e25250 con 0x558676ecdca0 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f0997400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ecd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60246/0 (socket says 172.21.15.59:60246) 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f0997e00640 1 -- 172.21.15.59:0/1849605318 learned_addr learned my addr 172.21.15.59:0/1849605318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.648+0000 7f098ea00640 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ed0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 -- 172.21.15.59:0/1849605318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 msgr2=0x558676ed0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ed0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 -- 172.21.15.59:0/1849605318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 msgr2=0x558676ecd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ecd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 -- 172.21.15.59:0/1849605318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558676b9a6a0 con 0x558676ec2ac0 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0994c00640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 0x7f097c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997e00640 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0990009f50 tx=0x7f09900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ff638c41c6523 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0994c00640 1 -- 172.21.15.59:0/2268742754 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f09880cf7e0 con 0x7f0998103ff0 2024-09-05T12:09:46.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.649+0000 7f0997400640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 0x7f097c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f0987e00640 1 -- 172.21.15.59:0/1849605318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990004d20 con 0x558676ec2ac0 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f0987e00640 1 -- 172.21.15.59:0/1849605318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0990004050 con 0x558676ec2ac0 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f0987e00640 1 -- 172.21.15.59:0/1849605318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990005280 con 0x558676ec2ac0 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f09a0e31740 1 -- 172.21.15.59:0/1849605318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 msgr2=0x558676ec2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f09a0e31740 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0990009f50 tx=0x7f09900118d0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.650+0000 7f0997400640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 0x7f097c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09880a4310 tx=0x7f09880d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 -- 172.21.15.59:0/1849605318 shutdown_connections 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ed0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ecd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 --2- 172.21.15.59:0/1849605318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec2ea0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 -- 172.21.15.59:0/1849605318 >> 172.21.15.59:0/1849605318 conn(0x558676ebf050 msgr2=0x558676ebf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 -- 172.21.15.59:0/1849605318 shutdown_connections 2024-09-05T12:09:46.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 -- 172.21.15.59:0/1849605318 wait complete. 2024-09-05T12:09:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 -- start start 2024-09-05T12:09:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.651+0000 7f09a0e31740 1 Processor -- start 2024-09-05T12:09:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ecd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ecae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558676ed0910 con 0x558676ecdca0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558676de3fa0 con 0x558676ec33e0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558676ed0a90 con 0x558676ec2ac0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f09a0e31740 10 client.0 _ll_register_callbacks cb 0x558676b37ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f098ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f098ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:36954/0 (socket says 172.21.15.59:36954) 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f098ea00640 1 -- 172.21.15.59:0/2092123509 learned_addr learned my addr 172.21.15.59:0/2092123509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f0997e00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ecae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.652+0000 7f0997e00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ecd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 -- 172.21.15.59:0/2092123509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 msgr2=0x558676ecd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ecd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 -- 172.21.15.59:0/2092123509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 msgr2=0x558676ecae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ecae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 -- 172.21.15.59:0/2092123509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558676ecb6c0 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f098ea00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f098002a930 tx=0x7f09800317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d72a47f6d7c610 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f0984c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980035040 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0980004240 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.653+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0990004050 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f098004f050 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 10 client.27341 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558676edc890 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0990004050 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0980004a10 con 0x558676ec33e0 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 10 client.27341 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 10 client.27341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 20 client.27341 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676eddd50 0x558676ee0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 4 client.27341 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.654+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558676e36800 con 0x558676eddd50 2024-09-05T12:09:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.655+0000 7f0997e00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676eddd50 0x558676ee0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.655+0000 7f0997e00640 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676eddd50 0x558676ee0170 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48db58f1cdec9ba3 server_cookie=94dc1a4ba916a08f in_seq=0 out_seq=0 2024-09-05T12:09:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.655+0000 7f0984c00640 10 client.27341 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f0984c00640 1 -- 172.21.15.59:0/2092123509 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558676e36800 con 0x558676eddd50 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f0984c00640 10 client.27341 handle_command_reply: tid=0 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 client.27341 shutdown 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 msgr2=0x558676eca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f098002a930 tx=0x7f09800317e0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 shutdown_connections 2024-09-05T12:09:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676eddd50 0x558676ee0170 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ecae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676eca8f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 --2- 172.21.15.59:0/2092123509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ecd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 >> 172.21.15.59:0/2092123509 conn(0x558676ebf050 msgr2=0x558676ec4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f0984c00640 0 client.27341 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 shutdown_connections 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.656+0000 7f09a0e31740 1 -- 172.21.15.59:0/2092123509 wait complete. 2024-09-05T12:09:46.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.657+0000 7f09a0e31740 20 client.27341 trim_cache size 0 max 16384 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 Processor -- start 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 -- start start 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ee5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ee5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09981af150 con 0x558676ec2ac0 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09981b19d0 con 0x558676ec33e0 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f09a0e31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09981af7c0 con 0x558676ecdca0 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60270/0 (socket says 172.21.15.59:60270) 2024-09-05T12:09:46.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f0997e00640 1 -- 172.21.15.59:0/3866568461 learned_addr learned my addr 172.21.15.59:0/3866568461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.675+0000 7f098ea00640 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ee5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ee5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 -- 172.21.15.59:0/3866568461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 msgr2=0x558676ee5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ee5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 -- 172.21.15.59:0/3866568461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 msgr2=0x558676ee5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ee5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 -- 172.21.15.59:0/3866568461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558676ec0100 con 0x558676ec2ac0 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0997e00640 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0990009ef0 tx=0x7f0990009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5bb79401b330a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0987400640 1 -- 172.21.15.59:0/3866568461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990030040 con 0x558676ec2ac0 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0987400640 1 -- 172.21.15.59:0/3866568461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0990027cd0 con 0x558676ec2ac0 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.676+0000 7f0987400640 1 -- 172.21.15.59:0/3866568461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000fb10 con 0x558676ec2ac0 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 -- 172.21.15.59:0/3866568461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 msgr2=0x558676ec97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0990009ef0 tx=0x7f0990009500 comp rx=0 tx=0).stop 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 -- 172.21.15.59:0/3866568461 shutdown_connections 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ecdca0 0x558676ee5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ee5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3866568461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ec2ac0 0x558676ec97a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 -- 172.21.15.59:0/3866568461 >> 172.21.15.59:0/3866568461 conn(0x558676ebf050 msgr2=0x558676ec9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 -- 172.21.15.59:0/3866568461 shutdown_connections 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.677+0000 7f09a0e31740 1 -- 172.21.15.59:0/3866568461 wait complete. 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 -- start start 2024-09-05T12:09:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 Processor -- start 2024-09-05T12:09:46.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ed6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ed2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09981afe90 con 0x558676ecdca0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09981b1300 con 0x558676ec33e0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09981b0560 con 0x558676ec2ac0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.678+0000 7f09a0e31740 10 client.0 _ll_register_callbacks cb 0x7f0980009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0997e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60274/0 (socket says 172.21.15.59:60274) 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ed2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0997e00640 1 -- 172.21.15.59:0/3641691822 learned_addr learned my addr 172.21.15.59:0/3641691822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0997e00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ed6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 -- 172.21.15.59:0/3641691822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 msgr2=0x558676ed6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ed6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 -- 172.21.15.59:0/3641691822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 msgr2=0x558676ed31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 -- 172.21.15.59:0/3641691822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558676ecb890 con 0x558676ec33e0 2024-09-05T12:09:46.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0997e00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f098ea00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ed2cb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x558676ecbea0 tx=0x7f0980003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff83dd41565a51aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.679+0000 7f0984c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980035070 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f09800048d0 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0990027cd0 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0980063050 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f09a0e31740 10 client.27347 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558676f496a0 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0990027cd0 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.680+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0980046480 con 0x558676ec33e0 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 10 client.27347 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 10 client.27347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:46.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 20 client.27347 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:46.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676f49e50 0x558676f4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:46.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 4 client.27347 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:09:46.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558676e30d10 con 0x558676f49e50 2024-09-05T12:09:46.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f0997e00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676f49e50 0x558676f4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f0997e00640 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676f49e50 0x558676f4c270 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446e79ec50544f02 server_cookie=2f33106346265afa in_seq=0 out_seq=0 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.681+0000 7f0984c00640 10 client.27347 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.682+0000 7f0984c00640 1 -- 172.21.15.59:0/3641691822 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558676e30d10 con 0x558676f49e50 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.682+0000 7f0984c00640 10 client.27347 handle_command_reply: tid=0 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.682+0000 7f09a0e31740 1 client.27347 shutdown 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 msgr2=0x558676ed2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ed2cb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x558676ecbea0 tx=0x7f0980003820 comp rx=0 tx=0).stop 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 shutdown_connections 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x558676f49e50 0x558676f4c270 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558676ecdca0 0x558676ed31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558676ec33e0 0x558676ed2cb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 --2- 172.21.15.59:0/3641691822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558676ec2ac0 0x558676ed6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f0984c00640 0 client.27347 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 >> 172.21.15.59:0/3641691822 conn(0x558676ebf050 msgr2=0x558676ebe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 shutdown_connections 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 1 -- 172.21.15.59:0/3641691822 wait complete. 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.683+0000 7f09a0e31740 20 client.27347 trim_cache size 0 max 16384 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "4c9f7ec7-8545-4f59-a171-99fc0a067315", 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:09:46.687 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 msgr2=0x7f097c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 0x7f097c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09880a4310 tx=0x7f09880d5530 comp rx=0 tx=0).stop 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 msgr2=0x7f099819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f098800e9f0 tx=0x7f098800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 shutdown_connections 2024-09-05T12:09:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f097c0a2b80 0x7f097c0a5040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0998109120 0x7f099819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09981049a0 0x7f099819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 --2- 172.21.15.59:0/2268742754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0998103ff0 0x7f099819a6a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 >> 172.21.15.59:0/2268742754 conn(0x7f09980ffa90 msgr2=0x7f0998100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 shutdown_connections 2024-09-05T12:09:46.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:46.684+0000 7f099e200640 1 -- 172.21.15.59:0/2268742754 wait complete. 2024-09-05T12:09:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:46 smithi089 ceph-mon[34499]: pgmap v757: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:46 smithi059 ceph-mon[30025]: pgmap v757: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:47.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:46 smithi189 ceph-mon[34772]: pgmap v757: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:47.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:09:47.788 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:47 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:09:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:47 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:47 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:09:48.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:47 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:47 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:09:48.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:47 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:47 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:09:48.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:47 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:47 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:09:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:47 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:47 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:09:48.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:47 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.348+0000 7f34b4800640 1 -- 172.21.15.59:0/1500159450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 msgr2=0x7f34ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.348+0000 7f34b4800640 1 --2- 172.21.15.59:0/1500159450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac1021c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009860 tx=0x7f34a002a340 comp rx=0 tx=0).stop 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 -- 172.21.15.59:0/1500159450 shutdown_connections 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 --2- 172.21.15.59:0/1500159450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac106f10 0x7f34ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 --2- 172.21.15.59:0/1500159450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac102790 0x7f34ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 --2- 172.21.15.59:0/1500159450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac1021c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 -- 172.21.15.59:0/1500159450 >> 172.21.15.59:0/1500159450 conn(0x7f34ac0fd8c0 msgr2=0x7f34ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 -- 172.21.15.59:0/1500159450 shutdown_connections 2024-09-05T12:09:48.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.349+0000 7f34b4800640 1 -- 172.21.15.59:0/1500159450 wait complete. 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 Processor -- start 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 -- start start 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 0x7f34ac19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac106f10 0x7f34ac19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac111f70 con 0x7f34ac101de0 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac111df0 con 0x7f34ac102790 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.350+0000 7f34b4800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1120f0 con 0x7f34ac106f10 2024-09-05T12:09:48.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60296/0 (socket says 172.21.15.59:60296) 2024-09-05T12:09:48.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 -- 172.21.15.59:0/535830022 learned_addr learned my addr 172.21.15.59:0/535830022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:48.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1400640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 0x7f34ac19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b2800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac106f10 0x7f34ac19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 -- 172.21.15.59:0/535830022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac106f10 msgr2=0x7f34ac19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac106f10 0x7f34ac19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 -- 172.21.15.59:0/535830022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 msgr2=0x7f34ac19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 0x7f34ac19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 -- 172.21.15.59:0/535830022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac19f5e0 con 0x7f34ac101de0 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1400640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 0x7f34ac19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.351+0000 7f34b1e00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f34a0035040 tx=0x7f34a0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.352+0000 7f34a6a00640 1 -- 172.21.15.59:0/535830022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a003f020 con 0x7f34ac101de0 2024-09-05T12:09:48.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.352+0000 7f34a6a00640 1 -- 172.21.15.59:0/535830022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a0032770 con 0x7f34ac101de0 2024-09-05T12:09:48.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.352+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac19f870 con 0x7f34ac101de0 2024-09-05T12:09:48.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.352+0000 7f34a6a00640 1 -- 172.21.15.59:0/535830022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0037040 con 0x7f34ac101de0 2024-09-05T12:09:48.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.352+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a71b0 con 0x7f34ac101de0 2024-09-05T12:09:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.356+0000 7f34b73dc740 1 Processor -- start 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.356+0000 7f34b73dc740 1 -- start start 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34a6a00640 1 -- 172.21.15.59:0/535830022 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f34a0042020 con 0x7f34ac101de0 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c461160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f87c3ba3e0 con 0x55f87c453b90 2024-09-05T12:09:48.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f87c361d80 con 0x55f87c45ed70 2024-09-05T12:09:48.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b73dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f87c3b63a0 con 0x55f87c4544b0 2024-09-05T12:09:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60300/0 (socket says 172.21.15.59:60300) 2024-09-05T12:09:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.357+0000 7f34b2800640 1 -- 172.21.15.59:0/3742880962 learned_addr learned my addr 172.21.15.59:0/3742880962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b1400640 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c461160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 -- 172.21.15.59:0/3742880962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 msgr2=0x55f87c45e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 -- 172.21.15.59:0/3742880962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 msgr2=0x55f87c461160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c461160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 -- 172.21.15.59:0/3742880962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c380ae0 con 0x55f87c453b90 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34b2800640 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009f50 tx=0x7f34a80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd52c50794c3e24c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f34a6a00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 0x7f34940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.358+0000 7f349b000640 1 -- 172.21.15.59:0/3742880962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8004d20 con 0x55f87c453b90 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f349b000640 1 -- 172.21.15.59:0/3742880962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a8004050 con 0x55f87c453b90 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f349b000640 1 -- 172.21.15.59:0/3742880962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8005280 con 0x55f87c453b90 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f34a6a00640 1 -- 172.21.15.59:0/535830022 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f34a00ef250 con 0x7f34ac101de0 2024-09-05T12:09:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f34b1e00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 0x7f34940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f34b73dc740 1 -- 172.21.15.59:0/3742880962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 msgr2=0x55f87c453f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.359+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009f50 tx=0x7f34a80118d0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 -- 172.21.15.59:0/3742880962 shutdown_connections 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c461160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b1e00640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 0x7f34940a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f34a00b7aa0 tx=0x7f34a00f2c90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3742880962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c453f70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 -- 172.21.15.59:0/3742880962 >> 172.21.15.59:0/3742880962 conn(0x55f87c450120 msgr2=0x55f87c450500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 -- 172.21.15.59:0/3742880962 shutdown_connections 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 -- 172.21.15.59:0/3742880962 wait complete. 2024-09-05T12:09:48.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 -- start start 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.360+0000 7f34b73dc740 1 Processor -- start 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c45bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f87c4619e0 con 0x55f87c45ed70 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f87c3750f0 con 0x55f87c453b90 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f87c461b60 con 0x55f87c4544b0 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b73dc740 10 client.0 _ll_register_callbacks cb 0x55f87c3669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60310/0 (socket says 172.21.15.59:60310) 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b2800640 1 -- 172.21.15.59:0/2916221937 learned_addr learned my addr 172.21.15.59:0/2916221937 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b1400640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.361+0000 7f34b2800640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c45bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 -- 172.21.15.59:0/2916221937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 msgr2=0x55f87c45c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 -- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 msgr2=0x55f87c45bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c45bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 -- 172.21.15.59:0/2916221937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87c12b3c0 con 0x55f87c45ed70 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b2800640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009250 tx=0x7f34a8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f46ba597c09ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f348f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a802b020 con 0x55f87c45ed70 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a8004050 con 0x55f87c45ed70 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.362+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f34a8027640 con 0x55f87c45ed70 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f34a8036050 con 0x55f87c45ed70 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 10 client.17358 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:48.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f87c46d690 con 0x55f87c45ed70 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a8004050 con 0x55f87c45ed70 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f34a800f780 con 0x55f87c45ed70 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 10 client.17358 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 20 client.17358 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c46eb50 0x55f87c470f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 4 client.17358 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.363+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f87c3c7950 con 0x55f87c46eb50 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.364+0000 7f34b1400640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c46eb50 0x55f87c470f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.364+0000 7f34b1400640 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c46eb50 0x55f87c470f40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c635826eae9cc7 server_cookie=1ccdf239367bd052 in_seq=0 out_seq=0 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.364+0000 7f348f400640 10 client.17358 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f348f400640 1 -- 172.21.15.59:0/2916221937 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f87c3c7950 con 0x55f87c46eb50 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f348f400640 10 client.17358 handle_command_reply: tid=0 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 client.17358 shutdown 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 msgr2=0x55f87c45c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009250 tx=0x7f34a8005bd0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 shutdown_connections 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c46eb50 0x55f87c470f40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c45c6e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c45c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 --2- 172.21.15.59:0/2916221937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c45bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f348f400640 0 client.17358 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.365+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 >> 172.21.15.59:0/2916221937 conn(0x55f87c450120 msgr2=0x55f87c455e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.366+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 shutdown_connections 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.366+0000 7f34b73dc740 1 -- 172.21.15.59:0/2916221937 wait complete. 2024-09-05T12:09:48.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.366+0000 7f34b73dc740 20 client.17358 trim_cache size 0 max 16384 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 Processor -- start 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 -- start start 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c476c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c45bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1af0b0 con 0x55f87c453b90 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1b1930 con 0x55f87c45ed70 2024-09-05T12:09:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b73dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1af720 con 0x55f87c4544b0 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60324/0 (socket says 172.21.15.59:60324) 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.384+0000 7f34b2800640 1 -- 172.21.15.59:0/3839482256 learned_addr learned my addr 172.21.15.59:0/3839482256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b1400640 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c476c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 -- 172.21.15.59:0/3839482256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 msgr2=0x55f87c476c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c476c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 -- 172.21.15.59:0/3839482256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 msgr2=0x55f87c45bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c45bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 -- 172.21.15.59:0/3839482256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c4512b0 con 0x55f87c453b90 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f34b2800640 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f34a80049d0 tx=0x7f34a8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa10438efb486ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f349a600640 1 -- 172.21.15.59:0/3839482256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a802b070 con 0x55f87c453b90 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f349a600640 1 -- 172.21.15.59:0/3839482256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a8015030 con 0x55f87c453b90 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.385+0000 7f349a600640 1 -- 172.21.15.59:0/3839482256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8027730 con 0x55f87c453b90 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 -- 172.21.15.59:0/3839482256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 msgr2=0x55f87c4766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f34a80049d0 tx=0x7f34a8009ed0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 -- 172.21.15.59:0/3839482256 shutdown_connections 2024-09-05T12:09:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c45ed70 0x55f87c45bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c476c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 --2- 172.21.15.59:0/3839482256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c453b90 0x55f87c4766d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 -- 172.21.15.59:0/3839482256 >> 172.21.15.59:0/3839482256 conn(0x55f87c450120 msgr2=0x55f87c45ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 -- 172.21.15.59:0/3839482256 shutdown_connections 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.386+0000 7f34b73dc740 1 -- 172.21.15.59:0/3839482256 wait complete. 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 -- start start 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 Processor -- start 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c466d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c4672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1afdf0 con 0x55f87c45ed70 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1b1260 con 0x55f87c453b90 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1b04c0 con 0x55f87c4544b0 2024-09-05T12:09:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.387+0000 7f34b73dc740 10 client.0 _ll_register_callbacks cb 0x7f34a8004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:48.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60328/0 (socket says 172.21.15.59:60328) 2024-09-05T12:09:48.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 -- 172.21.15.59:0/85317775 learned_addr learned my addr 172.21.15.59:0/85317775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:48.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b1400640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c4672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c466d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 -- 172.21.15.59:0/85317775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 msgr2=0x55f87c4672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c4672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 -- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 msgr2=0x55f87c466d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c466d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 -- 172.21.15.59:0/85317775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87c458a50 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b1400640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c4672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f34b2800640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f34a800fc40 tx=0x7f34a8032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b8b38869448308 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.388+0000 7f348f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a802b070 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a8015030 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f34a800f680 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f34a800f8a0 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f34b73dc740 10 client.17364 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f87c4d8f50 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.389+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a8015030 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f34a803a430 con 0x55f87c45ed70 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 10 client.17364 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 20 client.17364 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c4d9460 0x55f87c4db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 4 client.17364 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f87c3c1e60 con 0x55f87c4d9460 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b1400640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c4d9460 0x55f87c4db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f34b1400640 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c4d9460 0x55f87c4db880 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0f10b29be9dffc server_cookie=c68b1625108cb1d7 in_seq=0 out_seq=0 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.390+0000 7f348f400640 10 client.17364 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f348f400640 1 -- 172.21.15.59:0/85317775 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f87c3c1e60 con 0x55f87c4d9460 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f348f400640 10 client.17364 handle_command_reply: tid=0 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 client.17364 shutdown 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 msgr2=0x55f87c458490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f34a800fc40 tx=0x7f34a8032040 comp rx=0 tx=0).stop 2024-09-05T12:09:48.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 shutdown_connections 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f87c4d9460 0x55f87c4db880 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f87c45ed70 0x55f87c458490 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f87c4544b0 0x55f87c4672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 --2- 172.21.15.59:0/85317775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f87c453b90 0x55f87c466d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f348f400640 0 client.17364 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 >> 172.21.15.59:0/85317775 conn(0x55f87c450120 msgr2=0x55f87c45bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 shutdown_connections 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 1 -- 172.21.15.59:0/85317775 wait complete. 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.391+0000 7f34b73dc740 20 client.17364 trim_cache size 0 max 16384 2024-09-05T12:09:48.396 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "4c9f7ec7-8545-4f59-a171-99fc0a067315": { 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "4c9f7ec7-8545-4f59-a171-99fc0a067315", 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:48.397 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 msgr2=0x7f34940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 0x7f34940a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f34a00b7aa0 tx=0x7f34a00f2c90 comp rx=0 tx=0).stop 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 msgr2=0x7f34ac19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f34a0035040 tx=0x7f34a0033d90 comp rx=0 tx=0).stop 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 shutdown_connections 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f34940a2c00 0x7f34940a50c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34ac106f10 0x7f34ac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34ac102790 0x7f34ac19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 --2- 172.21.15.59:0/535830022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34ac101de0 0x7f34ac19a630 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:48.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.392+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 >> 172.21.15.59:0/535830022 conn(0x7f34ac0fd8c0 msgr2=0x7f34ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:48.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.393+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 shutdown_connections 2024-09-05T12:09:48.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:48.393+0000 7f34b4800640 1 -- 172.21.15.59:0/535830022 wait complete. 2024-09-05T12:09:49.090 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:48 smithi059 ceph-mon[30025]: pgmap v758: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:49.128 INFO:tasks.cephfs.filesystem:scrub status for tag:4c9f7ec7-8545-4f59-a171-99fc0a067315 - {'path': '/', 'tag': '4c9f7ec7-8545-4f59-a171-99fc0a067315', 'options': 'recursive,force'} 2024-09-05T12:09:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:48 smithi089 ceph-mon[34499]: pgmap v758: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:49.352 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:48 smithi189 ceph-mon[34772]: pgmap v758: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:50 smithi089 ceph-mon[34499]: pgmap v759: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:50 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:09:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:50 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:09:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:50 smithi059 ceph-mon[30025]: pgmap v759: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:50 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:09:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:50 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:09:51.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:50 smithi189 ceph-mon[34772]: pgmap v759: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:50 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:09:51.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:50 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:09:51.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:09:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:09:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:09:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:09:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:09:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:09:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:09:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:09:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:09:52.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 -- 172.21.15.59:0/2897040307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 msgr2=0x7f12441043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 --2- 172.21.15.59:0/2897040307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441043f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1234009900 tx=0x7f123402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 -- 172.21.15.59:0/2897040307 shutdown_connections 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 --2- 172.21.15.59:0/2897040307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1244109140 0x7f1244110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 --2- 172.21.15.59:0/2897040307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12441049c0 0x7f1244108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 --2- 172.21.15.59:0/2897040307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441043f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 -- 172.21.15.59:0/2897040307 >> 172.21.15.59:0/2897040307 conn(0x7f12440ffa50 msgr2=0x7f1244101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 -- 172.21.15.59:0/2897040307 shutdown_connections 2024-09-05T12:09:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.540+0000 7f124a200640 1 -- 172.21.15.59:0/2897040307 wait complete. 2024-09-05T12:09:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 Processor -- start 2024-09-05T12:09:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 -- start start 2024-09-05T12:09:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12441049c0 0x7f12441a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 0x7f124419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1244114140 con 0x7f1244104010 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1244113fc0 con 0x7f12441049c0 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.541+0000 7f124a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12441142c0 con 0x7f1244109140 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59870/0 (socket says 172.21.15.59:59870) 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243400640 1 -- 172.21.15.59:0/3302594486 learned_addr learned my addr 172.21.15.59:0/3302594486 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f123be00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12441049c0 0x7f12441a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 0x7f124419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 -- 172.21.15.59:0/3302594486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12441049c0 msgr2=0x7f12441a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12441049c0 0x7f12441a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 -- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 msgr2=0x7f12441a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243e00640 1 -- 172.21.15.59:0/3302594486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124419acc0 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.542+0000 7f1243400640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f1243e00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 0x7f124419a590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f123c00da10 tx=0x7f123c00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c00bd70 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124419afb0 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f123c003e50 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0106a0 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.543+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12441a72a0 con 0x7f1244109140 2024-09-05T12:09:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.544+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12441043f0 con 0x7f1244109140 2024-09-05T12:09:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.546+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f123c004b40 con 0x7f1244109140 2024-09-05T12:09:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.548+0000 7f1240c00640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 0x7f12280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.548+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f123c0cfad0 con 0x7f1244109140 2024-09-05T12:09:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.548+0000 7f1243400640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 0x7f12280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.549+0000 7f1243400640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 0x7f12280a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f12340096c0 tx=0x7f1234005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:52.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.550+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f123c09c640 con 0x7f1244109140 2024-09-05T12:09:52.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.770+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f124419bac0 con 0x7f1244109140 2024-09-05T12:09:52.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.771+0000 7f1240c00640 1 -- 172.21.15.59:0/3302594486 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f124419bac0 con 0x7f1244109140 2024-09-05T12:09:52.773 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:09:52.773 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:09:52.774 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 msgr2=0x7f12280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 0x7f12280a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f12340096c0 tx=0x7f1234005a20 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 msgr2=0x7f124419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 0x7f124419a590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f123c00da10 tx=0x7f123c00ded0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 shutdown_connections 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f12280a2c00 0x7f12280a50c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1244109140 0x7f124419a590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12441049c0 0x7f12441a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 --2- 172.21.15.59:0/3302594486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1244104010 0x7f12441a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 >> 172.21.15.59:0/3302594486 conn(0x7f12440ffa50 msgr2=0x7f1244100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 shutdown_connections 2024-09-05T12:09:52.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:52.774+0000 7f124a200640 1 -- 172.21.15.59:0/3302594486 wait complete. 2024-09-05T12:09:53.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:52 smithi059 ceph-mon[30025]: pgmap v760: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:53.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:52 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3302594486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:52 smithi089 ceph-mon[34499]: pgmap v760: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:52 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3302594486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:52 smithi189 ceph-mon[34772]: pgmap v760: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-05T12:09:53.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:52 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3302594486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:09:53.573 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id b68fb54b --release --await' 2024-09-05T12:09:53.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id b68fb54b --release --await 2024-09-05T12:09:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 -- 172.21.15.59:0/1438661165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894103ff0 msgr2=0x7fd8941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 --2- 172.21.15.59:0/1438661165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894103ff0 0x7fd8941043d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 -- 172.21.15.59:0/1438661165 shutdown_connections 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 --2- 172.21.15.59:0/1438661165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd894109120 0x7fd894110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 --2- 172.21.15.59:0/1438661165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8941049a0 0x7fd8941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 --2- 172.21.15.59:0/1438661165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894103ff0 0x7fd8941043d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 -- 172.21.15.59:0/1438661165 >> 172.21.15.59:0/1438661165 conn(0x7fd8940ffa90 msgr2=0x7fd894101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 -- 172.21.15.59:0/1438661165 shutdown_connections 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.550+0000 7fd89be00640 1 -- 172.21.15.59:0/1438661165 wait complete. 2024-09-05T12:09:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.551+0000 7fd89be00640 1 Processor -- start 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.551+0000 7fd89be00640 1 -- start start 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.551+0000 7fd89be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.551+0000 7fd89be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941049a0 0x7fd89419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.551+0000 7fd89be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 0x7fd89419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd89be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd894114120 con 0x7fd894109120 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd89be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd894113fa0 con 0x7fd8941049a0 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd89be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941142a0 con 0x7fd894103ff0 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 0x7fd89419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:54510/0 (socket says 172.21.15.59:54510) 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 0x7fd89419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59886/0 (socket says 172.21.15.59:59886) 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 -- 172.21.15.59:0/3860420369 learned_addr learned my addr 172.21.15.59:0/3860420369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd898a00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941049a0 0x7fd89419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 -- 172.21.15.59:0/3860420369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941049a0 msgr2=0x7fd89419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941049a0 0x7fd89419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 -- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 msgr2=0x7fd89419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 0x7fd89419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.552+0000 7fd899400640 1 -- 172.21.15.59:0/3860420369 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89419f5b0 con 0x7fd894103ff0 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd899400640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd892000640 1 -- 172.21.15.59:0/3860420369 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88803f030 con 0x7fd894103ff0 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd892000640 1 -- 172.21.15.59:0/3860420369 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888003ce0 con 0x7fd894103ff0 2024-09-05T12:09:54.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89419f840 con 0x7fd894103ff0 2024-09-05T12:09:54.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941a7180 con 0x7fd894103ff0 2024-09-05T12:09:54.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.553+0000 7fd892000640 1 -- 172.21.15.59:0/3860420369 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x7fd894103ff0 2024-09-05T12:09:54.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.556+0000 7fd89e9c1740 1 Processor -- start 2024-09-05T12:09:54.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd892000640 1 -- 172.21.15.59:0/3860420369 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd888042030 con 0x7fd894103ff0 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 -- start start 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56510e821650 con 0x56510e8bbc30 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56510e7c8f10 con 0x56510e8c64f0 2024-09-05T12:09:54.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56510e81d610 con 0x56510e8bb310 2024-09-05T12:09:54.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd899e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd899400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd899e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:54520/0 (socket says 172.21.15.59:54520) 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.557+0000 7fd899e00640 1 -- 172.21.15.59:0/3744924654 learned_addr learned my addr 172.21.15.59:0/3744924654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd898a00640 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 -- 172.21.15.59:0/3744924654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 msgr2=0x56510e8c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 -- 172.21.15.59:0/3744924654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 msgr2=0x56510e8c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 -- 172.21.15.59:0/3744924654 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56510e530580 con 0x56510e8bb310 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.558+0000 7fd899e00640 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009f90 tx=0x7fd88c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e93acb748132abe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd892000640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd886600640 1 -- 172.21.15.59:0/3744924654 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c011700 con 0x56510e8bb310 2024-09-05T12:09:54.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd886600640 1 -- 172.21.15.59:0/3744924654 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c011d80 con 0x56510e8bb310 2024-09-05T12:09:54.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd886600640 1 -- 172.21.15.59:0/3744924654 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c018380 con 0x56510e8bb310 2024-09-05T12:09:54.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd892000640 1 -- 172.21.15.59:0/3860420369 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd8880efac0 con 0x7fd894103ff0 2024-09-05T12:09:54.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.559+0000 7fd899400640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3744924654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 msgr2=0x56510e8bb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd899400640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 0x7fd87c0a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd8880bbfb0 tx=0x7fd8880f34a0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009f90 tx=0x7fd88c010860 comp rx=0 tx=0).stop 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3744924654 shutdown_connections 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3744924654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8bb6f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3744924654 >> 172.21.15.59:0/3744924654 conn(0x56510e8b78a0 msgr2=0x56510e8b7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3744924654 shutdown_connections 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.560+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3744924654 wait complete. 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.561+0000 7fd89e9c1740 1 -- start start 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.561+0000 7fd89e9c1740 1 Processor -- start 2024-09-05T12:09:54.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 0x56510e8c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8c64f0 0x56510e8c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56510e8c90c0 con 0x56510e8c64f0 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56510e7dc350 con 0x56510e8bbc30 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56510e8c9240 con 0x56510e8bb310 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd89e9c1740 10 client.0 _ll_register_callbacks cb 0x56510e7ff290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd899e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd899e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:54522/0 (socket says 172.21.15.59:54522) 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd899e00640 1 -- 172.21.15.59:0/3730073341 learned_addr learned my addr 172.21.15.59:0/3730073341 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd898a00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 0x56510e8c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.562+0000 7fd899e00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8c64f0 0x56510e8c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 -- 172.21.15.59:0/3730073341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 msgr2=0x56510e8c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 -- 172.21.15.59:0/3730073341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8c64f0 msgr2=0x56510e8c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8c64f0 0x56510e8c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 -- 172.21.15.59:0/3730073341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56510e5fa960 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd899e00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd898a00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 0x56510e8c3990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd88000baa0 tx=0x7fd88000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c1e9386c681f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd87aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880003df0 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.563+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd88c011d80 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd880005020 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd880024050 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd89e9c1740 10 client.27380 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56510e8d4df0 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c011d80 con 0x56510e8bbc30 2024-09-05T12:09:54.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.564+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd88001b470 con 0x56510e8bbc30 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 10 client.27380 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 10 client.27380 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 20 client.27380 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e8d6250 0x56510e8d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 4 client.27380 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56510e82ebc0 con 0x56510e8d6250 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd899e00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e8d6250 0x56510e8d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd899e00640 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e8d6250 0x56510e8d8670 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd0194cfb1c1444 server_cookie=ecd5392891e60446 in_seq=0 out_seq=0 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.565+0000 7fd87aa00640 10 client.27380 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.566+0000 7fd87aa00640 1 -- 172.21.15.59:0/3730073341 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56510e82ebc0 con 0x56510e8d6250 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.566+0000 7fd87aa00640 10 client.27380 handle_command_reply: tid=0 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.566+0000 7fd89e9c1740 1 client.27380 shutdown 2024-09-05T12:09:54.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.566+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 msgr2=0x56510e8c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.566+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 0x56510e8c3990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd88000baa0 tx=0x7fd88000bf60 comp rx=0 tx=0).stop 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 shutdown_connections 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e8d6250 0x56510e8d8670 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8c64f0 0x56510e8c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bbc30 0x56510e8c3990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/3730073341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd87aa00640 0 client.27380 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 >> 172.21.15.59:0/3730073341 conn(0x56510e8b78a0 msgr2=0x56510e8bd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 shutdown_connections 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 1 -- 172.21.15.59:0/3730073341 wait complete. 2024-09-05T12:09:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.567+0000 7fd89e9c1740 20 client.27380 trim_cache size 0 max 16384 2024-09-05T12:09:54.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 Processor -- start 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 -- start start 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941af090 con 0x56510e8bbc30 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941b1910 con 0x56510e8c64f0 2024-09-05T12:09:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.585+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941af700 con 0x56510e8bb310 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd898a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd898a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59916/0 (socket says 172.21.15.59:59916) 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd898a00640 1 -- 172.21.15.59:0/747693230 learned_addr learned my addr 172.21.15.59:0/747693230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 -- 172.21.15.59:0/747693230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 msgr2=0x56510e8de480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8de480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 -- 172.21.15.59:0/747693230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 msgr2=0x56510e8ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd899e00640 1 -- 172.21.15.59:0/747693230 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56510e8b3050 con 0x56510e8bb310 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.586+0000 7fd898a00640 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd899e00640 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c52b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56510e8d8ee0 tx=0x7fd88c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44db25fe8bcb844b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd885c00640 1 -- 172.21.15.59:0/747693230 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c019b60 con 0x56510e8bb310 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd885c00640 1 -- 172.21.15.59:0/747693230 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c020040 con 0x56510e8bb310 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd885c00640 1 -- 172.21.15.59:0/747693230 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c005300 con 0x56510e8bb310 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd89e9c1740 1 -- 172.21.15.59:0/747693230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 msgr2=0x56510e8c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.587+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c52b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56510e8d8ee0 tx=0x7fd88c0096a0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 -- 172.21.15.59:0/747693230 shutdown_connections 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8c64f0 0x56510e8de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/747693230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8bb310 0x56510e8c52b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 -- 172.21.15.59:0/747693230 >> 172.21.15.59:0/747693230 conn(0x56510e8b78a0 msgr2=0x56510e8c4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 -- 172.21.15.59:0/747693230 shutdown_connections 2024-09-05T12:09:54.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 -- 172.21.15.59:0/747693230 wait complete. 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.588+0000 7fd89e9c1740 1 -- start start 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 Processor -- start 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 0x56510e8cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8c64f0 0x56510e8ca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941afdd0 con 0x56510e8bbc30 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941b1240 con 0x56510e8bb310 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941b04a0 con 0x56510e8c64f0 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd89e9c1740 10 client.0 _ll_register_callbacks cb 0x7fd880004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd898a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd899e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 0x56510e8cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd898a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59930/0 (socket says 172.21.15.59:59930) 2024-09-05T12:09:54.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.589+0000 7fd898a00640 1 -- 172.21.15.59:0/1348873058 learned_addr learned my addr 172.21.15.59:0/1348873058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd899e00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8c64f0 0x56510e8ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 -- 172.21.15.59:0/1348873058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8c64f0 msgr2=0x56510e8ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8c64f0 0x56510e8ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 -- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 msgr2=0x56510e8cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 0x56510e8cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 -- 172.21.15.59:0/1348873058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56510e8cac90 con 0x56510e8bbc30 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd899e00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 0x56510e8cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd898a00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd88001ba30 tx=0x7fd880005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2ed9bf08f96673 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.590+0000 7fd87aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880003e70 con 0x56510e8bbc30 2024-09-05T12:09:54.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd88001b470 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd88c020040 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd88001ba60 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd89e9c1740 10 client.17388 fetch_fsmap learned FSMap version 21 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56510e940fb0 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.591+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c020040 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd880004660 con 0x56510e8bbc30 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 10 client.17388 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 10 client.17388 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:09:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 20 client.17388 populate_metadata read hostname 'smithi059' 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e9417b0 0x56510e943bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 4 client.17388 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b68fb54b", "release": true, "await": true}] 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b68fb54b", "release": true, "await": true}) -- 0x56510e8290d0 con 0x56510e9417b0 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.592+0000 7fd899e00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e9417b0 0x56510e943bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.593+0000 7fd899e00640 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e9417b0 0x56510e943bd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad94989bb70e6eb0 server_cookie=55fe2386e7a5b8d4 in_seq=0 out_seq=0 2024-09-05T12:09:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.593+0000 7fd87aa00640 10 client.17388 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd87aa00640 1 -- 172.21.15.59:0/1348873058 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56510e8290d0 con 0x56510e9417b0 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd87aa00640 10 client.17388 handle_command_reply: tid=0 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 client.17388 shutdown 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 msgr2=0x56510e8c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd88001ba30 tx=0x7fd880005d00 comp rx=0 tx=0).stop 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 shutdown_connections 2024-09-05T12:09:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56510e8c64f0 0x56510e8ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56510e9417b0 0x56510e943bd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56510e8bbc30 0x56510e8c9f50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 --2- 172.21.15.59:0/1348873058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56510e8bb310 0x56510e8cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 >> 172.21.15.59:0/1348873058 conn(0x56510e8b78a0 msgr2=0x56510e8b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.594+0000 7fd87aa00640 0 client.17388 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 shutdown_connections 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89e9c1740 1 -- 172.21.15.59:0/1348873058 wait complete. 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89e9c1740 20 client.17388 trim_cache size 0 max 16384 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:09:54.599 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 44, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "b68fb54b": { 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 44, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 150.0, 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:09:54.600 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:09:54.601 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 msgr2=0x7fd87c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 0x7fd87c0a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd8880bbfb0 tx=0x7fd8880f34a0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.595+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 msgr2=0x7fd89419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880042f0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 shutdown_connections 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd87c0a2b80 0x7fd87c0a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd894109120 0x7fd89419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941049a0 0x7fd89419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 --2- 172.21.15.59:0/3860420369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd894103ff0 0x7fd89419a600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:09:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 >> 172.21.15.59:0/3860420369 conn(0x7fd8940ffa90 msgr2=0x7fd894100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:09:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 shutdown_connections 2024-09-05T12:09:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:09:54.596+0000 7fd89be00640 1 -- 172.21.15.59:0/3860420369 wait complete. 2024-09-05T12:09:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:54 smithi089 ceph-mon[34499]: pgmap v761: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:09:55.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:54 smithi059 ceph-mon[30025]: pgmap v761: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:09:55.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:54 smithi189 ceph-mon[34772]: pgmap v761: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-05T12:09:55.350 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b68fb54b', total seconds elapsed: 33.7 2024-09-05T12:09:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:56 smithi089 ceph-mon[34499]: pgmap v762: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T12:09:57.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:56 smithi059 ceph-mon[30025]: pgmap v762: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T12:09:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:56 smithi189 ceph-mon[34772]: pgmap v762: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:375/600: open fd = 3 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:376/600: open fd = 3 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.631 INFO:tasks.workunit.client.0.smithi059.stdout:377/600: open fd = 3 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:378/600: open fd = 3 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:379/600: open fd = 3 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:380/600: open fd = 3 2024-09-05T12:09:58.632 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:381/600: open fd = 3 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:382/600: open fd = 3 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:383/600: open fd = 3 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.633 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:384/600: open fd = 3 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:385/600: open fd = 3 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:386/600: open fd = 3 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:387/600: open fd = 3 2024-09-05T12:09:58.634 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:388/600: open fd = 3 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:389/600: open fd = 3 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:390/600: open fd = 3 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.635 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:391/600: open fd = 3 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:392/600: open fd = 3 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:393/600: open fd = 3 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.636 INFO:tasks.workunit.client.0.smithi059.stdout:394/600: open fd = 3 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:395/600: open fd = 3 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:396/600: open fd = 3 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:397/600: open fd = 3 2024-09-05T12:09:58.637 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:398/600: open fd = 3 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:399/600: open fd = 3 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:400/600: open fd = 3 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.638 INFO:tasks.workunit.client.0.smithi059.stdout:401/600: open fd = 3 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:402/600: open fd = 3 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:403/600: open fd = 3 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:404/600: open fd = 3 2024-09-05T12:09:58.639 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:405/600: open fd = 3 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:406/600: open fd = 3 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:407/600: open fd = 3 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.640 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:408/600: open fd = 3 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:409/600: open fd = 3 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:410/600: open fd = 3 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:411/600: open fd = 3 2024-09-05T12:09:58.641 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:412/600: open fd = 3 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:413/600: open fd = 3 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:414/600: open fd = 3 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.642 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:415/600: open fd = 3 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:416/600: open fd = 3 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:417/600: open fd = 3 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.643 INFO:tasks.workunit.client.0.smithi059.stdout:418/600: open fd = 3 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:419/600: open fd = 3 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:420/600: open fd = 3 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:421/600: open fd = 3 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.644 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:422/600: open fd = 3 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:423/600: open fd = 3 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:424/600: open fd = 3 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.645 INFO:tasks.workunit.client.0.smithi059.stdout:425/600: open fd = 3 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:426/600: open fd = 3 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:427/600: open fd = 3 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:428/600: open fd = 3 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.646 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:429/600: open fd = 3 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:430/600: open fd = 3 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:431/600: open fd = 3 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:432/600: open fd = 3 2024-09-05T12:09:58.647 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:433/600: open fd = 3 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:434/600: open fd = 3 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:435/600: open fd = 3 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.648 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.649 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:58.649 INFO:tasks.workunit.client.0.smithi059.stdout:436/600: open fd = 3 2024-09-05T12:09:58.649 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.649 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:09:58.649 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:09:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:09:58 smithi089 ceph-mon[34499]: pgmap v763: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:09:59.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:09:58 smithi059 ceph-mon[30025]: pgmap v763: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:09:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:09:58 smithi189 ceph-mon[34772]: pgmap v763: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:00 smithi089 ceph-mon[34499]: pgmap v764: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:00 smithi089 ceph-mon[34499]: overall HEALTH_OK 2024-09-05T12:10:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:00 smithi059 ceph-mon[30025]: pgmap v764: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:00 smithi059 ceph-mon[30025]: overall HEALTH_OK 2024-09-05T12:10:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:00 smithi189 ceph-mon[34772]: pgmap v764: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:01.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:00 smithi189 ceph-mon[34772]: overall HEALTH_OK 2024-09-05T12:10:03.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:02 smithi059 ceph-mon[30025]: pgmap v765: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:02 smithi189 ceph-mon[34772]: pgmap v765: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:03.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:02 smithi089 ceph-mon[34499]: pgmap v765: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:10:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:04 smithi059 ceph-mon[30025]: pgmap v766: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:05.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:04 smithi189 ceph-mon[34772]: pgmap v766: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:05.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:04 smithi089 ceph-mon[34499]: pgmap v766: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:07.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:06 smithi059 ceph-mon[30025]: pgmap v767: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:07.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:06 smithi189 ceph-mon[34772]: pgmap v767: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:07.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:06 smithi089 ceph-mon[34499]: pgmap v767: 129 pgs: 129 active+clean; 5.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.9 KiB/s wr, 5 op/s 2024-09-05T12:10:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:08 smithi059 ceph-mon[30025]: pgmap v768: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:10:09.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:08 smithi189 ceph-mon[34772]: pgmap v768: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:10:09.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:08 smithi089 ceph-mon[34499]: pgmap v768: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-05T12:10:10.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:09 smithi189 ceph-mon[34772]: pgmap v769: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:10:10.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:09 smithi089 ceph-mon[34499]: pgmap v769: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:10:10.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:09 smithi059 ceph-mon[30025]: pgmap v769: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-09-05T12:10:12.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:12 smithi059 ceph-mon[30025]: pgmap v770: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:10:12.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:12 smithi189 ceph-mon[34772]: pgmap v770: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:10:12.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:12 smithi089 ceph-mon[34499]: pgmap v770: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:10:14.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:14 smithi059 ceph-mon[30025]: pgmap v771: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:10:14.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:14 smithi189 ceph-mon[34772]: pgmap v771: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:10:14.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:14 smithi089 ceph-mon[34499]: pgmap v771: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:10:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:16 smithi189 ceph-mon[34772]: pgmap v772: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:16.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:16 smithi089 ceph-mon[34499]: pgmap v772: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:16.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:16 smithi059 ceph-mon[30025]: pgmap v772: 129 pgs: 129 active+clean; 6.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:18.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:18 smithi189 ceph-mon[34772]: pgmap v773: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:18.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:18 smithi089 ceph-mon[34499]: pgmap v773: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:18.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:18 smithi059 ceph-mon[30025]: pgmap v773: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:19.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:10:19.524 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:10:20.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 -- 172.21.15.59:0/3548317065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 msgr2=0x7f40ec1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 --2- 172.21.15.59:0/3548317065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec1043d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009900 tx=0x7f40dc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 -- 172.21.15.59:0/3548317065 shutdown_connections 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 --2- 172.21.15.59:0/3548317065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec109120 0x7f40ec110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 --2- 172.21.15.59:0/3548317065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec1049a0 0x7f40ec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 --2- 172.21.15.59:0/3548317065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec1043d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 -- 172.21.15.59:0/3548317065 >> 172.21.15.59:0/3548317065 conn(0x7f40ec0ffa90 msgr2=0x7f40ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 -- 172.21.15.59:0/3548317065 shutdown_connections 2024-09-05T12:10:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.120+0000 7f40f3600640 1 -- 172.21.15.59:0/3548317065 wait complete. 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 Processor -- start 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 -- start start 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 0x7f40ec19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109120 0x7f40ec19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec114120 con 0x7f40ec103ff0 2024-09-05T12:10:20.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec113fa0 con 0x7f40ec1049a0 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.121+0000 7f40f3600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1142a0 con 0x7f40ec109120 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56198/0 (socket says 172.21.15.59:56198) 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 -- 172.21.15.59:0/1799164199 learned_addr learned my addr 172.21.15.59:0/1799164199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40e3e00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 0x7f40ec19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f1600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109120 0x7f40ec19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 -- 172.21.15.59:0/1799164199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109120 msgr2=0x7f40ec19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109120 0x7f40ec19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 -- 172.21.15.59:0/1799164199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 msgr2=0x7f40ec19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 0x7f40ec19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40f0c00640 1 -- 172.21.15.59:0/1799164199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec19f5b0 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.122+0000 7f40e3e00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 0x7f40ec19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006010 tx=0x7f40dc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40ea000640 1 -- 172.21.15.59:0/1799164199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc03f030 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40ea000640 1 -- 172.21.15.59:0/1799164199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc004440 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40ea000640 1 -- 172.21.15.59:0/1799164199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc035070 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec19f840 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.123+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a7180 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.126+0000 7f40ea000640 1 -- 172.21.15.59:0/1799164199 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40dc042030 con 0x7f40ec103ff0 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.126+0000 7f40f6089740 1 Processor -- start 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.126+0000 7f40f6089740 1 -- start start 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652e3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652e6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b16523f3e0 con 0x55b1652d8b90 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b1651e6d80 con 0x55b1652d94b0 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f6089740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b16523b3a0 con 0x55b1652e3d70 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56206/0 (socket says 172.21.15.59:56206) 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f1600640 1 -- 172.21.15.59:0/1388570771 learned_addr learned my addr 172.21.15.59:0/1388570771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40e3e00640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f1600640 1 -- 172.21.15.59:0/1388570771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 msgr2=0x55b1652e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.127+0000 7f40f1600640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652e6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f1600640 1 -- 172.21.15.59:0/1388570771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 msgr2=0x55b1652e3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f1600640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652e3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f1600640 1 -- 172.21.15.59:0/1388570771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b165205ae0 con 0x55b1652d8b90 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40e3e00640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40ea000640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 0x7f40d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f1600640 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009f50 tx=0x7f40e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deba7b95cf31a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40ea000640 1 -- 172.21.15.59:0/1799164199 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f40dc0ef800 con 0x7f40ec103ff0 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 0x7f40d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.128+0000 7f40e1c00640 1 -- 172.21.15.59:0/1388570771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4004d20 con 0x55b1652d8b90 2024-09-05T12:10:20.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40e1c00640 1 -- 172.21.15.59:0/1388570771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e4004050 con 0x55b1652d8b90 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40e1c00640 1 -- 172.21.15.59:0/1388570771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4005280 con 0x55b1652d8b90 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 -- 172.21.15.59:0/1388570771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 msgr2=0x55b1652d8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f0c00640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 0x7f40d00a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0b8670 tx=0x7f40dc0f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009f50 tx=0x7f40e40118d0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 -- 172.21.15.59:0/1388570771 shutdown_connections 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 --2- 172.21.15.59:0/1388570771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652d8f70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.129+0000 7f40f6089740 1 -- 172.21.15.59:0/1388570771 >> 172.21.15.59:0/1388570771 conn(0x55b1652d5120 msgr2=0x55b1652d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.130+0000 7f40f6089740 1 -- 172.21.15.59:0/1388570771 shutdown_connections 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.130+0000 7f40f6089740 1 -- 172.21.15.59:0/1388570771 wait complete. 2024-09-05T12:10:20.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.130+0000 7f40f6089740 1 -- start start 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.130+0000 7f40f6089740 1 Processor -- start 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d8b90 0x55b1652e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b1652e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b1652e69e0 con 0x55b1652d94b0 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b1651fa0f0 con 0x55b1652e3d70 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b1652e6b60 con 0x55b1652d8b90 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f6089740 10 client.0 _ll_register_callbacks cb 0x55b1651eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56218/0 (socket says 172.21.15.59:56218) 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40e3e00640 1 -- 172.21.15.59:0/3173939756 learned_addr learned my addr 172.21.15.59:0/3173939756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:20.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f1600640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d8b90 0x55b1652e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.131+0000 7f40f1600640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b1652e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 -- 172.21.15.59:0/3173939756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d8b90 msgr2=0x55b1652e0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d8b90 0x55b1652e0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 -- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 msgr2=0x55b1652e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b1652e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 -- 172.21.15.59:0/3173939756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b164fb03c0 con 0x55b1652d94b0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40f1600640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b1652e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f40d4012d30 tx=0x7f40d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d91005c5f1b28df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40da000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d400fe20 con 0x55b1652d94b0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f40d40120d0 con 0x55b1652d94b0 2024-09-05T12:10:20.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.132+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e4004050 con 0x55b1652d94b0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f40d4010430 con 0x55b1652d94b0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 10 client.17400 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b1652f2740 con 0x55b1652d94b0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e4004050 con 0x55b1652d94b0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f40d401c590 con 0x55b1652d94b0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 10 client.17400 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 20 client.17400 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1652f3ca0 0x55b1652f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 4 client.17400 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.133+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b16524c950 con 0x55b1652f3ca0 2024-09-05T12:10:20.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.134+0000 7f40f1600640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1652f3ca0 0x55b1652f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.134+0000 7f40f1600640 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1652f3ca0 0x55b1652f6090 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5cdcec8de591a4 server_cookie=6c352b3afc7f674b in_seq=0 out_seq=0 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.134+0000 7f40da000640 10 client.17400 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40da000640 1 -- 172.21.15.59:0/3173939756 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b16524c950 con 0x55b1652f3ca0 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40da000640 10 client.17400 handle_command_reply: tid=0 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 client.17400 shutdown 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 msgr2=0x55b1652e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f40d4012d30 tx=0x7f40d400f790 comp rx=0 tx=0).stop 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 shutdown_connections 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b1652e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1652f3ca0 0x55b1652f6090 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d94b0 0x55b1652e11e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 --2- 172.21.15.59:0/3173939756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d8b90 0x55b1652e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 >> 172.21.15.59:0/3173939756 conn(0x55b1652d5120 msgr2=0x55b1652dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.135+0000 7f40da000640 0 client.17400 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.136+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 shutdown_connections 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.136+0000 7f40f6089740 1 -- 172.21.15.59:0/3173939756 wait complete. 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.136+0000 7f40f6089740 20 client.17400 trim_cache size 0 max 16384 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 Processor -- start 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 -- start start 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1af070 con 0x55b1652d8b90 2024-09-05T12:10:20.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b18f0 con 0x55b1652d94b0 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.154+0000 7f40f6089740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1af6e0 con 0x55b1652e3d70 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56230/0 (socket says 172.21.15.59:56230) 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 -- 172.21.15.59:0/4064630957 learned_addr learned my addr 172.21.15.59:0/4064630957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40e3e00640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 -- 172.21.15.59:0/4064630957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 msgr2=0x55b1652fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 -- 172.21.15.59:0/4064630957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 msgr2=0x55b1652fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 -- 172.21.15.59:0/4064630957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1652d8910 con 0x55b1652d8b90 2024-09-05T12:10:20.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40e3e00640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.155+0000 7f40f1600640 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f40e4012fd0 tx=0x7f40e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5be7b53622dd631 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40e1200640 1 -- 172.21.15.59:0/4064630957 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4030040 con 0x55b1652d8b90 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40e1200640 1 -- 172.21.15.59:0/4064630957 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e4004530 con 0x55b1652d8b90 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40e1200640 1 -- 172.21.15.59:0/4064630957 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4035b90 con 0x55b1652d8b90 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 -- 172.21.15.59:0/4064630957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 msgr2=0x55b1652e1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f40e4012fd0 tx=0x7f40e40118d0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 -- 172.21.15.59:0/4064630957 shutdown_connections 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652e3d70 0x55b1652fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652d94b0 0x55b1652fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.156+0000 7f40f6089740 1 --2- 172.21.15.59:0/4064630957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e1570 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.157+0000 7f40f6089740 1 -- 172.21.15.59:0/4064630957 >> 172.21.15.59:0/4064630957 conn(0x55b1652d5120 msgr2=0x55b1652dfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.157+0000 7f40f6089740 1 -- 172.21.15.59:0/4064630957 shutdown_connections 2024-09-05T12:10:20.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.157+0000 7f40f6089740 1 -- 172.21.15.59:0/4064630957 wait complete. 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.157+0000 7f40f6089740 1 -- start start 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.157+0000 7f40f6089740 1 Processor -- start 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b16535c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1afdb0 con 0x55b1652d8b90 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b1220 con 0x55b1652e3d70 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b0480 con 0x55b1652d94b0 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f6089740 10 client.0 _ll_register_callbacks cb 0x7f40d4012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:47608/0 (socket says 172.21.15.59:47608) 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56240/0 (socket says 172.21.15.59:56240) 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 -- 172.21.15.59:0/3866817864 learned_addr learned my addr 172.21.15.59:0/3866817864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f1600640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b16535c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 -- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 msgr2=0x55b16535c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b16535c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 -- 172.21.15.59:0/3866817864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 msgr2=0x55b1652e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40e3e00640 1 -- 172.21.15.59:0/3866817864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16535cea0 con 0x55b1652d94b0 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.158+0000 7f40f1600640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.159+0000 7f40e3e00640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001eb0 tx=0x7f40d401c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9f1fd59f4b196a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.159+0000 7f40da000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.159+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d401c7c0 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.159+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e4004530 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.159+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f40d4005b90 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f40d4038050 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 10 client.27328 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b16535e010 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e4004530 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f40d40105d0 con 0x55b1652d94b0 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 10 client.27328 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 10 client.27328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:20.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 20 client.27328 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b16535e810 0x55b165360c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 4 client.27328 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.160+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b165246e60 con 0x55b16535e810 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40f1600640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b16535e810 0x55b165360c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40f1600640 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b16535e810 0x55b165360c30 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2cfd26fe8ef289 server_cookie=cb54b22a1daa5f32 in_seq=0 out_seq=0 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40da000640 10 client.27328 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40da000640 1 -- 172.21.15.59:0/3866817864 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b165246e60 con 0x55b16535e810 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40da000640 10 client.27328 handle_command_reply: tid=0 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40f6089740 1 client.27328 shutdown 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 msgr2=0x55b1652e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.161+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001eb0 tx=0x7f40d401c650 comp rx=0 tx=0).stop 2024-09-05T12:10:20.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 shutdown_connections 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b1652e3d70 0x55b16535c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1652d94b0 0x55b1652e73f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b16535e810 0x55b165360c30 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 --2- 172.21.15.59:0/3866817864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b1652d8b90 0x55b1652e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40da000640 0 client.27328 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 >> 172.21.15.59:0/3866817864 conn(0x55b1652d5120 msgr2=0x55b1652e2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 shutdown_connections 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 1 -- 172.21.15.59:0/3866817864 wait complete. 2024-09-05T12:10:20.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f6089740 20 client.27328 trim_cache size 0 max 16384 2024-09-05T12:10:20.216 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:10:20.216 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:10:20.216 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:10:20.216 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 msgr2=0x7f40d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 0x7f40d00a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0b8670 tx=0x7f40dc0f5530 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 msgr2=0x7f40ec19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.162+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006010 tx=0x7f40dc004040 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 shutdown_connections 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f40d00a2b80 0x7f40d00a5040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109120 0x7f40ec19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40ec1049a0 0x7f40ec19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 --2- 172.21.15.59:0/1799164199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ec103ff0 0x7f40ec19a600 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 >> 172.21.15.59:0/1799164199 conn(0x7f40ec0ffa90 msgr2=0x7f40ec100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 shutdown_connections 2024-09-05T12:10:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:20.163+0000 7f40f3600640 1 -- 172.21.15.59:0/1799164199 wait complete. 2024-09-05T12:10:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:20 smithi059 ceph-mon[30025]: pgmap v774: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:20.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:20 smithi189 ceph-mon[34772]: pgmap v774: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:20.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:20 smithi089 ceph-mon[34499]: pgmap v774: 129 pgs: 129 active+clean; 6.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:21.206 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:10:21.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:10:21.586 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:10:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 -- 172.21.15.59:0/2329828756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 msgr2=0x7f2c74076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 --2- 172.21.15.59:0/2329828756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c74076b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009900 tx=0x7f2c6402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 -- 172.21.15.59:0/2329828756 shutdown_connections 2024-09-05T12:10:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 --2- 172.21.15.59:0/2329828756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 0x7f2c74114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 --2- 172.21.15.59:0/2329828756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 --2- 172.21.15.59:0/2329828756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c74076b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 -- 172.21.15.59:0/2329828756 >> 172.21.15.59:0/2329828756 conn(0x7f2c740ffa90 msgr2=0x7f2c74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:22.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 -- 172.21.15.59:0/2329828756 shutdown_connections 2024-09-05T12:10:22.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.125+0000 7f2c78a00640 1 -- 172.21.15.59:0/2329828756 wait complete. 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.126+0000 7f2c78a00640 1 Processor -- start 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 -- start start 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c741a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 0x7f2c7419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741185f0 con 0x7f2c74112a30 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74118470 con 0x7f2c74076720 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c78a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74118770 con 0x7f2c74077040 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c71e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c741a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c72800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 0x7f2c7419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c71400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c71e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c741a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43090/0 (socket says 172.21.15.59:43090) 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c71400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36380/0 (socket says 172.21.15.59:36380) 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.127+0000 7f2c71e00640 1 -- 172.21.15.59:0/1175524393 learned_addr learned my addr 172.21.15.59:0/1175524393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 -- 172.21.15.59:0/1175524393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 msgr2=0x7f2c741a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c741a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 -- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 msgr2=0x7f2c7419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 0x7f2c7419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 -- 172.21.15.59:0/1175524393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7419efb0 con 0x7f2c74077040 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c71400640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000e9f0 tx=0x7f2c6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000ccc0 con 0x7f2c74077040 2024-09-05T12:10:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7419f2a0 con 0x7f2c74077040 2024-09-05T12:10:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c60010910 con 0x7f2c74077040 2024-09-05T12:10:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.128+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60018790 con 0x7f2c74077040 2024-09-05T12:10:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.129+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c741ab590 con 0x7f2c74077040 2024-09-05T12:10:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.130+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c74076b00 con 0x7f2c74077040 2024-09-05T12:10:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.132+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c60010ac0 con 0x7f2c74077040 2024-09-05T12:10:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.133+0000 7f2c6aa00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 0x7f2c580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:22.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.133+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2c600cfd10 con 0x7f2c74077040 2024-09-05T12:10:22.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.133+0000 7f2c71e00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 0x7f2c580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:22.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.134+0000 7f2c71e00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 0x7f2c580a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c640096c0 tx=0x7f2c640046c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.135+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c6009c800 con 0x7f2c74077040 2024-09-05T12:10:22.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.360+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c7419fdb0 con 0x7f2c74077040 2024-09-05T12:10:22.363 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:22 smithi059 ceph-mon[30025]: pgmap v775: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:22.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.361+0000 7f2c6aa00640 1 -- 172.21.15.59:0/1175524393 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f2c7419fdb0 con 0x7f2c74077040 2024-09-05T12:10:22.364 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:10:22.364 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:10:22.364 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:10:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 msgr2=0x7f2c580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 0x7f2c580a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c640096c0 tx=0x7f2c640046c0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 msgr2=0x7f2c741a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000e9f0 tx=0x7f2c6000edf0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 shutdown_connections 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2c580a2b80 0x7f2c580a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74112a30 0x7f2c7419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c74077040 0x7f2c741a4c00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 --2- 172.21.15.59:0/1175524393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c74076720 0x7f2c741a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 >> 172.21.15.59:0/1175524393 conn(0x7f2c740ffa90 msgr2=0x7f2c74101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 shutdown_connections 2024-09-05T12:10:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:22.363+0000 7f2c78a00640 1 -- 172.21.15.59:0/1175524393 wait complete. 2024-09-05T12:10:22.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:22 smithi189 ceph-mon[34772]: pgmap v775: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:22.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:22 smithi089 ceph-mon[34499]: pgmap v775: 129 pgs: 129 active+clean; 6.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:23.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:10:23.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:23 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1175524393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:10:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:10:23.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:23 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1175524393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:10:23.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:23 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1175524393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:10:24.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 -- 172.21.15.59:0/2163342019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 msgr2=0x7f0bf4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 --2- 172.21.15.59:0/2163342019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf4108a10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009900 tx=0x7f0bdc02a410 comp rx=0 tx=0).stop 2024-09-05T12:10:24.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 -- 172.21.15.59:0/2163342019 shutdown_connections 2024-09-05T12:10:24.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 --2- 172.21.15.59:0/2163342019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 --2- 172.21.15.59:0/2163342019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf4108a10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 --2- 172.21.15.59:0/2163342019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 -- 172.21.15.59:0/2163342019 >> 172.21.15.59:0/2163342019 conn(0x7f0bf40ffa50 msgr2=0x7f0bf4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.178+0000 7f0bfa600640 1 -- 172.21.15.59:0/2163342019 shutdown_connections 2024-09-05T12:10:24.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.179+0000 7f0bfa600640 1 -- 172.21.15.59:0/2163342019 wait complete. 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 Processor -- start 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 -- start start 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4114160 con 0x7f0bf41049c0 2024-09-05T12:10:24.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4113fe0 con 0x7f0bf4109140 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.180+0000 7f0bfa600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41142e0 con 0x7f0bf4104010 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53914/0 (socket says 172.21.15.59:53914) 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 -- 172.21.15.59:0/1612581478 learned_addr learned my addr 172.21.15.59:0/1612581478 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 -- 172.21.15.59:0/1612581478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 msgr2=0x7f0bf419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 -- 172.21.15.59:0/1612581478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 msgr2=0x7f0bf419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bebe00640 1 -- 172.21.15.59:0/1612581478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf419f690 con 0x7f0bf41049c0 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.181+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bebe00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006010 tx=0x7f0bdc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bf0c00640 1 -- 172.21.15.59:0/1612581478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc03f030 con 0x7f0bf41049c0 2024-09-05T12:10:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bf0c00640 1 -- 172.21.15.59:0/1612581478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bdc004440 con 0x7f0bf41049c0 2024-09-05T12:10:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bf0c00640 1 -- 172.21.15.59:0/1612581478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc035070 con 0x7f0bf41049c0 2024-09-05T12:10:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf419f920 con 0x7f0bf41049c0 2024-09-05T12:10:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.182+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a7260 con 0x7f0bf41049c0 2024-09-05T12:10:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.185+0000 7f0bfd00d740 1 Processor -- start 2024-09-05T12:10:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.185+0000 7f0bf0c00640 1 -- 172.21.15.59:0/1612581478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0bdc042030 con 0x7f0bf41049c0 2024-09-05T12:10:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.185+0000 7f0bfd00d740 1 -- start start 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a88d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 0x561b7a897910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a897e50 0x561b7a89a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561b7a7f33e0 con 0x561b7a88cc70 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561b7a79ad80 con 0x561b7a897e50 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b7a7ef3a0 con 0x561b7a88d590 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a88d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 0x561b7a897910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 0x561b7a897910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36408/0 (socket says 172.21.15.59:36408) 2024-09-05T12:10:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bf3400640 1 -- 172.21.15.59:0/4239270610 learned_addr learned my addr 172.21.15.59:0/4239270610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.186+0000 7f0bebe00640 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a897e50 0x561b7a89a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 -- 172.21.15.59:0/4239270610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 msgr2=0x561b7a897910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 0x561b7a897910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 -- 172.21.15.59:0/4239270610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a897e50 msgr2=0x561b7a89a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a897e50 0x561b7a89a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 -- 172.21.15.59:0/4239270610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7a7b9ae0 con 0x561b7a88cc70 2024-09-05T12:10:24.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a88d050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009f10 tx=0x7f0bec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6f0014f124359e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf0c00640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0be8c00640 1 -- 172.21.15.59:0/4239270610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec01c680 con 0x561b7a88cc70 2024-09-05T12:10:24.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0be8c00640 1 -- 172.21.15.59:0/4239270610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec01cd00 con 0x561b7a88cc70 2024-09-05T12:10:24.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0be8c00640 1 -- 172.21.15.59:0/4239270610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec01a380 con 0x561b7a88cc70 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.187+0000 7f0bf0c00640 1 -- 172.21.15.59:0/1612581478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0bdc0ef8d0 con 0x7f0bf41049c0 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 -- 172.21.15.59:0/4239270610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 msgr2=0x561b7a88d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a88d050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009f10 tx=0x7f0bec0107a0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 -- 172.21.15.59:0/4239270610 shutdown_connections 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a897e50 0x561b7a89a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88d590 0x561b7a897910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/4239270610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a88d050 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.188+0000 7f0bfd00d740 1 -- 172.21.15.59:0/4239270610 >> 172.21.15.59:0/4239270610 conn(0x561b7a889200 msgr2=0x561b7a8895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.189+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0be4004650 tx=0x7f0be4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.189+0000 7f0bfd00d740 1 -- 172.21.15.59:0/4239270610 shutdown_connections 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.189+0000 7f0bfd00d740 1 -- 172.21.15.59:0/4239270610 wait complete. 2024-09-05T12:10:24.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.189+0000 7f0bfd00d740 1 -- start start 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.189+0000 7f0bfd00d740 1 Processor -- start 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88cc70 0x561b7a894e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a895390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561b7a89aa20 con 0x561b7a897e50 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561b7a7ae0f0 con 0x561b7a88d590 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b7a89aba0 con 0x561b7a88cc70 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bfd00d740 10 client.0 _ll_register_callbacks cb 0x561b7a795bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:24.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53926/0 (socket says 172.21.15.59:53926) 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bf3e00640 1 -- 172.21.15.59:0/1780965106 learned_addr learned my addr 172.21.15.59:0/1780965106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88cc70 0x561b7a894e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.190+0000 7f0bebe00640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a895390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 -- 172.21.15.59:0/1780965106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88cc70 msgr2=0x561b7a894e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88cc70 0x561b7a894e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 -- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 msgr2=0x561b7a895390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a895390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 -- 172.21.15.59:0/1780965106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7a5643c0 con 0x561b7a897e50 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011010 tx=0x7f0bec011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53e32dcb8059894 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0be1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec02a060 con 0x561b7a897e50 2024-09-05T12:10:24.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0bec004c50 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.191+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec01cd00 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0bec031050 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0bfd00d740 10 client.17424 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561b7a8a6b80 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec01cd00 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0bec005180 con 0x561b7a897e50 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0bfd00d740 10 client.17424 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0bfd00d740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.192+0000 7f0bfd00d740 20 client.17424 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a8a7f80 0x561b7a8aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0bfd00d740 4 client.17424 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b7a800950 con 0x561b7a8a7f80 2024-09-05T12:10:24.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a8a7f80 0x561b7a8aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0bf3400640 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a8a7f80 0x561b7a8aa370 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453af7b62fcf0a38 server_cookie=5bbee921b67d2e89 in_seq=0 out_seq=0 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.193+0000 7f0be1600640 10 client.17424 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.194+0000 7f0be1600640 1 -- 172.21.15.59:0/1780965106 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b7a800950 con 0x561b7a8a7f80 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.194+0000 7f0be1600640 10 client.17424 handle_command_reply: tid=0 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.194+0000 7f0bfd00d740 1 client.17424 shutdown 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.194+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 msgr2=0x561b7a8958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.194+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011010 tx=0x7f0bec011670 comp rx=0 tx=0).stop 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 shutdown_connections 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a8a7f80 0x561b7a8aa370 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a897e50 0x561b7a8958d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a895390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/1780965106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a88cc70 0x561b7a894e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 >> 172.21.15.59:0/1780965106 conn(0x561b7a889200 msgr2=0x561b7a888380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0be1600640 0 client.17424 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 shutdown_connections 2024-09-05T12:10:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 1 -- 172.21.15.59:0/1780965106 wait complete. 2024-09-05T12:10:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.195+0000 7f0bfd00d740 20 client.17424 trim_cache size 0 max 16384 2024-09-05T12:10:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 Processor -- start 2024-09-05T12:10:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 -- start start 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88cc70 0x561b7a896dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41af120 con 0x561b7a88d590 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b19a0 con 0x561b7a88cc70 2024-09-05T12:10:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.209+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41af790 con 0x561b7a897e50 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53934/0 (socket says 172.21.15.59:53934) 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 -- 172.21.15.59:0/2108047654 learned_addr learned my addr 172.21.15.59:0/2108047654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bf3400640 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88cc70 0x561b7a896dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 -- 172.21.15.59:0/2108047654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 msgr2=0x561b7a8ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 -- 172.21.15.59:0/2108047654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88cc70 msgr2=0x561b7a896dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88cc70 0x561b7a896dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 -- 172.21.15.59:0/2108047654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7a884770 con 0x561b7a88d590 2024-09-05T12:10:24.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0bebe00640 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0bba80 tx=0x7f0bdc03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e3f15f766b6521 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.210+0000 7f0be3e00640 1 -- 172.21.15.59:0/2108047654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc049040 con 0x561b7a88d590 2024-09-05T12:10:24.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0be3e00640 1 -- 172.21.15.59:0/2108047654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bdc003e20 con 0x561b7a88d590 2024-09-05T12:10:24.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0be3e00640 1 -- 172.21.15.59:0/2108047654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0c49f0 con 0x561b7a88d590 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 -- 172.21.15.59:0/2108047654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 msgr2=0x561b7a8acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0bba80 tx=0x7f0bdc03b460 comp rx=0 tx=0).stop 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 -- 172.21.15.59:0/2108047654 shutdown_connections 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88d590 0x561b7a8acfb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/2108047654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88cc70 0x561b7a896dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 -- 172.21.15.59:0/2108047654 >> 172.21.15.59:0/2108047654 conn(0x561b7a889200 msgr2=0x561b7a8966b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 -- 172.21.15.59:0/2108047654 shutdown_connections 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.211+0000 7f0bfd00d740 1 -- 172.21.15.59:0/2108047654 wait complete. 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 -- start start 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 Processor -- start 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a8aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41afe60 con 0x561b7a88cc70 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b12d0 con 0x561b7a88d590 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.212+0000 7f0bfd00d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b0530 con 0x561b7a897e50 2024-09-05T12:10:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bfd00d740 10 client.0 _ll_register_callbacks cb 0x7f0bdc0f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53948/0 (socket says 172.21.15.59:53948) 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 -- 172.21.15.59:0/578794081 learned_addr learned my addr 172.21.15.59:0/578794081 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3e00640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bebe00640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a8aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 -- 172.21.15.59:0/578794081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 msgr2=0x561b7a8aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 -- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 msgr2=0x561b7a8aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a8aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.213+0000 7f0bf3400640 1 -- 172.21.15.59:0/578794081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7a8ab430 con 0x561b7a88cc70 2024-09-05T12:10:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0bf3400640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0be4012fa0 tx=0x7f0be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a909949fc3d54f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0be1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:10:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400fe50 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0be402b380 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b7a7fae60 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0be403d050 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0bfd00d740 10 client.17430 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561b7a912770 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.214+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561b7a7fae60 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0be40349b0 con 0x561b7a88cc70 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 10 client.17430 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 20 client.17430 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a912ea0 0x561b7a9152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 4 client.17430 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:10:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b7a777010 con 0x561b7a912ea0 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.215+0000 7f0bebe00640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a912ea0 0x561b7a9152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bebe00640 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a912ea0 0x561b7a9152c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5118ef60b3b4ed2d server_cookie=7786a5317375d7a8 in_seq=0 out_seq=0 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0be1600640 10 client.17430 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0be1600640 1 -- 172.21.15.59:0/578794081 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b7a777010 con 0x561b7a912ea0 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0be1600640 10 client.17430 handle_command_reply: tid=0 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 client.17430 shutdown 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 msgr2=0x561b7a8ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0be4012fa0 tx=0x7f0be40046c0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 shutdown_connections 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b7a897e50 0x561b7a8aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b7a88d590 0x561b7a8aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x561b7a912ea0 0x561b7a9152c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 --2- 172.21.15.59:0/578794081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561b7a88cc70 0x561b7a8ad560 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 >> 172.21.15.59:0/578794081 conn(0x561b7a889200 msgr2=0x561b7a888380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0be1600640 0 client.17430 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:24.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 shutdown_connections 2024-09-05T12:10:24.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 1 -- 172.21.15.59:0/578794081 wait complete. 2024-09-05T12:10:24.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.216+0000 7f0bfd00d740 20 client.17430 trim_cache size 0 max 16384 2024-09-05T12:10:24.223 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 msgr2=0x7f0bd80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0be4004650 tx=0x7f0be4005f90 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 msgr2=0x7f0bf419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006010 tx=0x7f0bdc004040 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 shutdown_connections 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0bd80a2c00 0x7f0bd80a50c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bf4109140 0x7f0bf419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 --2- 172.21.15.59:0/1612581478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 >> 172.21.15.59:0/1612581478 conn(0x7f0bf40ffa50 msgr2=0x7f0bf4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:24.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 shutdown_connections 2024-09-05T12:10:24.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:24.217+0000 7f0bfa600640 1 -- 172.21.15.59:0/1612581478 wait complete. 2024-09-05T12:10:24.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:24 smithi059 ceph-mon[30025]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:24 smithi189 ceph-mon[34772]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:24.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:24 smithi089 ceph-mon[34499]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:26.122 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:10:26.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:10:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:10:26.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:26 smithi059 ceph-mon[30025]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:27.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 -- 172.21.15.59:0/1401408877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98112a30 msgr2=0x7fce98114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 --2- 172.21.15.59:0/1401408877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98112a30 0x7fce98114e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fce90009ca0 tx=0x7fce9002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 -- 172.21.15.59:0/1401408877 shutdown_connections 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 --2- 172.21.15.59:0/1401408877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98112a30 0x7fce98114e20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 --2- 172.21.15.59:0/1401408877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98077040 0x7fce981124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 --2- 172.21.15.59:0/1401408877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98076720 0x7fce98076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 -- 172.21.15.59:0/1401408877 >> 172.21.15.59:0/1401408877 conn(0x7fce980ffa90 msgr2=0x7fce98101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 -- 172.21.15.59:0/1401408877 shutdown_connections 2024-09-05T12:10:27.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.084+0000 7fce9e000640 1 -- 172.21.15.59:0/1401408877 wait complete. 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.085+0000 7fce9e000640 1 Processor -- start 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.085+0000 7fce9e000640 1 -- start start 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98077040 0x7fce981a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98112a30 0x7fce9819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce981185d0 con 0x7fce98076720 2024-09-05T12:10:27.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fce98118450 con 0x7fce98112a30 2024-09-05T12:10:27.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce9e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce98118750 con 0x7fce98077040 2024-09-05T12:10:27.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:26 smithi189 ceph-mon[34772]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53962/0 (socket says 172.21.15.59:53962) 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce97400640 1 -- 172.21.15.59:0/1071449942 learned_addr learned my addr 172.21.15.59:0/1071449942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce97e00640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98112a30 0x7fce9819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.086+0000 7fce8ea00640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98077040 0x7fce981a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 -- 172.21.15.59:0/1071449942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98077040 msgr2=0x7fce981a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98077040 0x7fce981a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 -- 172.21.15.59:0/1071449942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98112a30 msgr2=0x7fce9819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98112a30 0x7fce9819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 -- 172.21.15.59:0/1071449942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9819efd0 con 0x7fce98076720 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce97400640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fce8800e9f0 tx=0x7fce8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce94c00640 1 -- 172.21.15.59:0/1071449942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8800cd80 con 0x7fce98076720 2024-09-05T12:10:27.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce94c00640 1 -- 172.21.15.59:0/1071449942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce88005500 con 0x7fce98076720 2024-09-05T12:10:27.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9819f2c0 con 0x7fce98076720 2024-09-05T12:10:27.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce94c00640 1 -- 172.21.15.59:0/1071449942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88010690 con 0x7fce98076720 2024-09-05T12:10:27.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.087+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce981ab5b0 con 0x7fce98076720 2024-09-05T12:10:27.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.090+0000 7fcea0a8a740 1 Processor -- start 2024-09-05T12:10:27.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fce94c00640 1 -- 172.21.15.59:0/1071449942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fce880046c0 con 0x7fce98076720 2024-09-05T12:10:27.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 -- start start 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564415214290 con 0x5644152adac0 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5644151bbb40 con 0x5644152ae3e0 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564415210250 con 0x5644152b8ca0 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fce97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fce97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53976/0 (socket says 172.21.15.59:53976) 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.091+0000 7fce97e00640 1 -- 172.21.15.59:0/3174424137 learned_addr learned my addr 172.21.15.59:0/3174424137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97400640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce8ea00640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 -- 172.21.15.59:0/3174424137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 msgr2=0x5644152bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152bb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 -- 172.21.15.59:0/3174424137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 msgr2=0x5644152b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 -- 172.21.15.59:0/3174424137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564414f856a0 con 0x5644152adac0 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce8ea00640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce97e00640 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce9807aa10 tx=0x7fce90005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9e843ecf7f15e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.092+0000 7fce94c00640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 0x7fce7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce87e00640 1 -- 172.21.15.59:0/3174424137 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90031db0 con 0x5644152adac0 2024-09-05T12:10:27.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce87e00640 1 -- 172.21.15.59:0/3174424137 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce90036030 con 0x5644152adac0 2024-09-05T12:10:27.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce94c00640 1 -- 172.21.15.59:0/1071449942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fce880cf830 con 0x7fce98076720 2024-09-05T12:10:27.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce97400640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 0x7fce7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce87e00640 1 -- 172.21.15.59:0/3174424137 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90004240 con 0x5644152adac0 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3174424137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 msgr2=0x5644152adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce9807aa10 tx=0x7fce90005f80 comp rx=0 tx=0).stop 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.093+0000 7fce97400640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 0x7fce7c0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fce880a4360 tx=0x7fce880d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3174424137 shutdown_connections 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3174424137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152adac0 0x5644152adea0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3174424137 >> 172.21.15.59:0/3174424137 conn(0x5644152aa050 msgr2=0x5644152aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3174424137 shutdown_connections 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3174424137 wait complete. 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 -- start start 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.094+0000 7fcea0a8a740 1 Processor -- start 2024-09-05T12:10:27.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152adac0 0x5644152b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644152bb730 con 0x5644152ae3e0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5644151cefa0 con 0x5644152adac0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644152bb8b0 con 0x5644152b8ca0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fcea0a8a740 10 client.0 _ll_register_callbacks cb 0x564414f22ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fce8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fce8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53988/0 (socket says 172.21.15.59:53988) 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fce8ea00640 1 -- 172.21.15.59:0/3719205771 learned_addr learned my addr 172.21.15.59:0/3719205771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fce97e00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152adac0 0x5644152b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.095+0000 7fce97e00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 -- 172.21.15.59:0/3719205771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 msgr2=0x5644152b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 -- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152adac0 msgr2=0x5644152b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152adac0 0x5644152b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 -- 172.21.15.59:0/3719205771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644152b6540 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce97e00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce8ea00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fce80012700 tx=0x7fce80012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc514db0aa452896 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce84c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004050 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fce800041f0 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.096+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce90036030 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce800043f0 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 10 client.17442 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5644152c76e0 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce90036030 con 0x5644152ae3e0 2024-09-05T12:10:27.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce80003950 con 0x5644152ae3e0 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 10 client.17442 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 20 client.17442 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152c8bf0 0x5644152cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 4 client.17442 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:10:27.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.097+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564415221800 con 0x5644152c8bf0 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.098+0000 7fce97e00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152c8bf0 0x5644152cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.098+0000 7fce97e00640 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152c8bf0 0x5644152cb070 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db3f99eeec3478d server_cookie=fa9d78dc066156cc in_seq=0 out_seq=0 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.098+0000 7fce84c00640 10 client.17442 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fce84c00640 1 -- 172.21.15.59:0/3719205771 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564415221800 con 0x5644152c8bf0 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fce84c00640 10 client.17442 handle_command_reply: tid=0 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 client.17442 shutdown 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 msgr2=0x5644152b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fce80012700 tx=0x7fce80012b00 comp rx=0 tx=0).stop 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 shutdown_connections 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152b8ca0 0x5644152b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152c8bf0 0x5644152cb070 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152b5800 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/3719205771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152adac0 0x5644152b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 >> 172.21.15.59:0/3719205771 conn(0x5644152aa050 msgr2=0x5644152afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.099+0000 7fce84c00640 0 client.17442 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.100+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 shutdown_connections 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.100+0000 7fcea0a8a740 1 -- 172.21.15.59:0/3719205771 wait complete. 2024-09-05T12:10:27.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.100+0000 7fcea0a8a740 20 client.17442 trim_cache size 0 max 16384 2024-09-05T12:10:27.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 Processor -- start 2024-09-05T12:10:27.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 -- start start 2024-09-05T12:10:27.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b3400 con 0x5644152aa050 2024-09-05T12:10:27.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b5c80 con 0x5644152ae3e0 2024-09-05T12:10:27.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b3a70 con 0x5644152adac0 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fce97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fce97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54000/0 (socket says 172.21.15.59:54000) 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.118+0000 7fce97e00640 1 -- 172.21.15.59:0/1276514588 learned_addr learned my addr 172.21.15.59:0/1276514588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce8ea00640 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 -- 172.21.15.59:0/1276514588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 msgr2=0x5644152d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 -- 172.21.15.59:0/1276514588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 msgr2=0x5644152d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 -- 172.21.15.59:0/1276514588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644152a85d0 con 0x5644152aa050 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce97e00640 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce9002f540 tx=0x7fce90005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6ff17c94f13332 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce87400640 1 -- 172.21.15.59:0/1276514588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90031590 con 0x5644152aa050 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce87400640 1 -- 172.21.15.59:0/1276514588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce90036070 con 0x5644152aa050 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.119+0000 7fce87400640 1 -- 172.21.15.59:0/1276514588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90004740 con 0x5644152aa050 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 -- 172.21.15.59:0/1276514588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 msgr2=0x5644152b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce9002f540 tx=0x7fce90005d80 comp rx=0 tx=0).stop 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 -- 172.21.15.59:0/1276514588 shutdown_connections 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152ae3e0 0x5644152d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/1276514588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152aa050 0x5644152b0d60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 -- 172.21.15.59:0/1276514588 >> 172.21.15.59:0/1276514588 conn(0x5644152b5ea0 msgr2=0x5644152b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 -- 172.21.15.59:0/1276514588 shutdown_connections 2024-09-05T12:10:27.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.120+0000 7fcea0a8a740 1 -- 172.21.15.59:0/1276514588 wait complete. 2024-09-05T12:10:27.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 -- start start 2024-09-05T12:10:27.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 Processor -- start 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 0x5644152bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b4140 con 0x5644152ae3e0 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b55b0 con 0x5644152aa050 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce981b4810 con 0x5644152adac0 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.121+0000 7fcea0a8a740 10 client.0 _ll_register_callbacks cb 0x7fce80009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 0x5644152bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 0x5644152bcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43190/0 (socket says 172.21.15.59:43190) 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce97e00640 1 -- 172.21.15.59:0/4287122276 learned_addr learned my addr 172.21.15.59:0/4287122276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce97e00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 -- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 msgr2=0x5644152bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 0x5644152bcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 -- 172.21.15.59:0/4287122276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 msgr2=0x5644152bd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152bd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 -- 172.21.15.59:0/4287122276 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564415332a80 con 0x5644152adac0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce8ea00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152bd020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce80003d70 tx=0x7fce800038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9ef6fc885959c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.122+0000 7fce84c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80005c50 con 0x5644152adac0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fce80005040 con 0x5644152adac0 2024-09-05T12:10:27.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce90036070 con 0x5644152adac0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fce8003b050 con 0x5644152adac0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fcea0a8a740 10 client.27370 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564415333cf0 con 0x5644152adac0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.123+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce90036070 con 0x5644152adac0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fce8000fc10 con 0x5644152adac0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 10 client.27370 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 10 client.27370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 20 client.27370 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152b8ca0 0x564415336420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 4 client.27370 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56441521bd10 con 0x5644152b8ca0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.124+0000 7fce97e00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152b8ca0 0x564415336420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:27.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.125+0000 7fce97e00640 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152b8ca0 0x564415336420 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183c856879c36438 server_cookie=b3433359b4b6ab16 in_seq=0 out_seq=0 2024-09-05T12:10:27.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.125+0000 7fce84c00640 10 client.27370 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:27.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fce84c00640 1 -- 172.21.15.59:0/4287122276 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56441521bd10 con 0x5644152b8ca0 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fce84c00640 10 client.27370 handle_command_reply: tid=0 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 client.27370 shutdown 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 msgr2=0x5644152bd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152bd020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce80003d70 tx=0x7fce800038a0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 shutdown_connections 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5644152b8ca0 0x564415336420 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644152ae3e0 0x5644152bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644152adac0 0x5644152bd020 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 --2- 172.21.15.59:0/4287122276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644152aa050 0x5644152bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 >> 172.21.15.59:0/4287122276 conn(0x5644152b5ea0 msgr2=0x5644152a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fce84c00640 0 client.27370 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 shutdown_connections 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.126+0000 7fcea0a8a740 1 -- 172.21.15.59:0/4287122276 wait complete. 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fcea0a8a740 20 client.27370 trim_cache size 0 max 16384 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "dd7a865d-b8db-453e-9f7d-ad464bd817ac", 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:10:27.131 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 msgr2=0x7fce7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 0x7fce7c0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fce880a4360 tx=0x7fce880d5530 comp rx=0 tx=0).stop 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 msgr2=0x7fce981a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fce8800e9f0 tx=0x7fce8800edf0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 shutdown_connections 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fce7c0a2c00 0x7fce7c0a50c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce98112a30 0x7fce9819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce98077040 0x7fce981a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 --2- 172.21.15.59:0/1071449942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce98076720 0x7fce981a46e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 >> 172.21.15.59:0/1071449942 conn(0x7fce980ffa90 msgr2=0x7fce98101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.127+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 shutdown_connections 2024-09-05T12:10:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:27.128+0000 7fce9e000640 1 -- 172.21.15.59:0/1071449942 wait complete. 2024-09-05T12:10:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:26 smithi089 ceph-mon[34499]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:27.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:10:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:10:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.888+0000 7f69d5400640 1 -- 172.21.15.59:0/2949800206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0101de0 msgr2=0x7f69d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.888+0000 7f69d5400640 1 --2- 172.21.15.59:0/2949800206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0101de0 0x7f69d01021c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009900 tx=0x7f69c002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 -- 172.21.15.59:0/2949800206 shutdown_connections 2024-09-05T12:10:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 --2- 172.21.15.59:0/2949800206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0106f10 0x7f69d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 --2- 172.21.15.59:0/2949800206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0102790 0x7f69d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 --2- 172.21.15.59:0/2949800206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0101de0 0x7f69d01021c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 -- 172.21.15.59:0/2949800206 >> 172.21.15.59:0/2949800206 conn(0x7f69d00fd8c0 msgr2=0x7f69d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 -- 172.21.15.59:0/2949800206 shutdown_connections 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.889+0000 7f69d5400640 1 -- 172.21.15.59:0/2949800206 wait complete. 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 Processor -- start 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 -- start start 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 0x7f69d019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 0x7f69d019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111fc0 con 0x7f69d0106f10 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111e40 con 0x7f69d0102790 2024-09-05T12:10:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69d5400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0112140 con 0x7f69d0101de0 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.890+0000 7f69ce800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 0x7f69d019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69ce800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36500/0 (socket says 172.21.15.59:36500) 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69ce800640 1 -- 172.21.15.59:0/3338037299 learned_addr learned my addr 172.21.15.59:0/3338037299 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69c5e00640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 0x7f69d019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 -- 172.21.15.59:0/3338037299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 msgr2=0x7f69d019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 -- 172.21.15.59:0/3338037299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 msgr2=0x7f69d019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 0x7f69d019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 -- 172.21.15.59:0/3338037299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d019f5f0 con 0x7f69d0106f10 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69ce800640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:28.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69c5e00640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 0x7f69d019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.891+0000 7f69cf200640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 0x7f69d019ef10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e940 tx=0x7f69c800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.892+0000 7f69c7e00640 1 -- 172.21.15.59:0/3338037299 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004d20 con 0x7f69d0106f10 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.892+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d019f8e0 con 0x7f69d0106f10 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.892+0000 7f69c7e00640 1 -- 172.21.15.59:0/3338037299 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c8005020 con 0x7f69d0106f10 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.892+0000 7f69c7e00640 1 -- 172.21.15.59:0/3338037299 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c80116b0 con 0x7f69d0106f10 2024-09-05T12:10:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.892+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01a71c0 con 0x7f69d0106f10 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.895+0000 7f69d7e95740 1 Processor -- start 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.895+0000 7f69c7e00640 1 -- 172.21.15.59:0/3338037299 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69c80054d0 con 0x7f69d0106f10 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.895+0000 7f69d7e95740 1 -- start start 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.895+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190561f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x55819056f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69d7e95740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5581904c83e0 con 0x558190561b90 2024-09-05T12:10:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69d7e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55819046fd80 con 0x55819056cd70 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69d7e95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581904c43a0 con 0x5581905624b0 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69ce800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69ce800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36508/0 (socket says 172.21.15.59:36508) 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69ce800640 1 -- 172.21.15.59:0/2734631055 learned_addr learned my addr 172.21.15.59:0/2734631055 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190561f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.896+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x55819056f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 -- 172.21.15.59:0/2734631055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 msgr2=0x55819056f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x55819056f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 -- 172.21.15.59:0/2734631055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 msgr2=0x558190561f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190561f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 -- 172.21.15.59:0/2734631055 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819048eae0 con 0x5581905624b0 2024-09-05T12:10:28.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69c7e00640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 0x7f69b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69ce800640 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69c0004470 tx=0x7f69c00044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3ea4a6ce5dddd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69c7e00640 1 -- 172.21.15.59:0/3338037299 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69c80cf460 con 0x7f69d0106f10 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.897+0000 7f69c5e00640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 0x7f69b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69bf400640 1 -- 172.21.15.59:0/2734631055 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0040070 con 0x5581905624b0 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69bf400640 1 -- 172.21.15.59:0/2734631055 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c00047a0 con 0x5581905624b0 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69bf400640 1 -- 172.21.15.59:0/2734631055 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0035040 con 0x5581905624b0 2024-09-05T12:10:28.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 -- 172.21.15.59:0/2734631055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 msgr2=0x55819056c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69c0004470 tx=0x7f69c00044a0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69c5e00640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 0x7f69b40a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55819055f4a0 tx=0x7f69b800bb20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 -- 172.21.15.59:0/2734631055 shutdown_connections 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x55819056f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056c830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2734631055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190561f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.898+0000 7f69d7e95740 1 -- 172.21.15.59:0/2734631055 >> 172.21.15.59:0/2734631055 conn(0x55819055e120 msgr2=0x55819055e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.899+0000 7f69d7e95740 1 -- 172.21.15.59:0/2734631055 shutdown_connections 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.899+0000 7f69d7e95740 1 -- 172.21.15.59:0/2734631055 wait complete. 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.899+0000 7f69d7e95740 1 -- start start 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.899+0000 7f69d7e95740 1 Processor -- start 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 0x55819056a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55819056f980 con 0x55819056cd70 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5581904830f0 con 0x558190561b90 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55819056fb00 con 0x5581905624b0 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69d7e95740 10 client.0 _ll_register_callbacks cb 0x5581904749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 0x55819056a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69ce800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43230/0 (socket says 172.21.15.59:43230) 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69ce800640 1 -- 172.21.15.59:0/118400365 learned_addr learned my addr 172.21.15.59:0/118400365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.900+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36524/0 (socket says 172.21.15.59:36524) 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 -- 172.21.15.59:0/118400365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 msgr2=0x55819056a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 -- 172.21.15.59:0/118400365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 msgr2=0x55819056a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 0x55819056a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 -- 172.21.15.59:0/118400365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581902393c0 con 0x558190561b90 2024-09-05T12:10:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69cf200640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 0x55819056a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69ce800640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69c00057c0 tx=0x7f69c0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f765ecdeabb0655e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.901+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0040070 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c00047a0 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69c0003ad0 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69c0056050 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 10 client.27458 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55819057b680 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c00047a0 con 0x558190561b90 2024-09-05T12:10:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f69c00316d0 con 0x558190561b90 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 10 client.27458 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 10 client.27458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 20 client.27458 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.902+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55819057c9d0 0x55819057edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.903+0000 7f69d7e95740 4 client.27458 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.903+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581904d5950 con 0x55819057c9d0 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.903+0000 7f69c5e00640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55819057c9d0 0x55819057edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.903+0000 7f69c5e00640 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55819057c9d0 0x55819057edc0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970c324ae7455129 server_cookie=9a1f555695bc5823 in_seq=0 out_seq=0 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.903+0000 7f69b3e00640 10 client.27458 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69b3e00640 1 -- 172.21.15.59:0/118400365 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581904d5950 con 0x55819057c9d0 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69b3e00640 10 client.27458 handle_command_reply: tid=0 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 client.27458 shutdown 2024-09-05T12:10:28.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 msgr2=0x558190569ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69c00057c0 tx=0x7f69c0009260 comp rx=0 tx=0).stop 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 shutdown_connections 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55819057c9d0 0x55819057edc0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55819056cd70 0x55819056a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581905624b0 0x55819056a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.904+0000 7f69d7e95740 1 --2- 172.21.15.59:0/118400365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558190561b90 0x558190569ca0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.905+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 >> 172.21.15.59:0/118400365 conn(0x55819055e120 msgr2=0x558190563e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.905+0000 7f69b3e00640 0 client.27458 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.905+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 shutdown_connections 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.905+0000 7f69d7e95740 1 -- 172.21.15.59:0/118400365 wait complete. 2024-09-05T12:10:28.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.905+0000 7f69d7e95740 20 client.27458 trim_cache size 0 max 16384 2024-09-05T12:10:28.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 Processor -- start 2024-09-05T12:10:28.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 -- start start 2024-09-05T12:10:28.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558190561b90 0x5581905645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x558190582320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01af0c0 con 0x5581905624b0 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b1940 con 0x55819056cd70 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.923+0000 7f69d7e95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01af730 con 0x558190561b90 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54076/0 (socket says 172.21.15.59:54076) 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x558190582320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69c5e00640 1 -- 172.21.15.59:0/2980237678 learned_addr learned my addr 172.21.15.59:0/2980237678 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69ce800640 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558190561b90 0x5581905645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 -- 172.21.15.59:0/2980237678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558190561b90 msgr2=0x5581905645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558190561b90 0x5581905645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 -- 172.21.15.59:0/2980237678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 msgr2=0x558190581de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 -- 172.21.15.59:0/2980237678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819055df50 con 0x55819056cd70 2024-09-05T12:10:28.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:28.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.924+0000 7f69cf200640 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x558190582320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69c80ccec0 tx=0x7f69c800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af55aef4874e7137 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69bea00640 1 -- 172.21.15.59:0/2980237678 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c80b4070 con 0x55819056cd70 2024-09-05T12:10:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69bea00640 1 -- 172.21.15.59:0/2980237678 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c80a4520 con 0x55819056cd70 2024-09-05T12:10:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69bea00640 1 -- 172.21.15.59:0/2980237678 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c801f040 con 0x55819056cd70 2024-09-05T12:10:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 -- 172.21.15.59:0/2980237678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 msgr2=0x558190582320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x558190582320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69c80ccec0 tx=0x7f69c800ef90 comp rx=0 tx=0).stop 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 -- 172.21.15.59:0/2980237678 shutdown_connections 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55819056cd70 0x558190582320 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5581905624b0 0x558190581de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2980237678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558190561b90 0x5581905645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 -- 172.21.15.59:0/2980237678 >> 172.21.15.59:0/2980237678 conn(0x55819055e120 msgr2=0x558190568570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 -- 172.21.15.59:0/2980237678 shutdown_connections 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.925+0000 7f69d7e95740 1 -- 172.21.15.59:0/2980237678 wait complete. 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 -- start start 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 Processor -- start 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 0x55819057fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55819056cd70 0x5581905801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01afe00 con 0x558190561b90 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b1270 con 0x5581905624b0 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.926+0000 7f69d7e95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b04d0 con 0x55819056cd70 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69d7e95740 10 client.0 _ll_register_callbacks cb 0x7f69c0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 0x55819057fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54086/0 (socket says 172.21.15.59:54086) 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 -- 172.21.15.59:0/2988775507 learned_addr learned my addr 172.21.15.59:0/2988775507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69cf200640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55819056cd70 0x5581905801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 -- 172.21.15.59:0/2988775507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55819056cd70 msgr2=0x5581905801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55819056cd70 0x5581905801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 -- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 msgr2=0x55819057fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:10:28.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 0x55819057fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69ce800640 1 -- 172.21.15.59:0/2988775507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558190563110 con 0x558190561b90 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.927+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 0x55819057fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69ce800640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f69c0031460 tx=0x7f69c00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ef54b4c7b5a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c005e020 con 0x558190561b90 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69c0005320 con 0x558190561b90 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c80a4520 con 0x558190561b90 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69c0066050 con 0x558190561b90 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69d7e95740 10 client.17466 fetch_fsmap learned FSMap version 21 2024-09-05T12:10:28.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.928+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5581905e7310 con 0x558190561b90 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c80a4520 con 0x558190561b90 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f69c0031ce0 con 0x558190561b90 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 10 client.17466 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 20 client.17466 populate_metadata read hostname 'smithi059' 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581905e7a50 0x5581905e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 4 client.17466 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:10:28.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581904cfe60 con 0x5581905e7a50 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.929+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581905e7a50 0x5581905e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69c5e00640 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581905e7a50 0x5581905e9e70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3d764f9352f6f9 server_cookie=3a432ffeb8fc1bc6 in_seq=0 out_seq=0 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69b3e00640 10 client.17466 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69b3e00640 1 -- 172.21.15.59:0/2988775507 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5581904cfe60 con 0x5581905e7a50 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69b3e00640 10 client.17466 handle_command_reply: tid=0 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 client.17466 shutdown 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 msgr2=0x558190582700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f69c0031460 tx=0x7f69c00317e0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 shutdown_connections 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55819056cd70 0x5581905801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581905624b0 0x55819057fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5581905e7a50 0x5581905e9e70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 --2- 172.21.15.59:0/2988775507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558190561b90 0x558190582700 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 >> 172.21.15.59:0/2988775507 conn(0x55819055e120 msgr2=0x55819055c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.930+0000 7f69b3e00640 0 client.17466 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 shutdown_connections 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d7e95740 1 -- 172.21.15.59:0/2988775507 wait complete. 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d7e95740 20 client.17466 trim_cache size 0 max 16384 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stdout: "dd7a865d-b8db-453e-9f7d-ad464bd817ac": { 2024-09-05T12:10:28.936 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "dd7a865d-b8db-453e-9f7d-ad464bd817ac", 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 msgr2=0x7f69b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 0x7f69b40a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55819055f4a0 tx=0x7f69b800bb20 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 msgr2=0x7f69d019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 0x7f69d019ef10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e940 tx=0x7f69c800ee00 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 shutdown_connections 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69b40a2b80 0x7f69b40a5040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69d0106f10 0x7f69d019ef10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69d0102790 0x7f69d019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 --2- 172.21.15.59:0/3338037299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69d0101de0 0x7f69d019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.931+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 >> 172.21.15.59:0/3338037299 conn(0x7f69d00fd8c0 msgr2=0x7f69d00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.932+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 shutdown_connections 2024-09-05T12:10:28.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:10:28.932+0000 7f69d5400640 1 -- 172.21.15.59:0/3338037299 wait complete. 2024-09-05T12:10:29.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:28 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:10:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:28 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:28 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:10:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:28 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:28 smithi189 ceph-mon[34772]: pgmap v778: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:28 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:10:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:28 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:28 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:10:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:28 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:28 smithi089 ceph-mon[34499]: pgmap v778: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:28 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:10:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:28 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:28 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:10:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:28 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:10:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:28 smithi059 ceph-mon[30025]: pgmap v778: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:29.658 INFO:tasks.cephfs.filesystem:scrub status for tag:dd7a865d-b8db-453e-9f7d-ad464bd817ac - {'path': '/', 'tag': 'dd7a865d-b8db-453e-9f7d-ad464bd817ac', 'options': 'recursive,force'} 2024-09-05T12:10:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:30 smithi189 ceph-mon[34772]: pgmap v779: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:10:31.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:30 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:10:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:30 smithi089 ceph-mon[34499]: pgmap v779: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:10:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:30 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:10:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:30 smithi059 ceph-mon[30025]: pgmap v779: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:10:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:30 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:10:33.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:32 smithi189 ceph-mon[34772]: pgmap v780: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:32 smithi089 ceph-mon[34499]: pgmap v780: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:32 smithi059 ceph-mon[30025]: pgmap v780: 129 pgs: 129 active+clean; 6.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:35.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:34 smithi189 ceph-mon[34772]: pgmap v781: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:10:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:34 smithi089 ceph-mon[34499]: pgmap v781: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:10:35.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:34 smithi059 ceph-mon[30025]: pgmap v781: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:10:37.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:36 smithi189 ceph-mon[34772]: pgmap v782: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:36 smithi089 ceph-mon[34499]: pgmap v782: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:36 smithi059 ceph-mon[30025]: pgmap v782: 129 pgs: 129 active+clean; 6.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:39.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:38 smithi189 ceph-mon[34772]: pgmap v783: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:38 smithi089 ceph-mon[34499]: pgmap v783: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:39.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:38 smithi059 ceph-mon[30025]: pgmap v783: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:41.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:40 smithi189 ceph-mon[34772]: pgmap v784: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:40 smithi089 ceph-mon[34499]: pgmap v784: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:41.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:40 smithi059 ceph-mon[30025]: pgmap v784: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:42 smithi189 ceph-mon[34772]: pgmap v785: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:42 smithi089 ceph-mon[34499]: pgmap v785: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:43.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:42 smithi059 ceph-mon[30025]: pgmap v785: 129 pgs: 129 active+clean; 6.3 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:45.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:44 smithi189 ceph-mon[34772]: pgmap v786: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:44 smithi089 ceph-mon[34499]: pgmap v786: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:45.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:44 smithi059 ceph-mon[30025]: pgmap v786: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:47.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:46 smithi189 ceph-mon[34772]: pgmap v787: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:46 smithi089 ceph-mon[34499]: pgmap v787: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:47.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:46 smithi059 ceph-mon[30025]: pgmap v787: 129 pgs: 129 active+clean; 6.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:10:49.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:48 smithi189 ceph-mon[34772]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:48 smithi089 ceph-mon[34499]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:49.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:48 smithi059 ceph-mon[30025]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:51.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:50 smithi189 ceph-mon[34772]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:50 smithi089 ceph-mon[34499]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:50 smithi059 ceph-mon[30025]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:52.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:51 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:10:52.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:51 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:10:52.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:51 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:10:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:52 smithi189 ceph-mon[34772]: pgmap v790: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:53.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:52 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:10:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:52 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:10:53.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:52 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:10:53.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:52 smithi059 ceph-mon[30025]: pgmap v790: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:53.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:52 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:10:53.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:52 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:10:53.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:52 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:10:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:52 smithi089 ceph-mon[34499]: pgmap v790: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:10:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:52 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:10:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:52 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:10:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:52 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:10:55.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:54 smithi189 ceph-mon[34772]: pgmap v791: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:54 smithi089 ceph-mon[34499]: pgmap v791: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:55.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:54 smithi059 ceph-mon[30025]: pgmap v791: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:10:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:56 smithi189 ceph-mon[34772]: pgmap v792: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:56 smithi089 ceph-mon[34499]: pgmap v792: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:57.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:56 smithi059 ceph-mon[30025]: pgmap v792: 129 pgs: 129 active+clean; 6.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:10:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:10:58 smithi189 ceph-mon[34772]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:10:58 smithi089 ceph-mon[34499]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:59.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:10:58 smithi059 ceph-mon[30025]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:10:59.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:11:00.051 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 -- 172.21.15.59:0/3846252704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 msgr2=0x7f4db4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 --2- 172.21.15.59:0/3846252704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db4104210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009ca0 tx=0x7f4da802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 -- 172.21.15.59:0/3846252704 shutdown_connections 2024-09-05T12:11:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 --2- 172.21.15.59:0/3846252704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 0x7f4db4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 --2- 172.21.15.59:0/3846252704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 0x7f4db4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 --2- 172.21.15.59:0/3846252704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db4104210 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 -- 172.21.15.59:0/3846252704 >> 172.21.15.59:0/3846252704 conn(0x7f4db40ff8d0 msgr2=0x7f4db4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 -- 172.21.15.59:0/3846252704 shutdown_connections 2024-09-05T12:11:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.617+0000 7f4dbb000640 1 -- 172.21.15.59:0/3846252704 wait complete. 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.618+0000 7f4dbb000640 1 Processor -- start 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.618+0000 7f4dbb000640 1 -- start start 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 0x7f4db419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 0x7f4db419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4113f80 con 0x7f4db4103e30 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4113e00 con 0x7f4db4108f60 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dbb000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4114100 con 0x7f4db41047e0 2024-09-05T12:11:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4db9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4db9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51954/0 (socket says 172.21.15.59:51954) 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4db9c00640 1 -- 172.21.15.59:0/509013614 learned_addr learned my addr 172.21.15.59:0/509013614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4db9200640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 0x7f4db419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.619+0000 7f4dba600640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 0x7f4db419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 -- 172.21.15.59:0/509013614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 msgr2=0x7f4db419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 0x7f4db419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 -- 172.21.15.59:0/509013614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 msgr2=0x7f4db419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 0x7f4db419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 -- 172.21.15.59:0/509013614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db419f410 con 0x7f4db4103e30 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db9c00640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4da802ade0 tx=0x7f4da80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db2000640 1 -- 172.21.15.59:0/509013614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da802fa20 con 0x7f4db4103e30 2024-09-05T12:11:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db2000640 1 -- 172.21.15.59:0/509013614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da802fbc0 con 0x7f4db4103e30 2024-09-05T12:11:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db419f6a0 con 0x7f4db4103e30 2024-09-05T12:11:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4db2000640 1 -- 172.21.15.59:0/509013614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da80466e0 con 0x7f4db4103e30 2024-09-05T12:11:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.620+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db41a6fe0 con 0x7f4db4103e30 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.623+0000 7f4dbf04b740 1 Processor -- start 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.623+0000 7f4db2000640 1 -- 172.21.15.59:0/509013614 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4da8046940 con 0x7f4db4103e30 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 -- start start 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x55829602f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 0x558296032320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558295f8b3e0 con 0x558296025670 2024-09-05T12:11:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558295f32d80 con 0x55829602ff30 2024-09-05T12:11:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558295f873a0 con 0x558296024d50 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4db9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x55829602f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dba600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4db9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x55829602f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51956/0 (socket says 172.21.15.59:51956) 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4dba600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:50436/0 (socket says 172.21.15.59:50436) 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4db9c00640 1 -- 172.21.15.59:0/2771232131 learned_addr learned my addr 172.21.15.59:0/2771232131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.624+0000 7f4db9200640 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 0x558296032320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 -- 172.21.15.59:0/2771232131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 msgr2=0x558296032320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 0x558296032320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 -- 172.21.15.59:0/2771232131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 msgr2=0x55829602f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x55829602f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 -- 172.21.15.59:0/2771232131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558295f51ae0 con 0x558296024d50 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4db9200640 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 0x558296032320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4db2000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 0x7f4d9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4dba600640 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009b80 tx=0x7f4dac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce057246d710c157 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4db2000640 1 -- 172.21.15.59:0/509013614 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4da80f0a50 con 0x7f4db4103e30 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.625+0000 7f4db9c00640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 0x7f4d9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4da6a00640 1 -- 172.21.15.59:0/2771232131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac004050 con 0x558296024d50 2024-09-05T12:11:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4da6a00640 1 -- 172.21.15.59:0/2771232131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dac005500 con 0x558296024d50 2024-09-05T12:11:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4da6a00640 1 -- 172.21.15.59:0/2771232131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac004d90 con 0x558296024d50 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2771232131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 msgr2=0x558296025130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009b80 tx=0x7f4dac011810 comp rx=0 tx=0).stop 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.626+0000 7f4db9c00640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 0x7f4d9c0a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006010 tx=0x7f4da80f6530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2771232131 shutdown_connections 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829602ff30 0x558296032320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x55829602f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2771232131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296025130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2771232131 >> 172.21.15.59:0/2771232131 conn(0x5582960212e0 msgr2=0x5582960216c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2771232131 shutdown_connections 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2771232131 wait complete. 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 -- start start 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.627+0000 7f4dbf04b740 1 Processor -- start 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x55829602ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 0x55829602d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558296032ba0 con 0x55829602ff30 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558295f460f0 con 0x558296025670 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558296032d20 con 0x558296024d50 2024-09-05T12:11:00.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dbf04b740 10 client.0 _ll_register_callbacks cb 0x558295f379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dba600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x55829602ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dba600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dba600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51968/0 (socket says 172.21.15.59:51968) 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4dba600640 1 -- 172.21.15.59:0/3866048303 learned_addr learned my addr 172.21.15.59:0/3866048303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.628+0000 7f4db9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 0x55829602d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 -- 172.21.15.59:0/3866048303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 msgr2=0x55829602ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x55829602ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 -- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 msgr2=0x55829602d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 0x55829602d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 -- 172.21.15.59:0/3866048303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558295cfc3c0 con 0x55829602ff30 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4db9200640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 0x55829602d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dba600640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4dac011d10 tx=0x7f4dac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7ff1113f5b79b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4d9b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac02d060 con 0x55829602ff30 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.629+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dac005500 con 0x55829602ff30 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4dac005170 con 0x55829602ff30 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4dac034050 con 0x55829602ff30 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4dbf04b740 10 client.17478 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:00.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55829603e900 con 0x55829602ff30 2024-09-05T12:11:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dac005500 con 0x55829602ff30 2024-09-05T12:11:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4dac028450 con 0x55829602ff30 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4dbf04b740 10 client.17478 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4dbf04b740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.630+0000 7f4dbf04b740 20 client.17478 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829603fcb0 0x5582960420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4dbf04b740 4 client.17478 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558295f98950 con 0x55829603fcb0 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4db9200640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829603fcb0 0x5582960420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4db9200640 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829603fcb0 0x5582960420a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a5681e8e4a96 server_cookie=e00e2ddd8595527d in_seq=0 out_seq=0 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.631+0000 7f4d9b400640 10 client.17478 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.632+0000 7f4d9b400640 1 -- 172.21.15.59:0/3866048303 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558295f98950 con 0x55829603fcb0 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.632+0000 7f4d9b400640 10 client.17478 handle_command_reply: tid=0 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.632+0000 7f4dbf04b740 1 client.17478 shutdown 2024-09-05T12:11:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 msgr2=0x55829602d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4dac011d10 tx=0x7f4dac000bc0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 shutdown_connections 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829603fcb0 0x5582960420a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55829602ff30 0x55829602d930 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558296025670 0x55829602d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/3866048303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x55829602ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 >> 172.21.15.59:0/3866048303 conn(0x5582960212e0 msgr2=0x558296027010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4d9b400640 0 client.17478 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 shutdown_connections 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 1 -- 172.21.15.59:0/3866048303 wait complete. 2024-09-05T12:11:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.633+0000 7f4dbf04b740 20 client.17478 trim_cache size 0 max 16384 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.651+0000 7f4dbf04b740 1 Processor -- start 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.651+0000 7f4dbf04b740 1 -- start start 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.651+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x55829602b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.651+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 0x55829602d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41aef60 con 0x558296024d50 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41b17e0 con 0x5582960212e0 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41af5d0 con 0x558296025670 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51982/0 (socket says 172.21.15.59:51982) 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 -- 172.21.15.59:0/1117102290 learned_addr learned my addr 172.21.15.59:0/1117102290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dba600640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x55829602b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dba600640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 0x55829602d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 -- 172.21.15.59:0/1117102290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 msgr2=0x55829602d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 0x55829602d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 -- 172.21.15.59:0/1117102290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 msgr2=0x55829602b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x55829602b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4db9200640 1 -- 172.21.15.59:0/1117102290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829602be60 con 0x558296024d50 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.652+0000 7f4dba600640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 0x55829602d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4db9200640 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ffcf0 tx=0x7f4da00139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf86de7400d50ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4da6000640 1 -- 172.21.15.59:0/1117102290 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004020 con 0x558296024d50 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4da6000640 1 -- 172.21.15.59:0/1117102290 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da00041c0 con 0x558296024d50 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4da6000640 1 -- 172.21.15.59:0/1117102290 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0010810 con 0x558296024d50 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 -- 172.21.15.59:0/1117102290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 msgr2=0x558296027800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ffcf0 tx=0x7f4da00139f0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 -- 172.21.15.59:0/1117102290 shutdown_connections 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296025670 0x55829602d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296024d50 0x558296027800 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/1117102290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x55829602b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.653+0000 7f4dbf04b740 1 -- 172.21.15.59:0/1117102290 >> 172.21.15.59:0/1117102290 conn(0x5582960423a0 msgr2=0x55829602ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.654+0000 7f4dbf04b740 1 -- 172.21.15.59:0/1117102290 shutdown_connections 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.654+0000 7f4dbf04b740 1 -- 172.21.15.59:0/1117102290 wait complete. 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.654+0000 7f4dbf04b740 1 -- start start 2024-09-05T12:11:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.654+0000 7f4dbf04b740 1 Processor -- start 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x558296038830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x558296034170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41afca0 con 0x558296025670 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41b1110 con 0x5582960212e0 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4db41b0370 con 0x558296024d50 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dbf04b740 10 client.0 _ll_register_callbacks cb 0x7f4dac009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:50472/0 (socket says 172.21.15.59:50472) 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 -- 172.21.15.59:0/2634162525 learned_addr learned my addr 172.21.15.59:0/2634162525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dba600640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x558296038830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4dba600640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x558296034170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 -- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 msgr2=0x558296038830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x558296038830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 -- 172.21.15.59:0/2634162525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 msgr2=0x558296034170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x558296034170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.655+0000 7f4db9200640 1 -- 172.21.15.59:0/2634162525 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582960349a0 con 0x558296024d50 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4db9200640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4da000bb60 tx=0x7f4da000bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac1f200fae8e193 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4d9b400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da002b070 con 0x558296024d50 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da00041c0 con 0x558296024d50 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4da00035f0 con 0x558296024d50 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4da00037f0 con 0x558296024d50 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4dbf04b740 10 client.27406 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.656+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5582960aa970 con 0x558296024d50 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da00041c0 con 0x558296024d50 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4da0010c60 con 0x558296024d50 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 10 client.27406 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 10 client.27406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 20 client.27406 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829602ff30 0x5582960ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 4 client.27406 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558295f92e60 con 0x55829602ff30 2024-09-05T12:11:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.657+0000 7f4dba600640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829602ff30 0x5582960ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:00.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dba600640 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829602ff30 0x5582960ad0f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eecbde8759f3cdb server_cookie=496a8ea8bfcec81c in_seq=0 out_seq=0 2024-09-05T12:11:00.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4d9b400640 10 client.27406 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4d9b400640 1 -- 172.21.15.59:0/2634162525 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558295f92e60 con 0x55829602ff30 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4d9b400640 10 client.27406 handle_command_reply: tid=0 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 client.27406 shutdown 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 msgr2=0x558296033b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4da000bb60 tx=0x7f4da000bb90 comp rx=0 tx=0).stop 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 shutdown_connections 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55829602ff30 0x5582960ad0f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558296025670 0x558296034170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558296024d50 0x558296033b60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4d9b400640 0 client.27406 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 --2- 172.21.15.59:0/2634162525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582960212e0 0x558296038830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 >> 172.21.15.59:0/2634162525 conn(0x5582960423a0 msgr2=0x55829601fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.658+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 shutdown_connections 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbf04b740 1 -- 172.21.15.59:0/2634162525 wait complete. 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbf04b740 20 client.27406 trim_cache size 0 max 16384 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:11:00.666 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 msgr2=0x7f4d9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 0x7f4d9c0a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006010 tx=0x7f4da80f6530 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 msgr2=0x7f4db419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.659+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4da802ade0 tx=0x7f4da80319f0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 shutdown_connections 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d9c0a2b80 0x7f4d9c0a5040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4db4108f60 0x7f4db419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4db41047e0 0x7f4db419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 --2- 172.21.15.59:0/509013614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db4103e30 0x7f4db419a460 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 >> 172.21.15.59:0/509013614 conn(0x7f4db40ff8d0 msgr2=0x7f4db4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 shutdown_connections 2024-09-05T12:11:00.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:00.660+0000 7f4dbb000640 1 -- 172.21.15.59:0/509013614 wait complete. 2024-09-05T12:11:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:00 smithi189 ceph-mon[34772]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:01.116 INFO:tasks.workunit.client.0.smithi059.stdout:437/600: open fd = 3 2024-09-05T12:11:01.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.116 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.116 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:438/600: open fd = 3 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:439/600: open fd = 3 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:440/600: open fd = 3 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.117 INFO:tasks.workunit.client.0.smithi059.stdout:441/600: open fd = 3 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:442/600: open fd = 3 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:443/600: open fd = 3 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:444/600: open fd = 3 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.118 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:445/600: open fd = 3 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:446/600: open fd = 3 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:447/600: open fd = 3 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.119 INFO:tasks.workunit.client.0.smithi059.stdout:448/600: open fd = 3 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:449/600: open fd = 3 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:450/600: open fd = 3 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:451/600: open fd = 3 2024-09-05T12:11:01.120 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:452/600: open fd = 3 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:453/600: open fd = 3 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:454/600: open fd = 3 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.121 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:455/600: open fd = 3 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:456/600: open fd = 3 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:457/600: open fd = 3 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.122 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:458/600: open fd = 3 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:459/600: open fd = 3 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.123 INFO:tasks.workunit.client.0.smithi059.stdout:460/600: open fd = 3 2024-09-05T12:11:01.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.124 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.124 INFO:tasks.workunit.client.0.smithi059.stdout:461/600: open fd = 3 2024-09-05T12:11:01.124 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:462/600: open fd = 3 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:463/600: open fd = 3 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:464/600: open fd = 3 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.125 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:465/600: open fd = 3 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:466/600: open fd = 3 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:467/600: open fd = 3 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.126 INFO:tasks.workunit.client.0.smithi059.stdout:468/600: open fd = 3 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:469/600: open fd = 3 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:470/600: open fd = 3 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:471/600: open fd = 3 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.127 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.128 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.128 INFO:tasks.workunit.client.0.smithi059.stdout:472/600: open fd = 3 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:473/600: open fd = 3 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:474/600: open fd = 3 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:475/600: open fd = 3 2024-09-05T12:11:01.129 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:476/600: open fd = 3 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:477/600: open fd = 3 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:478/600: open fd = 3 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.130 INFO:tasks.workunit.client.0.smithi059.stdout:479/600: open fd = 3 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:480/600: open fd = 3 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:481/600: open fd = 3 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:482/600: open fd = 3 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.131 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:483/600: open fd = 3 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:484/600: open fd = 3 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:485/600: open fd = 3 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.132 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:486/600: open fd = 3 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:487/600: open fd = 3 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:488/600: open fd = 3 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:489/600: open fd = 3 2024-09-05T12:11:01.133 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:490/600: open fd = 3 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:491/600: open fd = 3 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:492/600: open fd = 3 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.134 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:493/600: open fd = 3 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:494/600: open fd = 3 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:495/600: open fd = 3 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:496/600: open fd = 3 2024-09-05T12:11:01.135 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:497/600: open fd = 3 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:498/600: open fd = 3 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:11:01.136 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:11:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:00 smithi089 ceph-mon[34499]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:00 smithi059 ceph-mon[30025]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:01.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:11:01.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:11:01.882 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:02.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 -- 172.21.15.59:0/626922867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 msgr2=0x7fa3881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 --2- 172.21.15.59:0/626922867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa3881089f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa370009900 tx=0x7fa37002a410 comp rx=0 tx=0).stop 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 -- 172.21.15.59:0/626922867 shutdown_connections 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 --2- 172.21.15.59:0/626922867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388109120 0x7fa388110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 --2- 172.21.15.59:0/626922867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa3881089f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 --2- 172.21.15.59:0/626922867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388103ff0 0x7fa3881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 -- 172.21.15.59:0/626922867 >> 172.21.15.59:0/626922867 conn(0x7fa3880ffa90 msgr2=0x7fa388101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 -- 172.21.15.59:0/626922867 shutdown_connections 2024-09-05T12:11:02.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.423+0000 7fa38d800640 1 -- 172.21.15.59:0/626922867 wait complete. 2024-09-05T12:11:02.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 Processor -- start 2024-09-05T12:11:02.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 -- start start 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 0x7fa38819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa38819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 0x7fa38819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa388114140 con 0x7fa3881049a0 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa388113fc0 con 0x7fa388109120 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.424+0000 7fa38d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881142c0 con 0x7fa388103ff0 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa38819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa386a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 0x7fa38819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa386a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 0x7fa38819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58070/0 (socket says 172.21.15.59:58070) 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa386a00640 1 -- 172.21.15.59:0/189192841 learned_addr learned my addr 172.21.15.59:0/189192841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa387400640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 0x7fa38819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 -- 172.21.15.59:0/189192841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 msgr2=0x7fa38819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 0x7fa38819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 -- 172.21.15.59:0/189192841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 msgr2=0x7fa38819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:02.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 0x7fa38819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa37e000640 1 -- 172.21.15.59:0/189192841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38819f650 con 0x7fa3881049a0 2024-09-05T12:11:02.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.425+0000 7fa387400640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 0x7fa38819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa37e000640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa38819abe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa370005940 tx=0x7fa3700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37003f030 con 0x7fa3881049a0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38819f8e0 con 0x7fa3881049a0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3700044c0 con 0x7fa3881049a0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370035070 con 0x7fa3881049a0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.426+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a7220 con 0x7fa3881049a0 2024-09-05T12:11:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.427+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3881043d0 con 0x7fa3881049a0 2024-09-05T12:11:02.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.429+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa370042030 con 0x7fa3881049a0 2024-09-05T12:11:02.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.430+0000 7fa37fe00640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 0x7fa36c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:02.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.430+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa3700ef6d0 con 0x7fa3881049a0 2024-09-05T12:11:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.431+0000 7fa386a00640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 0x7fa36c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.431+0000 7fa386a00640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 0x7fa36c0a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa378009c90 tx=0x7fa378005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.435+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3700bbb70 con 0x7fa3881049a0 2024-09-05T12:11:02.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.658+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa38819ba00 con 0x7fa3881049a0 2024-09-05T12:11:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.659+0000 7fa37fe00640 1 -- 172.21.15.59:0/189192841 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fa3700bfe70 con 0x7fa3881049a0 2024-09-05T12:11:02.661 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:11:02.661 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:11:02.661 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 msgr2=0x7fa36c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 0x7fa36c0a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa378009c90 tx=0x7fa378005f90 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 msgr2=0x7fa38819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa38819abe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa370005940 tx=0x7fa3700040c0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 shutdown_connections 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa36c0a2c00 0x7fa36c0a50c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388109120 0x7fa38819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3881049a0 0x7fa38819abe0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 --2- 172.21.15.59:0/189192841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388103ff0 0x7fa38819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 >> 172.21.15.59:0/189192841 conn(0x7fa3880ffa90 msgr2=0x7fa388100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.662+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 shutdown_connections 2024-09-05T12:11:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:02.663+0000 7fa38d800640 1 -- 172.21.15.59:0/189192841 wait complete. 2024-09-05T12:11:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:02 smithi059 ceph-mon[30025]: pgmap v795: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:02.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/189192841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:02 smithi189 ceph-mon[34772]: pgmap v795: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/189192841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:02 smithi089 ceph-mon[34499]: pgmap v795: 129 pgs: 129 active+clean; 6.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/189192841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:03.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:11:03.912 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 -- 172.21.15.59:0/1639294221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b8104010 msgr2=0x7f71b81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 --2- 172.21.15.59:0/1639294221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b8104010 0x7f71b81043f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009900 tx=0x7f71ac02a410 comp rx=0 tx=0).stop 2024-09-05T12:11:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 -- 172.21.15.59:0/1639294221 shutdown_connections 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 --2- 172.21.15.59:0/1639294221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8109140 0x7f71b8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 --2- 172.21.15.59:0/1639294221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b81049c0 0x7f71b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 --2- 172.21.15.59:0/1639294221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b8104010 0x7f71b81043f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 -- 172.21.15.59:0/1639294221 >> 172.21.15.59:0/1639294221 conn(0x7f71b80ffa50 msgr2=0x7f71b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.474+0000 7f71bfe00640 1 -- 172.21.15.59:0/1639294221 shutdown_connections 2024-09-05T12:11:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.475+0000 7f71bfe00640 1 -- 172.21.15.59:0/1639294221 wait complete. 2024-09-05T12:11:04.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.476+0000 7f71bfe00640 1 Processor -- start 2024-09-05T12:11:04.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.476+0000 7f71bfe00640 1 -- start start 2024-09-05T12:11:04.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 0x7f71b819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b8109140 0x7f71b819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81141c0 con 0x7f71b81049c0 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8114040 con 0x7f71b8109140 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bfe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8114340 con 0x7f71b8104010 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 0x7f71b819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43910/0 (socket says 172.21.15.59:43910) 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bca00640 1 -- 172.21.15.59:0/3788902122 learned_addr learned my addr 172.21.15.59:0/3788902122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:04.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.477+0000 7f71bde00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b8109140 0x7f71b819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 -- 172.21.15.59:0/3788902122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b8109140 msgr2=0x7f71b819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b8109140 0x7f71b819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 -- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 msgr2=0x7f71b819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 -- 172.21.15.59:0/3788902122 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b819f5d0 con 0x7f71b8104010 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bca00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bd400640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 0x7f71b819a620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71ac02a930 tx=0x7f71ac003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71b6000640 1 -- 172.21.15.59:0/3788902122 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac03f070 con 0x7f71b8104010 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b819f860 con 0x7f71b8104010 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71b6000640 1 -- 172.21.15.59:0/3788902122 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac031c90 con 0x7f71b8104010 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.478+0000 7f71b6000640 1 -- 172.21.15.59:0/3788902122 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac035030 con 0x7f71b8104010 2024-09-05T12:11:04.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.479+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a71a0 con 0x7f71b8104010 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.481+0000 7f71c287c740 1 Processor -- start 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 -- start start 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71b6000640 1 -- 172.21.15.59:0/3788902122 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71ac042070 con 0x7f71b8104010 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 0x55e4115b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e41151a3e0 con 0x55e4115b3c70 2024-09-05T12:11:04.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e4114c1d80 con 0x55e4115b4590 2024-09-05T12:11:04.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71c287c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e4115163a0 con 0x55e4115bee50 2024-09-05T12:11:04.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 0x55e4115b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53408/0 (socket says 172.21.15.59:53408) 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.482+0000 7f71bd400640 1 -- 172.21.15.59:0/2046537839 learned_addr learned my addr 172.21.15.59:0/2046537839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bca00640 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 -- 172.21.15.59:0/2046537839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 msgr2=0x55e4115c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 -- 172.21.15.59:0/2046537839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 msgr2=0x55e4115b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 0x55e4115b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 -- 172.21.15.59:0/2046537839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4114e0ae0 con 0x55e4115b4590 2024-09-05T12:11:04.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bde00640 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 0x55e4115b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.483+0000 7f71bd400640 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0f3a00 tx=0x7f71ac0f3a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59c52e9bd9daafc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71aa600640 1 -- 172.21.15.59:0/2046537839 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac103070 con 0x55e4115b4590 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71aa600640 1 -- 172.21.15.59:0/2046537839 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac0fa930 con 0x55e4115b4590 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71aa600640 1 -- 172.21.15.59:0/2046537839 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0fc830 con 0x55e4115b4590 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71b6000640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 0x7f71a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71b6000640 1 -- 172.21.15.59:0/3788902122 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71ac0efae0 con 0x7f71b8104010 2024-09-05T12:11:04.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71bde00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 0x7f71a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 -- 172.21.15.59:0/2046537839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 msgr2=0x55e4115be910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0f3a00 tx=0x7f71ac0f3a30 comp rx=0 tx=0).stop 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 -- 172.21.15.59:0/2046537839 shutdown_connections 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115be910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 --2- 172.21.15.59:0/2046537839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b3c70 0x55e4115b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.484+0000 7f71c287c740 1 -- 172.21.15.59:0/2046537839 >> 172.21.15.59:0/2046537839 conn(0x55e4115b0200 msgr2=0x55e4115b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.485+0000 7f71c287c740 1 -- 172.21.15.59:0/2046537839 shutdown_connections 2024-09-05T12:11:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.485+0000 7f71c287c740 1 -- 172.21.15.59:0/2046537839 wait complete. 2024-09-05T12:11:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.485+0000 7f71bde00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 0x7f71a00a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.485+0000 7f71c287c740 1 -- start start 2024-09-05T12:11:04.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.485+0000 7f71c287c740 1 Processor -- start 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b3c70 0x55e4115bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e4115c17e0 con 0x55e4115bee50 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e4114d50f0 con 0x55e4115b4590 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e4115c1960 con 0x55e4115b3c70 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71c287c740 10 client.0 _ll_register_callbacks cb 0x55e4114bcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71bca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71bd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b3c70 0x55e4115bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71bca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53424/0 (socket says 172.21.15.59:53424) 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.486+0000 7f71bca00640 1 -- 172.21.15.59:0/4102259629 learned_addr learned my addr 172.21.15.59:0/4102259629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 -- 172.21.15.59:0/4102259629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b3c70 msgr2=0x55e4115bbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b3c70 0x55e4115bbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 -- 172.21.15.59:0/4102259629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 msgr2=0x55e4115bc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115bc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 -- 172.21.15.59:0/4102259629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41128b3c0 con 0x55e4115b4590 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f71bca00640 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71a40119a0 tx=0x7f71a4011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5e9a3e5049519e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f719ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.487+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004260 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac0fa930 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71a4004400 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f71a4024050 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 10 client.27500 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e4115cd7f0 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac0fa930 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f71a400f3c0 con 0x55e4115b4590 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 10 client.27500 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 20 client.27500 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e4115cec50 0x55e4115d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 4 client.27500 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.488+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e411527950 con 0x55e4115cec50 2024-09-05T12:11:04.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.489+0000 7f71bd400640 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e4115cec50 0x55e4115d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.489+0000 7f71bd400640 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e4115cec50 0x55e4115d1070 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1ae84b82a8998a server_cookie=f3d7f3e3fe3df82a in_seq=0 out_seq=0 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.489+0000 7f719ea00640 10 client.27500 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f719ea00640 1 -- 172.21.15.59:0/4102259629 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e411527950 con 0x55e4115cec50 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f719ea00640 10 client.27500 handle_command_reply: tid=0 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 client.27500 shutdown 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 msgr2=0x55e4115bc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71a40119a0 tx=0x7f71a4011e60 comp rx=0 tx=0).stop 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 shutdown_connections 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e4115cec50 0x55e4115d1070 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115bc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b4590 0x55e4115bc230 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f719ea00640 0 client.27500 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 --2- 172.21.15.59:0/4102259629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b3c70 0x55e4115bbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.490+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 >> 172.21.15.59:0/4102259629 conn(0x55e4115b0200 msgr2=0x55e4115af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.491+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 shutdown_connections 2024-09-05T12:11:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.491+0000 7f71c287c740 1 -- 172.21.15.59:0/4102259629 wait complete. 2024-09-05T12:11:04.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.491+0000 7f71c287c740 20 client.27500 trim_cache size 0 max 16384 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.504+0000 7f71c287c740 1 Processor -- start 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 -- start start 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b4590 0x55e4115d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81af100 con 0x55e4115bee50 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81b1980 con 0x55e4115b3c70 2024-09-05T12:11:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71c287c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81af770 con 0x55e4115b4590 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43940/0 (socket says 172.21.15.59:43940) 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71bde00640 1 -- 172.21.15.59:0/3454307591 learned_addr learned my addr 172.21.15.59:0/3454307591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71bd400640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.505+0000 7f71bca00640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b4590 0x55e4115d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 -- 172.21.15.59:0/3454307591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b4590 msgr2=0x55e4115d6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b4590 0x55e4115d6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 -- 172.21.15.59:0/3454307591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 msgr2=0x55e4115d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115d6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 -- 172.21.15.59:0/3454307591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4115ab770 con 0x55e4115bee50 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bd400640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71bde00640 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f71b000f670 tx=0x7f71b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadbb3feb0a0f5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71a9c00640 1 -- 172.21.15.59:0/3454307591 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0003a30 con 0x55e4115bee50 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71a9c00640 1 -- 172.21.15.59:0/3454307591 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0003bd0 con 0x55e4115bee50 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.506+0000 7f71a9c00640 1 -- 172.21.15.59:0/3454307591 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0010b30 con 0x55e4115bee50 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- 172.21.15.59:0/3454307591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 msgr2=0x55e4115b8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f71b000f670 tx=0x7f71b0019000 comp rx=0 tx=0).stop 2024-09-05T12:11:04.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- 172.21.15.59:0/3454307591 shutdown_connections 2024-09-05T12:11:04.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115bee50 0x55e4115b8f50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115b4590 0x55e4115d6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 --2- 172.21.15.59:0/3454307591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- 172.21.15.59:0/3454307591 >> 172.21.15.59:0/3454307591 conn(0x55e4115b0200 msgr2=0x55e4115bdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- 172.21.15.59:0/3454307591 shutdown_connections 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- 172.21.15.59:0/3454307591 wait complete. 2024-09-05T12:11:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.507+0000 7f71c287c740 1 -- start start 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 Processor -- start 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81afe40 con 0x55e4115b4590 2024-09-05T12:11:04.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81b12b0 con 0x55e4115b3c70 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81b0510 con 0x55e4115bee50 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71c287c740 10 client.0 _ll_register_callbacks cb 0x7f71a400bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53446/0 (socket says 172.21.15.59:53446) 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43952/0 (socket says 172.21.15.59:43952) 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.508+0000 7f71bd400640 1 -- 172.21.15.59:0/2081055128 learned_addr learned my addr 172.21.15.59:0/2081055128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bde00640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 -- 172.21.15.59:0/2081055128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 msgr2=0x55e4115c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 -- 172.21.15.59:0/2081055128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 msgr2=0x55e4115c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 -- 172.21.15.59:0/2081055128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4115b0e90 con 0x55e4115b3c70 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bca00640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bde00640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71bd400640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0bbf10 tx=0x7f71ac031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef90d0685a9d174 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f719ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac049040 con 0x55e4115b3c70 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71ac0ba280 con 0x55e4115b3c70 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.509+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0003bd0 con 0x55e4115b3c70 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f71ac0b8220 con 0x55e4115b3c70 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 10 client.27506 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e411639c20 con 0x55e4115b3c70 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0003bd0 con 0x55e4115b3c70 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f71ac0ed1f0 con 0x55e4115b3c70 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 10 client.27506 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 20 client.27506 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e41163a1b0 0x55e41163c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 4 client.27506 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.510+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e411521e60 con 0x55e41163a1b0 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f71bca00640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e41163a1b0 0x55e41163c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f71bca00640 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e41163a1b0 0x55e41163c5a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a10e2614cb9c17c server_cookie=6e1901e11f326737 in_seq=0 out_seq=0 2024-09-05T12:11:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f719ea00640 10 client.27506 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f719ea00640 1 -- 172.21.15.59:0/2081055128 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e411521e60 con 0x55e41163a1b0 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f719ea00640 10 client.27506 handle_command_reply: tid=0 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f71c287c740 1 client.27506 shutdown 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 msgr2=0x55e4115c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.511+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0bbf10 tx=0x7f71ac031490 comp rx=0 tx=0).stop 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 shutdown_connections 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4115bee50 0x55e4115c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e41163a1b0 0x55e41163c5a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e4115b4590 0x55e4115c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 --2- 172.21.15.59:0/2081055128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e4115b3c70 0x55e4115c29f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f719ea00640 0 client.27506 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 >> 172.21.15.59:0/2081055128 conn(0x55e4115b0200 msgr2=0x55e4115aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 shutdown_connections 2024-09-05T12:11:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 1 -- 172.21.15.59:0/2081055128 wait complete. 2024-09-05T12:11:04.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71c287c740 20 client.27506 trim_cache size 0 max 16384 2024-09-05T12:11:04.522 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:11:04.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 msgr2=0x7f71a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 0x7f71a00a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0009500 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 msgr2=0x7f71b819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.512+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 0x7f71b819a620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71ac02a930 tx=0x7f71ac003b90 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 shutdown_connections 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f71a00a2b80 0x7f71a00a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b8109140 0x7f71b819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71b81049c0 0x7f71b819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 --2- 172.21.15.59:0/3788902122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b8104010 0x7f71b819a620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 >> 172.21.15.59:0/3788902122 conn(0x7f71b80ffa50 msgr2=0x7f71b8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 shutdown_connections 2024-09-05T12:11:04.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:04.513+0000 7f71bfe00640 1 -- 172.21.15.59:0/3788902122 wait complete. 2024-09-05T12:11:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:04 smithi189 ceph-mon[34772]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:04 smithi089 ceph-mon[34499]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:05.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:04 smithi059 ceph-mon[30025]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:06.381 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:11:06.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:11:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:06 smithi189 ceph-mon[34772]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:07.098 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:06 smithi059 ceph-mon[30025]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:06 smithi089 ceph-mon[34499]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 -- 172.21.15.59:0/4222894575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10077040 msgr2=0x7f6a101124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 --2- 172.21.15.59:0/4222894575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10077040 0x7f6a101124f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009900 tx=0x7f69fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 -- 172.21.15.59:0/4222894575 shutdown_connections 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 --2- 172.21.15.59:0/4222894575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10112a30 0x7f6a10114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 --2- 172.21.15.59:0/4222894575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10077040 0x7f6a101124f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 --2- 172.21.15.59:0/4222894575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 0x7f6a10076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.371+0000 7f6a16400640 1 -- 172.21.15.59:0/4222894575 >> 172.21.15.59:0/4222894575 conn(0x7f6a100ffa90 msgr2=0x7f6a10101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.372+0000 7f6a16400640 1 -- 172.21.15.59:0/4222894575 shutdown_connections 2024-09-05T12:11:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.372+0000 7f6a16400640 1 -- 172.21.15.59:0/4222894575 wait complete. 2024-09-05T12:11:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.372+0000 7f6a16400640 1 Processor -- start 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 -- start start 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 0x7f6a101a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10077040 0x7f6a101a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10118610 con 0x7f6a10077040 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10118490 con 0x7f6a10112a30 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a16400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10118790 con 0x7f6a10076720 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10077040 0x7f6a101a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53454/0 (socket says 172.21.15.59:53454) 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a0fe00640 1 -- 172.21.15.59:0/3541835451 learned_addr learned my addr 172.21.15.59:0/3541835451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.373+0000 7f6a0f400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 0x7f6a101a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 -- 172.21.15.59:0/3541835451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 msgr2=0x7f6a101a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 0x7f6a101a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 -- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10077040 msgr2=0x7f6a101a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10077040 0x7f6a101a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 -- 172.21.15.59:0/3541835451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1019eff0 con 0x7f6a10112a30 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800eb10 tx=0x7f6a0800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a07e00640 1 -- 172.21.15.59:0/3541835451 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08004b20 con 0x7f6a10112a30 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a07e00640 1 -- 172.21.15.59:0/3541835451 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a08004cc0 con 0x7f6a10112a30 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a07e00640 1 -- 172.21.15.59:0/3541835451 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08010790 con 0x7f6a10112a30 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1019f2e0 con 0x7f6a10112a30 2024-09-05T12:11:07.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.374+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a101ab5d0 con 0x7f6a10112a30 2024-09-05T12:11:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.377+0000 7f6a18e83740 1 Processor -- start 2024-09-05T12:11:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a07e00640 1 -- 172.21.15.59:0/3541835451 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a08005020 con 0x7f6a10112a30 2024-09-05T12:11:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 -- start start 2024-09-05T12:11:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe49ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fdb0290 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fd57b40 con 0x55ff6fe54ca0 2024-09-05T12:11:07.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.378+0000 7f6a18e83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fdac250 con 0x55ff6fe49ac0 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe49ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe49ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58136/0 (socket says 172.21.15.59:58136) 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0ea00640 1 -- 172.21.15.59:0/1877782799 learned_addr learned my addr 172.21.15.59:0/1877782799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 -- 172.21.15.59:0/1877782799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 msgr2=0x55ff6fe49ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe49ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 -- 172.21.15.59:0/1877782799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 msgr2=0x55ff6fe57090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe57090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0f400640 1 -- 172.21.15.59:0/1877782799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fb216a0 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.379+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe54760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009bd0 tx=0x7f6a00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a7f0e6f4fe16e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a07e00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 0x7f69f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a07e00640 1 -- 172.21.15.59:0/3541835451 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a080cfb30 con 0x7f6a10112a30 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a04c00640 1 -- 172.21.15.59:0/1877782799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004230 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a04c00640 1 -- 172.21.15.59:0/1877782799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a00009e40 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a04c00640 1 -- 172.21.15.59:0/1877782799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004d10 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.380+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 0x7f69f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- 172.21.15.59:0/1877782799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 msgr2=0x55ff6fe54760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe54760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009bd0 tx=0x7f6a00011810 comp rx=0 tx=0).stop 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 0x7f69f40a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55ff6fe473d0 tx=0x7f69fc02a3f0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- 172.21.15.59:0/1877782799 shutdown_connections 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe54760 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1877782799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe49ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- 172.21.15.59:0/1877782799 >> 172.21.15.59:0/1877782799 conn(0x55ff6fe46050 msgr2=0x55ff6fe46430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- 172.21.15.59:0/1877782799 shutdown_connections 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- 172.21.15.59:0/1877782799 wait complete. 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.381+0000 7f6a18e83740 1 -- start start 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 Processor -- start 2024-09-05T12:11:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe51780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fe57960 con 0x55ff6fe54ca0 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fd6afa0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fe57ae0 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.382+0000 7f6a18e83740 10 client.0 _ll_register_callbacks cb 0x55ff6fabeba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53470/0 (socket says 172.21.15.59:53470) 2024-09-05T12:11:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 -- 172.21.15.59:0/156035532 learned_addr learned my addr 172.21.15.59:0/156035532 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:07.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe51780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 -- 172.21.15.59:0/156035532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe4a3e0 msgr2=0x55ff6fe51780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe51780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 -- 172.21.15.59:0/156035532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 msgr2=0x55ff6fe51cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe51cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 -- 172.21.15.59:0/156035532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6fe524f0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe51cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f6a0f400640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005890 tx=0x7f6a000038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871a70141500f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.383+0000 7f69f9600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00027020 con 0x55ff6fe49ac0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff6fdbd800 con 0x55ff6fe49ac0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a00003cc0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a00032050 con 0x55ff6fe49ac0 2024-09-05T12:11:07.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f6a18e83740 10 client.27518 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff6fe637b0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ff6fdbd800 con 0x55ff6fe49ac0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.384+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6a00018070 con 0x55ff6fe49ac0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 10 client.27518 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 10 client.27518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 20 client.27518 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fe64bb0 0x55ff6fe66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 4 client.27518 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff6fdb7d10 con 0x55ff6fe64bb0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.385+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fe64bb0 0x55ff6fe66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.386+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fe64bb0 0x55ff6fe66fa0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0abd1a9fdf86225 server_cookie=dd48b1a8a9f04953 in_seq=0 out_seq=0 2024-09-05T12:11:07.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.386+0000 7f69f9600640 10 client.27518 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.386+0000 7f69f9600640 1 -- 172.21.15.59:0/156035532 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff6fdb7d10 con 0x55ff6fe64bb0 2024-09-05T12:11:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.386+0000 7f69f9600640 10 client.27518 handle_command_reply: tid=0 2024-09-05T12:11:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.386+0000 7f6a18e83740 1 client.27518 shutdown 2024-09-05T12:11:07.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 msgr2=0x55ff6fe544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005890 tx=0x7f6a000038a0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 shutdown_connections 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fe64bb0 0x55ff6fe66fa0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe51cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe51780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 --2- 172.21.15.59:0/156035532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe544d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f69f9600640 0 client.27518 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 >> 172.21.15.59:0/156035532 conn(0x55ff6fe46050 msgr2=0x55ff6fe451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 shutdown_connections 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 1 -- 172.21.15.59:0/156035532 wait complete. 2024-09-05T12:11:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.387+0000 7f6a18e83740 20 client.27518 trim_cache size 0 max 16384 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 Processor -- start 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 -- start start 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe6cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b3440 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b5cc0 con 0x55ff6fe54ca0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.405+0000 7f6a18e83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b3ab0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe6cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58154/0 (socket says 172.21.15.59:58154) 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 -- 172.21.15.59:0/1449826547 learned_addr learned my addr 172.21.15.59:0/1449826547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 -- 172.21.15.59:0/1449826547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 msgr2=0x55ff6fe5cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe5cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 -- 172.21.15.59:0/1449826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 msgr2=0x55ff6fe6cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe6cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0f400640 1 -- 172.21.15.59:0/1449826547 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fe67530 con 0x55ff6fe49ac0 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.406+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe6cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a0f400640 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a00025000 tx=0x7f6a000059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe90df23b84f83c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f69fbe00640 1 -- 172.21.15.59:0/1449826547 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00027070 con 0x55ff6fe49ac0 2024-09-05T12:11:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f69fbe00640 1 -- 172.21.15.59:0/1449826547 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a00003df0 con 0x55ff6fe49ac0 2024-09-05T12:11:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f69fbe00640 1 -- 172.21.15.59:0/1449826547 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0001e040 con 0x55ff6fe49ac0 2024-09-05T12:11:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 -- 172.21.15.59:0/1449826547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 msgr2=0x55ff6fe6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a00025000 tx=0x7f6a000059b0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 -- 172.21.15.59:0/1449826547 shutdown_connections 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe6cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.407+0000 7f6a18e83740 1 --2- 172.21.15.59:0/1449826547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe6c760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.408+0000 7f6a18e83740 1 -- 172.21.15.59:0/1449826547 >> 172.21.15.59:0/1449826547 conn(0x55ff6fe46050 msgr2=0x55ff6fe451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.408+0000 7f6a18e83740 1 -- 172.21.15.59:0/1449826547 shutdown_connections 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.408+0000 7f6a18e83740 1 -- 172.21.15.59:0/1449826547 wait complete. 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.408+0000 7f6a18e83740 1 -- start start 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.408+0000 7f6a18e83740 1 Processor -- start 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe59620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b4180 con 0x55ff6fe49ac0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b55f0 con 0x55ff6fe4a3e0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101b4850 con 0x55ff6fe54ca0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a18e83740 10 client.0 _ll_register_callbacks cb 0x7f6a00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44004/0 (socket says 172.21.15.59:44004) 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0f400640 1 -- 172.21.15.59:0/4093171519 learned_addr learned my addr 172.21.15.59:0/4093171519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe59620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0ea00640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 -- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe4a3e0 msgr2=0x55ff6fe590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 -- 172.21.15.59:0/4093171519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 msgr2=0x55ff6fe58ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0fe00640 1 -- 172.21.15.59:0/4093171519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6fe4b040 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.409+0000 7f6a0f400640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f6a0fe00640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe59620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a080cd190 tx=0x7f6a0809bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9efd7133a66962d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f69f9600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a080bad60 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a00003df0 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6a0809b370 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a08084050 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f6a18e83740 10 client.27448 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff6fe45e80 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.410+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a00003df0 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6a08005310 con 0x55ff6fe54ca0 2024-09-05T12:11:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 10 client.27448 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 10 client.27448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 20 client.27448 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fecff10 0x55ff6fed2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 4 client.27448 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff6fd33fb0 con 0x55ff6fecff10 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a0f400640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fecff10 0x55ff6fed2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f6a0f400640 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fecff10 0x55ff6fed2300 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd0c91f6947a0ab server_cookie=5495927dfd342fc8 in_seq=0 out_seq=0 2024-09-05T12:11:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.411+0000 7f69f9600640 10 client.27448 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f69f9600640 1 -- 172.21.15.59:0/4093171519 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff6fd33fb0 con 0x55ff6fecff10 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f69f9600640 10 client.27448 handle_command_reply: tid=0 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 client.27448 shutdown 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 msgr2=0x55ff6fe59620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe59620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a080cd190 tx=0x7f6a0809bd70 comp rx=0 tx=0).stop 2024-09-05T12:11:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 shutdown_connections 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff6fe54ca0 0x55ff6fe59620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ff6fe4a3e0 0x55ff6fe590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ff6fecff10 0x55ff6fed2300 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 --2- 172.21.15.59:0/4093171519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ff6fe49ac0 0x55ff6fe58ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 >> 172.21.15.59:0/4093171519 conn(0x55ff6fe46050 msgr2=0x55ff6fe5fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f69f9600640 0 client.27448 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 shutdown_connections 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 1 -- 172.21.15.59:0/4093171519 wait complete. 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.413+0000 7f6a18e83740 20 client.27448 trim_cache size 0 max 16384 2024-09-05T12:11:07.418 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "f496e932-fdd6-4431-b3cb-996368b32a4d", 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 msgr2=0x7f69f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 0x7f69f40a50c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55ff6fe473d0 tx=0x7f69fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 msgr2=0x7f6a1019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:07.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800eb10 tx=0x7f6a0800c2d0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 shutdown_connections 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f69f40a2c00 0x7f69f40a50c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a10112a30 0x7f6a1019e8c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a10077040 0x7f6a101a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 --2- 172.21.15.59:0/3541835451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a10076720 0x7f6a101a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 >> 172.21.15.59:0/3541835451 conn(0x7f6a100ffa90 msgr2=0x7f6a10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 shutdown_connections 2024-09-05T12:11:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:07.414+0000 7f6a16400640 1 -- 172.21.15.59:0/3541835451 wait complete. 2024-09-05T12:11:08.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:11:08.640 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:09.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:08 smithi059 ceph-mon[30025]: pgmap v798: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:09.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:08 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:11:09.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:08 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:09.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:08 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:11:09.132 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:08 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:08 smithi089 ceph-mon[34499]: pgmap v798: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:08 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:11:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:08 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:08 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:11:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:08 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 -- 172.21.15.59:0/3288843166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940077040 msgr2=0x7f0940112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 --2- 172.21.15.59:0/3288843166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940077040 0x7f0940112510 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f092802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 -- 172.21.15.59:0/3288843166 shutdown_connections 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 --2- 172.21.15.59:0/3288843166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940112a50 0x7f0940114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 --2- 172.21.15.59:0/3288843166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940077040 0x7f0940112510 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 --2- 172.21.15.59:0/3288843166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940076720 0x7f0940076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 -- 172.21.15.59:0/3288843166 >> 172.21.15.59:0/3288843166 conn(0x7f09400ffa50 msgr2=0x7f0940101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 -- 172.21.15.59:0/3288843166 shutdown_connections 2024-09-05T12:11:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.246+0000 7f0947800640 1 -- 172.21.15.59:0/3288843166 wait complete. 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 Processor -- start 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 -- start start 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940077040 0x7f09401a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940112a50 0x7f094019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09401185e0 con 0x7f0940112a50 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0940118460 con 0x7f0940076720 2024-09-05T12:11:09.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.247+0000 7f0947800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0940118760 con 0x7f0940077040 2024-09-05T12:11:09.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53480/0 (socket says 172.21.15.59:53480) 2024-09-05T12:11:09.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 -- 172.21.15.59:0/3255346075 learned_addr learned my addr 172.21.15.59:0/3255346075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:09.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0937e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940077040 0x7f09401a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0945800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940112a50 0x7f094019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 -- 172.21.15.59:0/3255346075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940077040 msgr2=0x7f09401a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940077040 0x7f09401a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 -- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940112a50 msgr2=0x7f094019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940112a50 0x7f094019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.248+0000 7f0944e00640 1 -- 172.21.15.59:0/3255346075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094019f0e0 con 0x7f0940076720 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f0944e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9f0 tx=0x7f093000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f093e000640 1 -- 172.21.15.59:0/3255346075 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093000cd80 con 0x7f0940076720 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f093e000640 1 -- 172.21.15.59:0/3255346075 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930005500 con 0x7f0940076720 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094019f3d0 con 0x7f0940076720 2024-09-05T12:11:09.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f093e000640 1 -- 172.21.15.59:0/3255346075 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930010690 con 0x7f0940076720 2024-09-05T12:11:09.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.249+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401ab6c0 con 0x7f0940076720 2024-09-05T12:11:09.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.252+0000 7f094a418740 1 Processor -- start 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.252+0000 7f094a418740 1 -- start start 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.252+0000 7f093e000640 1 -- 172.21.15.59:0/3255346075 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09300046c0 con 0x7f0940076720 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.252+0000 7f094a418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.252+0000 7f094a418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f094a418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af7d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f094a418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55732aed63e0 con 0x55732af704b0 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f094a418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55732ae7dd80 con 0x55732af7ad70 2024-09-05T12:11:09.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f094a418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55732aed23a0 con 0x55732af6fb90 2024-09-05T12:11:09.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58190/0 (socket says 172.21.15.59:58190) 2024-09-05T12:11:09.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0944e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 -- 172.21.15.59:0/98099058 learned_addr learned my addr 172.21.15.59:0/98099058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:09.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0937e00640 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af7d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 -- 172.21.15.59:0/98099058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 msgr2=0x55732af7d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af7d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 -- 172.21.15.59:0/98099058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 msgr2=0x55732af7a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af7a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.253+0000 7f0945800640 1 -- 172.21.15.59:0/98099058 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55732ae9cae0 con 0x55732af6fb90 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.254+0000 7f0945800640 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0938009f30 tx=0x7f09380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24484140f647c84e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.254+0000 7f0936000640 1 -- 172.21.15.59:0/98099058 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938004d20 con 0x55732af6fb90 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.254+0000 7f0936000640 1 -- 172.21.15.59:0/98099058 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0938004050 con 0x55732af6fb90 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.254+0000 7f0936000640 1 -- 172.21.15.59:0/98099058 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938005260 con 0x55732af6fb90 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.254+0000 7f093e000640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- 172.21.15.59:0/98099058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 msgr2=0x55732af6ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0938009f30 tx=0x7f09380118d0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f093e000640 1 -- 172.21.15.59:0/3255346075 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f09300cf890 con 0x7f0940076720 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f0944e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- 172.21.15.59:0/98099058 shutdown_connections 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af7d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af7a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 --2- 172.21.15.59:0/98099058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af6fb90 0x55732af6ff70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- 172.21.15.59:0/98099058 >> 172.21.15.59:0/98099058 conn(0x55732af6c120 msgr2=0x55732af6c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- 172.21.15.59:0/98099058 shutdown_connections 2024-09-05T12:11:09.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- 172.21.15.59:0/98099058 wait complete. 2024-09-05T12:11:09.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f0944e00640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 0x7f09240a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f09300ccf20 tx=0x7f09300d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.255+0000 7f094a418740 1 -- start start 2024-09-05T12:11:09.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 Processor -- start 2024-09-05T12:11:09.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af78210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af78750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55732af7d9e0 con 0x55732af6fb90 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55732ae910f0 con 0x55732af7ad70 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55732af7db60 con 0x55732af704b0 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f094a418740 10 client.0 _ll_register_callbacks cb 0x55732ae829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f0945800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f0945800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44040/0 (socket says 172.21.15.59:44040) 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.256+0000 7f0945800640 1 -- 172.21.15.59:0/1990272254 learned_addr learned my addr 172.21.15.59:0/1990272254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0937e00640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af78210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af78750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 -- 172.21.15.59:0/1990272254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 msgr2=0x55732af78210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af78210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 -- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 msgr2=0x55732af78750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af78750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 -- 172.21.15.59:0/1990272254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55732ac473c0 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f0945800640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0938009d40 tx=0x7f0938004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b62f957f04f55e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.257+0000 7f092ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938004c50 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0938004df0 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0938004050 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0938035050 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 10 client.17538 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55732af89710 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0938004050 con 0x55732af6fb90 2024-09-05T12:11:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f093800f5a0 con 0x55732af6fb90 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 10 client.17538 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 20 client.17538 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.258+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732af8ab10 0x55732af8cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.259+0000 7f094a418740 4 client.17538 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.259+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55732aee3950 con 0x55732af8ab10 2024-09-05T12:11:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.259+0000 7f0937e00640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732af8ab10 0x55732af8cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.259+0000 7f0937e00640 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732af8ab10 0x55732af8cf00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca6bd4f3b13dd87 server_cookie=2aa0bc6f12e3f41c in_seq=0 out_seq=0 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.259+0000 7f092ea00640 10 client.17538 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.260+0000 7f092ea00640 1 -- 172.21.15.59:0/1990272254 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55732aee3950 con 0x55732af8ab10 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.260+0000 7f092ea00640 10 client.17538 handle_command_reply: tid=0 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.260+0000 7f094a418740 1 client.17538 shutdown 2024-09-05T12:11:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.260+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 msgr2=0x55732af77cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.260+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0938009d40 tx=0x7f0938004720 comp rx=0 tx=0).stop 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 shutdown_connections 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af7ad70 0x55732af78750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af78210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732af8ab10 0x55732af8cf00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 --2- 172.21.15.59:0/1990272254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af6fb90 0x55732af77cd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 >> 172.21.15.59:0/1990272254 conn(0x55732af6c120 msgr2=0x55732af71e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f092ea00640 0 client.17538 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 shutdown_connections 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 1 -- 172.21.15.59:0/1990272254 wait complete. 2024-09-05T12:11:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.261+0000 7f094a418740 20 client.17538 trim_cache size 0 max 16384 2024-09-05T12:11:09.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 Processor -- start 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 -- start start 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af92620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af92b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 0x55732af79530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b3500 con 0x55732af704b0 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b5d80 con 0x55732af6fb90 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.279+0000 7f094a418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b3b70 con 0x55732af7ad70 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af92b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0945800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af92620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0945800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af92620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53502/0 (socket says 172.21.15.59:53502) 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0945800640 1 -- 172.21.15.59:0/2493294016 learned_addr learned my addr 172.21.15.59:0/2493294016 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0945800640 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 0x55732af79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 -- 172.21.15.59:0/2493294016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 msgr2=0x55732af79530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 0x55732af79530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 -- 172.21.15.59:0/2493294016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 msgr2=0x55732af92620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af92620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 -- 172.21.15.59:0/2493294016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55732af7a640 con 0x55732af704b0 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0945800640 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 0x55732af79530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.280+0000 7f0937e00640 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af92b60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0928005940 tx=0x7f09280318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec1a6591c1a53fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f0935600640 1 -- 172.21.15.59:0/2493294016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928035040 con 0x55732af704b0 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f0935600640 1 -- 172.21.15.59:0/2493294016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0928005020 con 0x55732af704b0 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f0935600640 1 -- 172.21.15.59:0/2493294016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928004a40 con 0x55732af704b0 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 -- 172.21.15.59:0/2493294016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 msgr2=0x55732af92b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af92b60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0928005940 tx=0x7f09280318a0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 -- 172.21.15.59:0/2493294016 shutdown_connections 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af7ad70 0x55732af79530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af704b0 0x55732af92b60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 --2- 172.21.15.59:0/2493294016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af92620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 -- 172.21.15.59:0/2493294016 >> 172.21.15.59:0/2493294016 conn(0x55732af6c120 msgr2=0x55732af76dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.281+0000 7f094a418740 1 -- 172.21.15.59:0/2493294016 shutdown_connections 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.282+0000 7f094a418740 1 -- 172.21.15.59:0/2493294016 wait complete. 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.282+0000 7f094a418740 1 -- start start 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.282+0000 7f094a418740 1 Processor -- start 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af7e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af7ad70 0x55732aff35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b4240 con 0x55732af7ad70 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b56b0 con 0x55732af6fb90 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b4910 con 0x55732af704b0 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f094a418740 10 client.0 _ll_register_callbacks cb 0x7f0938009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:58214/0 (socket says 172.21.15.59:58214) 2024-09-05T12:11:09.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 -- 172.21.15.59:0/2666714607 learned_addr learned my addr 172.21.15.59:0/2666714607 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0945800640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af7e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 -- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 msgr2=0x55732af7e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af7e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 -- 172.21.15.59:0/2666714607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af7ad70 msgr2=0x55732aff35e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.283+0000 7f0937e00640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af7ad70 0x55732aff35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f0937e00640 1 -- 172.21.15.59:0/2666714607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55732aff3c20 con 0x55732af704b0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f0937e00640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0928031870 tx=0x7f0928031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374b77b11a10e82e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f092ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928056020 con 0x55732af704b0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0928005020 con 0x55732af704b0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0928004240 con 0x55732af704b0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.284+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0928061050 con 0x55732af704b0 2024-09-05T12:11:09.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 10 client.27466 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:09.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55732aff4ef0 con 0x55732af704b0 2024-09-05T12:11:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0928005020 con 0x55732af704b0 2024-09-05T12:11:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f09280035f0 con 0x55732af704b0 2024-09-05T12:11:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 10 client.27466 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 20 client.27466 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732aff5640 0x55732aff7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 4 client.27466 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.285+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55732aedde60 con 0x55732aff5640 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f0945800640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732aff5640 0x55732aff7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f0945800640 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732aff5640 0x55732aff7a60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875a14a9389eca3b server_cookie=7520b39e975fc7b4 in_seq=0 out_seq=0 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f092ea00640 10 client.27466 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f092ea00640 1 -- 172.21.15.59:0/2666714607 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55732aedde60 con 0x55732aff5640 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f092ea00640 10 client.27466 handle_command_reply: tid=0 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f094a418740 1 client.27466 shutdown 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 msgr2=0x55732af7ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.286+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0928031870 tx=0x7f0928031630 comp rx=0 tx=0).stop 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 shutdown_connections 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55732aff5640 0x55732aff7a60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55732af7ad70 0x55732aff35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55732af704b0 0x55732af7ea50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 --2- 172.21.15.59:0/2666714607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55732af6fb90 0x55732af7e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 >> 172.21.15.59:0/2666714607 conn(0x55732af6c120 msgr2=0x55732af6a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f092ea00640 0 client.27466 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 shutdown_connections 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 1 -- 172.21.15.59:0/2666714607 wait complete. 2024-09-05T12:11:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.287+0000 7f094a418740 20 client.27466 trim_cache size 0 max 16384 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "f496e932-fdd6-4431-b3cb-996368b32a4d": { 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "f496e932-fdd6-4431-b3cb-996368b32a4d", 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:09.293 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 msgr2=0x7f09240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 0x7f09240a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f09300ccf20 tx=0x7f09300d5530 comp rx=0 tx=0).stop 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 msgr2=0x7f09401a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9f0 tx=0x7f093000edf0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 shutdown_connections 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f09240a2b80 0x7f09240a5040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0940112a50 0x7f094019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0940077040 0x7f09401a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 --2- 172.21.15.59:0/3255346075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0940076720 0x7f09401a47f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 >> 172.21.15.59:0/3255346075 conn(0x7f09400ffa50 msgr2=0x7f0940114810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 shutdown_connections 2024-09-05T12:11:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:09.288+0000 7f0947800640 1 -- 172.21.15.59:0/3255346075 wait complete. 2024-09-05T12:11:09.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:08 smithi189 ceph-mon[34772]: pgmap v798: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:09.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:08 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:11:09.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:08 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:09.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:08 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:11:09.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:08 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:10.151 INFO:tasks.cephfs.filesystem:scrub status for tag:f496e932-fdd6-4431-b3cb-996368b32a4d - {'path': '/', 'tag': 'f496e932-fdd6-4431-b3cb-996368b32a4d', 'options': 'recursive,force'} 2024-09-05T12:11:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:10 smithi089 ceph-mon[34499]: pgmap v799: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:10 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:11:11.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:10 smithi059 ceph-mon[30025]: pgmap v799: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:10 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:11:11.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:10 smithi189 ceph-mon[34772]: pgmap v799: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:11.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:10 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:11:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:12 smithi089 ceph-mon[34499]: pgmap v800: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:13.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:12 smithi059 ceph-mon[30025]: pgmap v800: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:13.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:12 smithi189 ceph-mon[34772]: pgmap v800: 129 pgs: 129 active+clean; 6.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:14 smithi089 ceph-mon[34499]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:11:15.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:14 smithi059 ceph-mon[30025]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:11:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:14 smithi189 ceph-mon[34772]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-05T12:11:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:16 smithi089 ceph-mon[34499]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:17.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:16 smithi059 ceph-mon[30025]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:17.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:16 smithi189 ceph-mon[34772]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:18 smithi089 ceph-mon[34499]: pgmap v803: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:19.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:18 smithi059 ceph-mon[30025]: pgmap v803: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:19.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:18 smithi189 ceph-mon[34772]: pgmap v803: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:20 smithi089 ceph-mon[34499]: pgmap v804: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:11:21.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:20 smithi059 ceph-mon[30025]: pgmap v804: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:11:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:20 smithi189 ceph-mon[34772]: pgmap v804: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:11:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:22 smithi089 ceph-mon[34499]: pgmap v805: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:23.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:22 smithi059 ceph-mon[30025]: pgmap v805: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:23.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:22 smithi189 ceph-mon[34772]: pgmap v805: 129 pgs: 129 active+clean; 6.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:24.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:24 smithi059 ceph-mon[30025]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:24 smithi089 ceph-mon[34499]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:25.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:24 smithi189 ceph-mon[34772]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:26 smithi089 ceph-mon[34499]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:27.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:26 smithi059 ceph-mon[30025]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:27.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:26 smithi189 ceph-mon[34772]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:28 smithi089 ceph-mon[34499]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:29.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:28 smithi059 ceph-mon[30025]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:29.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:28 smithi189 ceph-mon[34772]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:31.140 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:30 smithi189 ceph-mon[34772]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:30 smithi089 ceph-mon[34499]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:30 smithi059 ceph-mon[30025]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:32 smithi089 ceph-mon[34499]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:32 smithi059 ceph-mon[30025]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:33.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:32 smithi189 ceph-mon[34772]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:34 smithi089 ceph-mon[34499]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:35.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:34 smithi059 ceph-mon[30025]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:34 smithi189 ceph-mon[34772]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-09-05T12:11:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:36 smithi089 ceph-mon[34499]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:37.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:36 smithi059 ceph-mon[30025]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:36 smithi189 ceph-mon[34772]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:38 smithi089 ceph-mon[34499]: pgmap v813: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:39.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:38 smithi059 ceph-mon[30025]: pgmap v813: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:38 smithi189 ceph-mon[34772]: pgmap v813: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:11:40.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:11:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:40 smithi059 ceph-mon[30025]: pgmap v814: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:41.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 -- 172.21.15.59:0/3404210975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 msgr2=0x7f7c4c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 --2- 172.21.15.59:0/3404210975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c110cc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009ca0 tx=0x7f7c4402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 -- 172.21.15.59:0/3404210975 shutdown_connections 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 --2- 172.21.15.59:0/3404210975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c110cc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 --2- 172.21.15.59:0/3404210975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 --2- 172.21.15.59:0/3404210975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c104010 0x7f7c4c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 -- 172.21.15.59:0/3404210975 >> 172.21.15.59:0/3404210975 conn(0x7f7c4c0ffa50 msgr2=0x7f7c4c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 -- 172.21.15.59:0/3404210975 shutdown_connections 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.098+0000 7f7c53200640 1 -- 172.21.15.59:0/3404210975 wait complete. 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.099+0000 7f7c53200640 1 Processor -- start 2024-09-05T12:11:41.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.099+0000 7f7c53200640 1 -- start start 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.099+0000 7f7c53200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c104010 0x7f7c4c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.099+0000 7f7c53200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c53200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c53200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1141a0 con 0x7f7c4c109140 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c53200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c114020 con 0x7f7c4c1049c0 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c53200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c114320 con 0x7f7c4c104010 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55088/0 (socket says 172.21.15.59:55088) 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c51200640 1 -- 172.21.15.59:0/157909981 learned_addr learned my addr 172.21.15.59:0/157909981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c4be00640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c104010 0x7f7c4c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 -- 172.21.15.59:0/157909981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c104010 msgr2=0x7f7c4c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c104010 0x7f7c4c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 -- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 msgr2=0x7f7c4c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c43400640 1 -- 172.21.15.59:0/157909981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c19f650 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.100+0000 7f7c51200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c43400640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c19abe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c34005fd0 tx=0x7f7c3400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c49600640 1 -- 172.21.15.59:0/157909981 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34009c30 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c49600640 1 -- 172.21.15.59:0/157909981 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34009dd0 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c49600640 1 -- 172.21.15.59:0/157909981 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010460 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c19f940 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.101+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a7220 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.104+0000 7f7c55c12740 1 Processor -- start 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.104+0000 7f7c49600640 1 -- 172.21.15.59:0/157909981 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c34003ef0 con 0x7f7c4c1049c0 2024-09-05T12:11:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.104+0000 7f7c55c12740 1 -- start start 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56315942a3e0 con 0x5631594ced70 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631593d1d80 con 0x5631594c44b0 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c55c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631594263a0 con 0x5631594c3b90 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c51200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c51200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57272/0 (socket says 172.21.15.59:57272) 2024-09-05T12:11:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c51200640 1 -- 172.21.15.59:0/2022796262 learned_addr learned my addr 172.21.15.59:0/2022796262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.105+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58334/0 (socket says 172.21.15.59:58334) 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 -- 172.21.15.59:0/2022796262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 msgr2=0x5631594c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 -- 172.21.15.59:0/2022796262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 msgr2=0x5631594d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 -- 172.21.15.59:0/2022796262 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631593f0ae0 con 0x5631594c44b0 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c49600640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 0x7f7c300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c4be00640 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00bbf0 tx=0x7f7c3c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae3b9d64deddf55 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c49600640 1 -- 172.21.15.59:0/157909981 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7c340cee80 con 0x7f7c4c1049c0 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c51200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 0x7f7c300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c41600640 1 -- 172.21.15.59:0/2022796262 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004030 con 0x5631594c44b0 2024-09-05T12:11:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c41600640 1 -- 172.21.15.59:0/2022796262 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c0041d0 con 0x5631594c44b0 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.106+0000 7f7c41600640 1 -- 172.21.15.59:0/2022796262 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0116c0 con 0x5631594c44b0 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c51200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 0x7f7c300a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009a60 tx=0x7f7c44005d80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 -- 172.21.15.59:0/2022796262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 msgr2=0x5631594ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00bbf0 tx=0x7f7c3c0107a0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 -- 172.21.15.59:0/2022796262 shutdown_connections 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594ce830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 --2- 172.21.15.59:0/2022796262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 -- 172.21.15.59:0/2022796262 >> 172.21.15.59:0/2022796262 conn(0x5631594c0120 msgr2=0x5631594c0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 -- 172.21.15.59:0/2022796262 shutdown_connections 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.107+0000 7f7c55c12740 1 -- 172.21.15.59:0/2022796262 wait complete. 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 -- start start 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 Processor -- start 2024-09-05T12:11:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5631594d19e0 con 0x5631594ced70 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631593e50f0 con 0x5631594c44b0 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631594d1b60 con 0x5631594c3b90 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.108+0000 7f7c55c12740 10 client.0 _ll_register_callbacks cb 0x5631593d69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c43400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55118/0 (socket says 172.21.15.59:55118) 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57274/0 (socket says 172.21.15.59:57274) 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c51200640 1 -- 172.21.15.59:0/1340098073 learned_addr learned my addr 172.21.15.59:0/1340098073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 -- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 msgr2=0x5631594cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 -- 172.21.15.59:0/1340098073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 msgr2=0x5631594cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 -- 172.21.15.59:0/1340098073 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315919b3c0 con 0x5631594c3b90 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c51200640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c4be00640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011010 tx=0x7f7c3c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d980b0afb9a09278 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.109+0000 7f7c3a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c005040 con 0x5631594c3b90 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3c0041d0 con 0x5631594c3b90 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7c3c0051e0 con 0x5631594c3b90 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7c3c02e050 con 0x5631594c3b90 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c55c12740 10 client.27478 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5631594dd750 con 0x5631594c3b90 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.110+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c0041d0 con 0x5631594c3b90 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7c3c004750 con 0x5631594c3b90 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 10 client.27478 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 10 client.27478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 20 client.27478 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594deaa0 0x5631594e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 4 client.27478 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563159437950 con 0x5631594deaa0 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.111+0000 7f7c43400640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594deaa0 0x5631594e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.112+0000 7f7c43400640 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594deaa0 0x5631594e0e90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55a98cca8648e02 server_cookie=ff3b2c988ef1072b in_seq=0 out_seq=0 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.112+0000 7f7c3a000640 10 client.27478 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.112+0000 7f7c3a000640 1 -- 172.21.15.59:0/1340098073 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563159437950 con 0x5631594deaa0 2024-09-05T12:11:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.112+0000 7f7c3a000640 10 client.27478 handle_command_reply: tid=0 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.112+0000 7f7c55c12740 1 client.27478 shutdown 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 msgr2=0x5631594cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011010 tx=0x7f7c3c00bfd0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 shutdown_connections 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594deaa0 0x5631594e0e90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594ced70 0x5631594cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 --2- 172.21.15.59:0/1340098073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c3b90 0x5631594cbd20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 >> 172.21.15.59:0/1340098073 conn(0x5631594c0120 msgr2=0x5631594c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c3a000640 0 client.27478 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 shutdown_connections 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 1 -- 172.21.15.59:0/1340098073 wait complete. 2024-09-05T12:11:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.113+0000 7f7c55c12740 20 client.27478 trim_cache size 0 max 16384 2024-09-05T12:11:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.131+0000 7f7c55c12740 1 Processor -- start 2024-09-05T12:11:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.131+0000 7f7c55c12740 1 -- start start 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1af1d0 con 0x5631594c3b90 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1b1a50 con 0x5631594c44b0 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c55c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1af840 con 0x5631594c0120 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57290/0 (socket says 172.21.15.59:57290) 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55128/0 (socket says 172.21.15.59:55128) 2024-09-05T12:11:41.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c4be00640 1 -- 172.21.15.59:0/3187318473 learned_addr learned my addr 172.21.15.59:0/3187318473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c51200640 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 -- 172.21.15.59:0/3187318473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 msgr2=0x5631594c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 -- 172.21.15.59:0/3187318473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 msgr2=0x5631594c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.132+0000 7f7c43400640 1 -- 172.21.15.59:0/3187318473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631594c42e0 con 0x5631594c3b90 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.133+0000 7f7c43400640 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x5631594c14a0 tx=0x7f7c340d28c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818b5d0342e3bc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.133+0000 7f7c40c00640 1 -- 172.21.15.59:0/3187318473 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34020040 con 0x5631594c3b90 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.133+0000 7f7c40c00640 1 -- 172.21.15.59:0/3187318473 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3409b320 con 0x5631594c3b90 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.133+0000 7f7c40c00640 1 -- 172.21.15.59:0/3187318473 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34097990 con 0x5631594c3b90 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- 172.21.15.59:0/3187318473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 msgr2=0x5631594c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x5631594c14a0 tx=0x7f7c340d28c0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- 172.21.15.59:0/3187318473 shutdown_connections 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c44b0 0x5631594c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c3b90 0x5631594c8f80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 --2- 172.21.15.59:0/3187318473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- 172.21.15.59:0/3187318473 >> 172.21.15.59:0/3187318473 conn(0x5631594cc900 msgr2=0x5631594c55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- 172.21.15.59:0/3187318473 shutdown_connections 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- 172.21.15.59:0/3187318473 wait complete. 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 -- start start 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.134+0000 7f7c55c12740 1 Processor -- start 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c3b90 0x5631594e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1aff10 con 0x5631594c44b0 2024-09-05T12:11:41.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1b1380 con 0x5631594c3b90 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1b05e0 con 0x5631594c0120 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c55c12740 10 client.0 _ll_register_callbacks cb 0x7f7c3c00ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c51200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55130/0 (socket says 172.21.15.59:55130) 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c51200640 1 -- 172.21.15.59:0/323994731 learned_addr learned my addr 172.21.15.59:0/323994731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c4be00640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.135+0000 7f7c43400640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c3b90 0x5631594e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 -- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c3b90 msgr2=0x5631594e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c3b90 0x5631594e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 -- 172.21.15.59:0/323994731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 msgr2=0x5631595483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 -- 172.21.15.59:0/323994731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563159548920 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c51200640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c4be00640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594e2010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c004b50 tx=0x7f7c3c004df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aae2bb6f6b7f6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.136+0000 7f7c3a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c037060 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7c3c0040c0 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3409b320 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7c3c03e050 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c55c12740 10 client.27484 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.137+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563159549ab0 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3409b320 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7c3c003af0 con 0x5631594c0120 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 10 client.27484 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 10 client.27484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:41.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 20 client.27484 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594ced70 0x56315954c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 4 client.27484 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563159431e60 con 0x5631594ced70 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.138+0000 7f7c43400640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594ced70 0x56315954c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c43400640 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594ced70 0x56315954c0f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4d76e1f23bbaaa server_cookie=314b71135aac6000 in_seq=0 out_seq=0 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c3a000640 10 client.27484 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c3a000640 1 -- 172.21.15.59:0/323994731 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563159431e60 con 0x5631594ced70 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c3a000640 10 client.27484 handle_command_reply: tid=0 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 client.27484 shutdown 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 msgr2=0x5631594e2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594e2010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c004b50 tx=0x7f7c3c004df0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 shutdown_connections 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5631594ced70 0x56315954c0f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5631594c44b0 0x5631595483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631594c3b90 0x5631594e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 --2- 172.21.15.59:0/323994731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631594c0120 0x5631594e2010 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c3a000640 0 client.27484 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 >> 172.21.15.59:0/323994731 conn(0x5631594cc900 msgr2=0x5631594da5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.139+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 shutdown_connections 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c55c12740 1 -- 172.21.15.59:0/323994731 wait complete. 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c55c12740 20 client.27484 trim_cache size 0 max 16384 2024-09-05T12:11:41.147 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:41.148 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:11:41.148 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:11:41.148 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 msgr2=0x7f7c300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 0x7f7c300a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009a60 tx=0x7f7c44005d80 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 msgr2=0x7f7c4c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c19abe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c34005fd0 tx=0x7f7c3400dde0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 shutdown_connections 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7c300a2c00 0x7f7c300a50c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c4c109140 0x7f7c4c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c4c1049c0 0x7f7c4c19abe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 --2- 172.21.15.59:0/157909981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c4c104010 0x7f7c4c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.140+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 >> 172.21.15.59:0/157909981 conn(0x7f7c4c0ffa50 msgr2=0x7f7c4c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.141+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 shutdown_connections 2024-09-05T12:11:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:41.141+0000 7f7c53200640 1 -- 172.21.15.59:0/157909981 wait complete. 2024-09-05T12:11:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:40 smithi089 ceph-mon[34499]: pgmap v814: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:41.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:40 smithi189 ceph-mon[34772]: pgmap v814: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:42.003 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:11:42.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:11:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 -- 172.21.15.59:0/4098168921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580077040 msgr2=0x7f35801124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 --2- 172.21.15.59:0/4098168921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580077040 0x7f35801124f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f356c009900 tx=0x7f356c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 -- 172.21.15.59:0/4098168921 shutdown_connections 2024-09-05T12:11:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 --2- 172.21.15.59:0/4098168921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580112a30 0x7f3580114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 --2- 172.21.15.59:0/4098168921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580077040 0x7f35801124f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 --2- 172.21.15.59:0/4098168921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580076720 0x7f3580076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 -- 172.21.15.59:0/4098168921 >> 172.21.15.59:0/4098168921 conn(0x7f35800ffa90 msgr2=0x7f3580101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 -- 172.21.15.59:0/4098168921 shutdown_connections 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.904+0000 7f3586a00640 1 -- 172.21.15.59:0/4098168921 wait complete. 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.905+0000 7f3586a00640 1 Processor -- start 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.905+0000 7f3586a00640 1 -- start start 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580077040 0x7f358019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580112a30 0x7f35801a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35801185b0 con 0x7f3580076720 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3580118430 con 0x7f3580112a30 2024-09-05T12:11:42.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3586a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3580118730 con 0x7f3580077040 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55160/0 (socket says 172.21.15.59:55160) 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 -- 172.21.15.59:0/4020500678 learned_addr learned my addr 172.21.15.59:0/4020500678 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357f400640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580077040 0x7f358019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f3584a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580112a30 0x7f35801a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 -- 172.21.15.59:0/4020500678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580077040 msgr2=0x7f358019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580077040 0x7f358019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 -- 172.21.15.59:0/4020500678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580112a30 msgr2=0x7f35801a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580112a30 0x7f35801a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.906+0000 7f357fe00640 1 -- 172.21.15.59:0/4020500678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35801a3ba0 con 0x7f3580076720 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f357fe00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9f0 tx=0x7f357000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357000cd80 con 0x7f3580076720 2024-09-05T12:11:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570010910 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801a3e90 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570018990 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.907+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801ab720 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.908+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580076b00 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.910+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3570010ac0 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.911+0000 7f357cc00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 0x7f35640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.912+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f35700cf870 con 0x7f3580076720 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.912+0000 7f357f400640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 0x7f35640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.913+0000 7f357f400640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 0x7f35640a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f356c009230 tx=0x7f356c0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:42.916+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f357009c360 con 0x7f3580076720 2024-09-05T12:11:43.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.138+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f358010dbc0 con 0x7f3580076720 2024-09-05T12:11:43.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:42 smithi059 ceph-mon[30025]: pgmap v815: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:43.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.139+0000 7f357cc00640 1 -- 172.21.15.59:0/4020500678 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f357009bc60 con 0x7f3580076720 2024-09-05T12:11:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:11:43.141 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:11:43.142 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:11:43.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 msgr2=0x7f35640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 0x7f35640a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f356c009230 tx=0x7f356c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 msgr2=0x7f358019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9f0 tx=0x7f357000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 shutdown_connections 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f35640a2b80 0x7f35640a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3580112a30 0x7f35801a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3580077040 0x7f358019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 --2- 172.21.15.59:0/4020500678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3580076720 0x7f358019eb70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 >> 172.21.15.59:0/4020500678 conn(0x7f35800ffa90 msgr2=0x7f3580101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 shutdown_connections 2024-09-05T12:11:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:43.142+0000 7f3586a00640 1 -- 172.21.15.59:0/4020500678 wait complete. 2024-09-05T12:11:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:42 smithi089 ceph-mon[34499]: pgmap v815: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:42 smithi189 ceph-mon[34772]: pgmap v815: 129 pgs: 129 active+clean; 7.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:44.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:11:44.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:43 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4020500678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:44.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:43 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4020500678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:44.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:43 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4020500678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:44.397 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 -- 172.21.15.59:0/607055637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c077040 msgr2=0x7f6f3c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 --2- 172.21.15.59:0/607055637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c077040 0x7f6f3c112330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 -- 172.21.15.59:0/607055637 shutdown_connections 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 --2- 172.21.15.59:0/607055637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c112870 0x7f6f3c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 --2- 172.21.15.59:0/607055637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c077040 0x7f6f3c112330 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 --2- 172.21.15.59:0/607055637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c076720 0x7f6f3c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 -- 172.21.15.59:0/607055637 >> 172.21.15.59:0/607055637 conn(0x7f6f3c0ff910 msgr2=0x7f6f3c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 -- 172.21.15.59:0/607055637 shutdown_connections 2024-09-05T12:11:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.992+0000 7f6f41200640 1 -- 172.21.15.59:0/607055637 wait complete. 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 Processor -- start 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 -- start start 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c077040 0x7f6f3c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c112870 0x7f6f3c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c118450 con 0x7f6f3c077040 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1182d0 con 0x7f6f3c112870 2024-09-05T12:11:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.994+0000 7f6f41200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1185d0 con 0x7f6f3c076720 2024-09-05T12:11:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57350/0 (socket says 172.21.15.59:57350) 2024-09-05T12:11:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 -- 172.21.15.59:0/1448446574 learned_addr learned my addr 172.21.15.59:0/1448446574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c077040 0x7f6f3c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3be00640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c112870 0x7f6f3c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 -- 172.21.15.59:0/1448446574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c112870 msgr2=0x7f6f3c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c112870 0x7f6f3c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 -- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c077040 msgr2=0x7f6f3c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c077040 0x7f6f3c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.995+0000 7f6f3b400640 1 -- 172.21.15.59:0/1448446574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c19ef60 con 0x7f6f3c076720 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f3b400640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f33e00640 1 -- 172.21.15.59:0/1448446574 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00cd80 con 0x7f6f3c076720 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c19f250 con 0x7f6f3c076720 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c1ab4a0 con 0x7f6f3c076720 2024-09-05T12:11:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f33e00640 1 -- 172.21.15.59:0/1448446574 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c005500 con 0x7f6f3c076720 2024-09-05T12:11:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.996+0000 7f6f33e00640 1 -- 172.21.15.59:0/1448446574 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c010690 con 0x7f6f3c076720 2024-09-05T12:11:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.999+0000 7f6f451ef740 1 Processor -- start 2024-09-05T12:11:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.999+0000 7f6f33e00640 1 -- 172.21.15.59:0/1448446574 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f2c0046c0 con 0x7f6f3c076720 2024-09-05T12:11:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:44.999+0000 7f6f451ef740 1 -- start start 2024-09-05T12:11:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa4070c3e0 con 0x55fa407b1020 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fa406b3d80 con 0x55fa407a5e40 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f451ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa407083a0 con 0x55fa407a6760 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58398/0 (socket says 172.21.15.59:58398) 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 -- 172.21.15.59:0/885408428 learned_addr learned my addr 172.21.15.59:0/885408428 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3b400640 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 -- 172.21.15.59:0/885408428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 msgr2=0x55fa407b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 -- 172.21.15.59:0/885408428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 msgr2=0x55fa407b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.000+0000 7f6f3be00640 1 -- 172.21.15.59:0/885408428 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa406d2ae0 con 0x55fa407a5e40 2024-09-05T12:11:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.001+0000 7f6f3be00640 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009f30 tx=0x7f6f340118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de612afe011d732 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.001+0000 7f6f30c00640 1 -- 172.21.15.59:0/885408428 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34004d20 con 0x55fa407a5e40 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.001+0000 7f6f30c00640 1 -- 172.21.15.59:0/885408428 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34004050 con 0x55fa407a5e40 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.001+0000 7f6f30c00640 1 -- 172.21.15.59:0/885408428 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34005260 con 0x55fa407a5e40 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.001+0000 7f6f33e00640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 0x7f6f200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f33e00640 1 -- 172.21.15.59:0/1448446574 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f2c0cf670 con 0x7f6f3c076720 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f3b400640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 0x7f6f200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 -- 172.21.15.59:0/885408428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 msgr2=0x55fa407a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009f30 tx=0x7f6f340118d0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 -- 172.21.15.59:0/885408428 shutdown_connections 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 --2- 172.21.15.59:0/885408428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407a6220 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 -- 172.21.15.59:0/885408428 >> 172.21.15.59:0/885408428 conn(0x55fa407a23d0 msgr2=0x55fa407a27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 -- 172.21.15.59:0/885408428 shutdown_connections 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f451ef740 1 -- 172.21.15.59:0/885408428 wait complete. 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.002+0000 7f6f3b400640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 0x7f6f200a50c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c09b7c0 tx=0x7f6f2c0d5550 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 -- start start 2024-09-05T12:11:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 Processor -- start 2024-09-05T12:11:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 0x55fa407adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa407b3bf0 con 0x55fa407b1020 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fa406c70f0 con 0x55fa407a6760 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.003+0000 7f6f451ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa407b3d70 con 0x55fa407a5e40 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f451ef740 10 client.0 _ll_register_callbacks cb 0x55fa406aebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 0x55fa407adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58404/0 (socket says 172.21.15.59:58404) 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 -- 172.21.15.59:0/2532007876 learned_addr learned my addr 172.21.15.59:0/2532007876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 -- 172.21.15.59:0/2532007876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 msgr2=0x55fa407adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 0x55fa407adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 -- 172.21.15.59:0/2532007876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 msgr2=0x55fa407ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3aa00640 1 -- 172.21.15.59:0/2532007876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4047d3c0 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.004+0000 7f6f3be00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 0x55fa407adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f28033000 tx=0x7f6f28004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aefec34ff39925d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f25600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035040 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34004050 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f28005450 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f2804f050 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f451ef740 10 client.27578 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.005+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fa407bfcf0 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34004050 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6f280318b0 con 0x55fa407a6760 2024-09-05T12:11:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 10 client.27578 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 10 client.27578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 20 client.27578 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa407c1250 0x55fa407c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 4 client.27578 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa40719950 con 0x55fa407c1250 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.006+0000 7f6f3be00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa407c1250 0x55fa407c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.007+0000 7f6f3be00640 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa407c1250 0x55fa407c3640 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc57d4d783512ee3 server_cookie=63d371998e421183 in_seq=0 out_seq=0 2024-09-05T12:11:45.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.007+0000 7f6f25600640 10 client.27578 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.007+0000 7f6f25600640 1 -- 172.21.15.59:0/2532007876 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa40719950 con 0x55fa407c1250 2024-09-05T12:11:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.007+0000 7f6f25600640 10 client.27578 handle_command_reply: tid=0 2024-09-05T12:11:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 client.27578 shutdown 2024-09-05T12:11:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 msgr2=0x55fa407ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f28033000 tx=0x7f6f28004390 comp rx=0 tx=0).stop 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 shutdown_connections 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa407c1250 0x55fa407c3640 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a6760 0x55fa407ae470 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 --2- 172.21.15.59:0/2532007876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a5e40 0x55fa407adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 >> 172.21.15.59:0/2532007876 conn(0x55fa407a23d0 msgr2=0x55fa407a80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f25600640 0 client.27578 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 shutdown_connections 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 1 -- 172.21.15.59:0/2532007876 wait complete. 2024-09-05T12:11:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.008+0000 7f6f451ef740 20 client.27578 trim_cache size 0 max 16384 2024-09-05T12:11:45.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 Processor -- start 2024-09-05T12:11:45.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 -- start start 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00dd80 con 0x55fa407b1020 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c010600 con 0x55fa407a5e40 2024-09-05T12:11:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f451ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00e3f0 con 0x55fa407a6760 2024-09-05T12:11:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55194/0 (socket says 172.21.15.59:55194) 2024-09-05T12:11:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.022+0000 7f6f3be00640 1 -- 172.21.15.59:0/3604606663 learned_addr learned my addr 172.21.15.59:0/3604606663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 -- 172.21.15.59:0/3604606663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 msgr2=0x55fa407c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 -- 172.21.15.59:0/3604606663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 msgr2=0x55fa407c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 -- 172.21.15.59:0/3604606663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa407acee0 con 0x55fa407b1020 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f3be00640 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401b040 tx=0x7f6f34003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862dee67ca483598 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f27e00640 1 -- 172.21.15.59:0/3604606663 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34004dc0 con 0x55fa407b1020 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.023+0000 7f6f27e00640 1 -- 172.21.15.59:0/3604606663 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34003d60 con 0x55fa407b1020 2024-09-05T12:11:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f27e00640 1 -- 172.21.15.59:0/3604606663 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340053f0 con 0x55fa407b1020 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 -- 172.21.15.59:0/3604606663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 msgr2=0x55fa407ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401b040 tx=0x7f6f34003a30 comp rx=0 tx=0).stop 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 -- 172.21.15.59:0/3604606663 shutdown_connections 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407b1020 0x55fa407ab190 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407a6760 0x55fa407c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 --2- 172.21.15.59:0/3604606663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 -- 172.21.15.59:0/3604606663 >> 172.21.15.59:0/3604606663 conn(0x55fa407a23d0 msgr2=0x55fa407a1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 -- 172.21.15.59:0/3604606663 shutdown_connections 2024-09-05T12:11:45.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.024+0000 7f6f451ef740 1 -- 172.21.15.59:0/3604606663 wait complete. 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 -- start start 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 Processor -- start 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407b1020 0x55fa407b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00eac0 con 0x55fa407a6760 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00ff30 con 0x55fa407a5e40 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c00f190 con 0x55fa407b1020 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.025+0000 7f6f451ef740 10 client.0 _ll_register_callbacks cb 0x7f6f28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55200/0 (socket says 172.21.15.59:55200) 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 -- 172.21.15.59:0/1744829376 learned_addr learned my addr 172.21.15.59:0/1744829376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3be00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407b1020 0x55fa407b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 -- 172.21.15.59:0/1744829376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407b1020 msgr2=0x55fa407b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407b1020 0x55fa407b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 -- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 msgr2=0x55fa407b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 -- 172.21.15.59:0/1744829376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4082b100 con 0x55fa407a6760 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3be00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f3aa00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031e20 tx=0x7f6f28005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb924508319e6d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.026+0000 7f6f25600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035030 con 0x55fa407a6760 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6f280317c0 con 0x55fa407a6760 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f34003d60 con 0x55fa407a6760 2024-09-05T12:11:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f28061050 con 0x55fa407a6760 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f451ef740 10 client.17583 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fa4082c3c0 con 0x55fa407a6760 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34003d60 con 0x55fa407a6760 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.027+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6f28003d70 con 0x55fa407a6760 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 10 client.17583 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 20 client.17583 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa4082cb40 0x55fa4082ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 4 client.17583 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa40713e60 con 0x55fa4082cb40 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f3be00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa4082cb40 0x55fa4082ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f3be00640 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa4082cb40 0x55fa4082ef60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfefa5fbbd38d16 server_cookie=f081f184042a24fe in_seq=0 out_seq=0 2024-09-05T12:11:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.028+0000 7f6f25600640 10 client.17583 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f25600640 1 -- 172.21.15.59:0/1744829376 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa40713e60 con 0x55fa4082cb40 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f25600640 10 client.17583 handle_command_reply: tid=0 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 client.17583 shutdown 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 msgr2=0x55fa407b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031e20 tx=0x7f6f28005480 comp rx=0 tx=0).stop 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 shutdown_connections 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa407b1020 0x55fa407b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fa4082cb40 0x55fa4082ef60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa407a6760 0x55fa407b5530 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 --2- 172.21.15.59:0/1744829376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa407a5e40 0x55fa407b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 >> 172.21.15.59:0/1744829376 conn(0x55fa407a23d0 msgr2=0x55fa407a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f25600640 0 client.17583 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:45.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 shutdown_connections 2024-09-05T12:11:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 1 -- 172.21.15.59:0/1744829376 wait complete. 2024-09-05T12:11:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.029+0000 7f6f451ef740 20 client.17583 trim_cache size 0 max 16384 2024-09-05T12:11:45.036 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:11:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 msgr2=0x7f6f200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 0x7f6f200a50c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c09b7c0 tx=0x7f6f2c0d5550 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 msgr2=0x7f6f3c1a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 shutdown_connections 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6f200a2c00 0x7f6f200a50c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f3c112870 0x7f6f3c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f3c077040 0x7f6f3c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 --2- 172.21.15.59:0/1448446574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f3c076720 0x7f6f3c1a4640 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 >> 172.21.15.59:0/1448446574 conn(0x7f6f3c0ff910 msgr2=0x7f6f3c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 shutdown_connections 2024-09-05T12:11:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:45.030+0000 7f6f41200640 1 -- 172.21.15.59:0/1448446574 wait complete. 2024-09-05T12:11:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:44 smithi089 ceph-mon[34499]: pgmap v816: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:11:45.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:44 smithi059 ceph-mon[30025]: pgmap v816: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:11:45.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:44 smithi189 ceph-mon[34772]: pgmap v816: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-09-05T12:11:46.781 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-05T12:11:46.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:11:46.924 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:11:46.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:11:47.183 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:47.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:46 smithi089 ceph-mon[34499]: pgmap v817: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:47.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:46 smithi059 ceph-mon[30025]: pgmap v817: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:47.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:46 smithi189 ceph-mon[34772]: pgmap v817: 129 pgs: 129 active+clean; 7.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:47.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.790+0000 7f53fba00640 1 -- 172.21.15.59:0/3942429889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4109120 msgr2=0x7f53f4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:47.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.790+0000 7f53fba00640 1 --2- 172.21.15.59:0/3942429889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4109120 0x7f53f4110ca0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ca0 tx=0x7f53ec02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 -- 172.21.15.59:0/3942429889 shutdown_connections 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 --2- 172.21.15.59:0/3942429889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4109120 0x7f53f4110ca0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 --2- 172.21.15.59:0/3942429889 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f41049a0 0x7f53f41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 --2- 172.21.15.59:0/3942429889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4103ff0 0x7f53f41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 -- 172.21.15.59:0/3942429889 >> 172.21.15.59:0/3942429889 conn(0x7f53f40ffa90 msgr2=0x7f53f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 -- 172.21.15.59:0/3942429889 shutdown_connections 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.791+0000 7f53fba00640 1 -- 172.21.15.59:0/3942429889 wait complete. 2024-09-05T12:11:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.792+0000 7f53fba00640 1 Processor -- start 2024-09-05T12:11:47.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.792+0000 7f53fba00640 1 -- start start 2024-09-05T12:11:47.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 0x7f53f419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:47.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 0x7f53f419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 0x7f53f419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4114150 con 0x7f53f4103ff0 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4113fd0 con 0x7f53f4109120 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53fba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41142d0 con 0x7f53f41049a0 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53f9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 0x7f53f419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 0x7f53f419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53f9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 0x7f53f419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55216/0 (socket says 172.21.15.59:55216) 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53f9000640 1 -- 172.21.15.59:0/1108077394 learned_addr learned my addr 172.21.15.59:0/1108077394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53f9a00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 0x7f53f419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 -- 172.21.15.59:0/1108077394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 msgr2=0x7f53f419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 0x7f53f419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 -- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 msgr2=0x7f53f419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 0x7f53f419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.793+0000 7f53ebe00640 1 -- 172.21.15.59:0/1108077394 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f419f5e0 con 0x7f53f41049a0 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53f9a00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 0x7f53f419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53ebe00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 0x7f53f419ab70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005fd0 tx=0x7f53dc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009c30 con 0x7f53f41049a0 2024-09-05T12:11:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f419f8d0 con 0x7f53f41049a0 2024-09-05T12:11:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53dc009dd0 con 0x7f53f41049a0 2024-09-05T12:11:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc010460 con 0x7f53f41049a0 2024-09-05T12:11:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.794+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a71b0 con 0x7f53f41049a0 2024-09-05T12:11:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.796+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f41043d0 con 0x7f53f41049a0 2024-09-05T12:11:47.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.798+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53dc003ef0 con 0x7f53f41049a0 2024-09-05T12:11:47.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.799+0000 7f53f2000640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 0x7f53d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:47.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.799+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f53dc0ceff0 con 0x7f53f41049a0 2024-09-05T12:11:47.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.799+0000 7f53f9000640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 0x7f53d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:47.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.800+0000 7f53f9000640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 0x7f53d80a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009cd0 tx=0x7f53e4005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:47.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:47.803+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53dc017020 con 0x7f53f41049a0 2024-09-05T12:11:47.869 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.044+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53f41a7440 con 0x7f53f41049a0 2024-09-05T12:11:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.046+0000 7f53f2000640 1 -- 172.21.15.59:0/1108077394 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f53dc09bae0 con 0x7f53f41049a0 2024-09-05T12:11:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:11:48.047 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:11:48.048 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:11:48.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 msgr2=0x7f53d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 0x7f53d80a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009cd0 tx=0x7f53e4005f90 comp rx=0 tx=0).stop 2024-09-05T12:11:48.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 msgr2=0x7f53f419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 0x7f53f419ab70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005fd0 tx=0x7f53dc00dde0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 shutdown_connections 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f53d80a2b80 0x7f53d80a5040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53f4109120 0x7f53f419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f41049a0 0x7f53f419ab70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 --2- 172.21.15.59:0/1108077394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53f4103ff0 0x7f53f419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 >> 172.21.15.59:0/1108077394 conn(0x7f53f40ffa90 msgr2=0x7f53f41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 shutdown_connections 2024-09-05T12:11:48.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.048+0000 7f53fba00640 1 -- 172.21.15.59:0/1108077394 wait complete. 2024-09-05T12:11:48.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 -- 172.21.15.59:0/3271415499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb24102360 msgr2=0x7ffb2410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3271415499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb24102360 0x7ffb2410e890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009cd0 tx=0x7ffb1c02a8f0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 -- 172.21.15.59:0/3271415499 shutdown_connections 2024-09-05T12:11:48.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3271415499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb24102360 0x7ffb2410e890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3271415499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb24101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3271415499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb241079c0 0x7ffb24107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.611+0000 7ffb2a200640 1 -- 172.21.15.59:0/3271415499 >> 172.21.15.59:0/3271415499 conn(0x7ffb240fd6e0 msgr2=0x7ffb240ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.612+0000 7ffb2a200640 1 -- 172.21.15.59:0/3271415499 shutdown_connections 2024-09-05T12:11:48.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.612+0000 7ffb2a200640 1 -- 172.21.15.59:0/3271415499 wait complete. 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 Processor -- start 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 -- start start 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb24102360 0x7ffb2419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb241079c0 0x7ffb2419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24111dc0 con 0x7ffb241019c0 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24111c40 con 0x7ffb24102360 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb2a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24111f40 con 0x7ffb241079c0 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.613+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55222/0 (socket says 172.21.15.59:55222) 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 -- 172.21.15.59:0/3911943333 learned_addr learned my addr 172.21.15.59:0/3911943333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb23e00640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb24102360 0x7ffb2419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb29800640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb241079c0 0x7ffb2419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 -- 172.21.15.59:0/3911943333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb241079c0 msgr2=0x7ffb2419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb241079c0 0x7ffb2419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 -- 172.21.15.59:0/3911943333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb24102360 msgr2=0x7ffb2419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb24102360 0x7ffb2419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 -- 172.21.15.59:0/3911943333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2419f3f0 con 0x7ffb241019c0 2024-09-05T12:11:48.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9f0 tx=0x7ffb1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.614+0000 7ffb21600640 1 -- 172.21.15.59:0/3911943333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800cd80 con 0x7ffb241019c0 2024-09-05T12:11:48.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.615+0000 7ffb21600640 1 -- 172.21.15.59:0/3911943333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb18010910 con 0x7ffb241019c0 2024-09-05T12:11:48.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.615+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2419f6e0 con 0x7ffb241019c0 2024-09-05T12:11:48.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.615+0000 7ffb21600640 1 -- 172.21.15.59:0/3911943333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18018990 con 0x7ffb241019c0 2024-09-05T12:11:48.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.615+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb241a6fc0 con 0x7ffb241019c0 2024-09-05T12:11:48.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 Processor -- start 2024-09-05T12:11:48.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 -- start start 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb21600640 1 -- 172.21.15.59:0/3911943333 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb18010ac0 con 0x7ffb241019c0 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd95d0 0x5633efbe3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 0x5633efbe6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5633efb3f290 con 0x5633efbe3e90 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633efae6b40 con 0x5633efbd8cb0 2024-09-05T12:11:48.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.618+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5633efb3b250 con 0x5633efbd95d0 2024-09-05T12:11:48.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb29800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd95d0 0x5633efbe3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb29800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:58458/0 (socket says 172.21.15.59:58458) 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb29800640 1 -- 172.21.15.59:0/913388124 learned_addr learned my addr 172.21.15.59:0/913388124 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 0x5633efbe6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 -- 172.21.15.59:0/913388124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd95d0 msgr2=0x5633efbe3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd95d0 0x5633efbe3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 -- 172.21.15.59:0/913388124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 msgr2=0x5633efbd9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb23e00640 1 -- 172.21.15.59:0/913388124 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633ef8b06a0 con 0x5633efbe3e90 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.619+0000 7ffb29800640 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb23e00640 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 0x5633efbe6280 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009bf0 tx=0x7ffb10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ab871affbaa551 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb16000640 1 -- 172.21.15.59:0/913388124 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1001e070 con 0x5633efbe3e90 2024-09-05T12:11:48.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb16000640 1 -- 172.21.15.59:0/913388124 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb10009da0 con 0x5633efbe3e90 2024-09-05T12:11:48.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb16000640 1 -- 172.21.15.59:0/913388124 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10018810 con 0x5633efbe3e90 2024-09-05T12:11:48.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb21600640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 0x7ffb0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.620+0000 7ffb21600640 1 -- 172.21.15.59:0/3911943333 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb180cf740 con 0x7ffb241019c0 2024-09-05T12:11:48.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb29800640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 0x7ffb0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 -- 172.21.15.59:0/913388124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 msgr2=0x5633efbe6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 0x5633efbe6280 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009bf0 tx=0x7ffb10011810 comp rx=0 tx=0).stop 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 -- 172.21.15.59:0/913388124 shutdown_connections 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbe3e90 0x5633efbe6280 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd95d0 0x5633efbe3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/913388124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd8cb0 0x5633efbd9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 -- 172.21.15.59:0/913388124 >> 172.21.15.59:0/913388124 conn(0x5633efbd5240 msgr2=0x5633efbd5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb29800640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 0x7ffb0c0a5040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c004af0 tx=0x7ffb1c004a80 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 -- 172.21.15.59:0/913388124 shutdown_connections 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.621+0000 7ffb2e3df740 1 -- 172.21.15.59:0/913388124 wait complete. 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.622+0000 7ffb2e3df740 1 -- start start 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.622+0000 7ffb2e3df740 1 Processor -- start 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbe0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5633efbe6b00 con 0x5633efbd8cb0 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633efaf9fa0 con 0x5633efbd95d0 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5633efbe6c80 con 0x5633efbe3e90 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb2e3df740 10 client.0 _ll_register_callbacks cb 0x5633ef84dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55230/0 (socket says 172.21.15.59:55230) 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb29800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbe0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb28e00640 1 -- 172.21.15.59:0/2254476737 learned_addr learned my addr 172.21.15.59:0/2254476737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:48.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.623+0000 7ffb23e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 -- 172.21.15.59:0/2254476737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 msgr2=0x5633efbe0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbe0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 -- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 msgr2=0x5633efbe0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 -- 172.21.15.59:0/2254476737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633efbe17d0 con 0x5633efbd8cb0 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb29800640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbe0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb23e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:48.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb28e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb180a4270 tx=0x7ffb18097f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53598418667a14ad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1809ce60 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb180cfb30 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb10009da0 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.624+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ffb18086040 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 10 client.17601 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5633efbf2a40 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb10009da0 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ffb180cf070 con 0x5633efbd8cb0 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 10 client.17601 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 20 client.17601 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:48.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efbf3f00 0x5633efbf6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 4 client.17601 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.625+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633efb4c800 con 0x5633efbf3f00 2024-09-05T12:11:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.626+0000 7ffb23e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efbf3f00 0x5633efbf6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.626+0000 7ffb23e00640 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efbf3f00 0x5633efbf6310 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5b289cc2238b27 server_cookie=c719e52b4dc49c25 in_seq=0 out_seq=0 2024-09-05T12:11:48.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.626+0000 7ffb0aa00640 10 client.17601 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.627+0000 7ffb0aa00640 1 -- 172.21.15.59:0/2254476737 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633efb4c800 con 0x5633efbf3f00 2024-09-05T12:11:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.627+0000 7ffb0aa00640 10 client.17601 handle_command_reply: tid=0 2024-09-05T12:11:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.627+0000 7ffb2e3df740 1 client.17601 shutdown 2024-09-05T12:11:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.627+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 msgr2=0x5633efbe3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.627+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb180a4270 tx=0x7ffb18097f90 comp rx=0 tx=0).stop 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 shutdown_connections 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbe0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efbf3f00 0x5633efbf6310 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/2254476737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbe3790 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 >> 172.21.15.59:0/2254476737 conn(0x5633efbd5240 msgr2=0x5633efbdaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb0aa00640 0 client.17601 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 shutdown_connections 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 1 -- 172.21.15.59:0/2254476737 wait complete. 2024-09-05T12:11:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.628+0000 7ffb2e3df740 20 client.17601 trim_cache size 0 max 16384 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.649+0000 7ffb2e3df740 1 Processor -- start 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.649+0000 7ffb2e3df740 1 -- start start 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd95d0 0x5633efbfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbe3e90 0x5633efbfc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241aefd0 con 0x5633efbd95d0 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b1850 con 0x5633efbe3e90 2024-09-05T12:11:48.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241af640 con 0x5633efbd8cb0 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57448/0 (socket says 172.21.15.59:57448) 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 -- 172.21.15.59:0/3620741117 learned_addr learned my addr 172.21.15.59:0/3620741117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb23e00640 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd95d0 0x5633efbfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb29800640 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbe3e90 0x5633efbfc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 -- 172.21.15.59:0/3620741117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbe3e90 msgr2=0x5633efbfc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbe3e90 0x5633efbfc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 -- 172.21.15.59:0/3620741117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd95d0 msgr2=0x5633efbfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd95d0 0x5633efbfbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.650+0000 7ffb28e00640 1 -- 172.21.15.59:0/3620741117 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633efbdf9f0 con 0x5633efbd8cb0 2024-09-05T12:11:48.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.651+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffb18098040 tx=0x7ffb18000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694cd13fbf49af1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.651+0000 7ffb15600640 1 -- 172.21.15.59:0/3620741117 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180b8030 con 0x5633efbd8cb0 2024-09-05T12:11:48.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.651+0000 7ffb15600640 1 -- 172.21.15.59:0/3620741117 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb180d5220 con 0x5633efbd8cb0 2024-09-05T12:11:48.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.651+0000 7ffb15600640 1 -- 172.21.15.59:0/3620741117 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180ba2c0 con 0x5633efbd8cb0 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3620741117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 msgr2=0x5633efbd4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffb18098040 tx=0x7ffb18000bc0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3620741117 shutdown_connections 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbe3e90 0x5633efbfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd95d0 0x5633efbfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3620741117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbd8cb0 0x5633efbd4160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3620741117 >> 172.21.15.59:0/3620741117 conn(0x5633efbd5240 msgr2=0x5633efbd3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3620741117 shutdown_connections 2024-09-05T12:11:48.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.652+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3620741117 wait complete. 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 -- start start 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 Processor -- start 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241afd10 con 0x5633efbd8cb0 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b1180 con 0x5633efbd95d0 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b03e0 con 0x5633efbe3e90 2024-09-05T12:11:48.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb2e3df740 10 client.0 _ll_register_callbacks cb 0x5633efbe19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.653+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb29800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55252/0 (socket says 172.21.15.59:55252) 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb29800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57462/0 (socket says 172.21.15.59:57462) 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 -- 172.21.15.59:0/3735625723 learned_addr learned my addr 172.21.15.59:0/3735625723 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 -- 172.21.15.59:0/3735625723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 msgr2=0x5633efbea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 -- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 msgr2=0x5633efbe9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 -- 172.21.15.59:0/3735625723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633efbd9400 con 0x5633efbd8cb0 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb23e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:48.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb28e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffb18018cd0 tx=0x7ffb180ba5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d020e4a5022c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180aa060 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.654+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb1809b030 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb180d5220 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ffb180b8050 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 10 client.17607 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5633efbd9400 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb180d5220 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ffb180cf040 con 0x5633efbd8cb0 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 10 client.17607 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 20 client.17607 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efc5ee40 0x5633efc61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:48.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 4 client.17607 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:11:48.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.655+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5633efb46d10 con 0x5633efc5ee40 2024-09-05T12:11:48.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.656+0000 7ffb23e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efc5ee40 0x5633efc61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.656+0000 7ffb23e00640 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efc5ee40 0x5633efc61230 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d8b9f90dd2e342 server_cookie=e63ec9762f415c6b in_seq=0 out_seq=0 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.656+0000 7ffb0aa00640 10 client.17607 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.657+0000 7ffb0aa00640 1 -- 172.21.15.59:0/3735625723 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5633efb46d10 con 0x5633efc5ee40 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.657+0000 7ffb0aa00640 10 client.17607 handle_command_reply: tid=0 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.657+0000 7ffb2e3df740 1 client.17607 shutdown 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.657+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 msgr2=0x5633efbec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.657+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffb18018cd0 tx=0x7ffb180ba5d0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 shutdown_connections 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633efbe3e90 0x5633efbea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633efbd95d0 0x5633efbe9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5633efc5ee40 0x5633efc61230 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 --2- 172.21.15.59:0/3735625723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5633efbd8cb0 0x5633efbec760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 >> 172.21.15.59:0/3735625723 conn(0x5633efbd5240 msgr2=0x5633efbd4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb0aa00640 0 client.17607 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 shutdown_connections 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 1 -- 172.21.15.59:0/3735625723 wait complete. 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.658+0000 7ffb2e3df740 20 client.17607 trim_cache size 0 max 16384 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "c1d17743-bb5b-4494-a64e-d8790ca106d3", 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:11:48.663 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 msgr2=0x7ffb0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 0x7ffb0c0a5040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c004af0 tx=0x7ffb1c004a80 comp rx=0 tx=0).stop 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 msgr2=0x7ffb2419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9f0 tx=0x7ffb1800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 shutdown_connections 2024-09-05T12:11:48.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ffb0c0a2b80 0x7ffb0c0a5040 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb241079c0 0x7ffb2419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb24102360 0x7ffb2419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 --2- 172.21.15.59:0/3911943333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb241019c0 0x7ffb2419a440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 >> 172.21.15.59:0/3911943333 conn(0x7ffb240fd6e0 msgr2=0x7ffb2410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 shutdown_connections 2024-09-05T12:11:48.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:48.659+0000 7ffb2a200640 1 -- 172.21.15.59:0/3911943333 wait complete. 2024-09-05T12:11:49.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:48 smithi059 ceph-mon[30025]: pgmap v818: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:49.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:48 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1108077394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:48 smithi089 ceph-mon[34499]: pgmap v818: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:48 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1108077394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:48 smithi189 ceph-mon[34772]: pgmap v818: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:48 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1108077394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:49.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:11:49.757 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-05T12:11:49.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-05T12:11:49.860 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:49.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:49 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:11:49.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:49 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.000 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:49 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:11:50.000 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:49 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:49 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:11:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:49 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:49 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:11:50.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:49 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:49 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:11:50.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:49 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:49 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:11:50.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:49 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 -- 172.21.15.59:0/2325872958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 msgr2=0x7f3ff8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2325872958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff8110cc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009ca0 tx=0x7f3ff002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 -- 172.21.15.59:0/2325872958 shutdown_connections 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2325872958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff8110cc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2325872958 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff81049c0 0x7f3ff8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2325872958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff8104010 0x7f3ff81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 -- 172.21.15.59:0/2325872958 >> 172.21.15.59:0/2325872958 conn(0x7f3ff80ffa50 msgr2=0x7f3ff8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 -- 172.21.15.59:0/2325872958 shutdown_connections 2024-09-05T12:11:50.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.444+0000 7f3ffe200640 1 -- 172.21.15.59:0/2325872958 wait complete. 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.445+0000 7f3ffe200640 1 Processor -- start 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.445+0000 7f3ffe200640 1 -- start start 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.445+0000 7f3ffe200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 0x7f3ff819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ffe200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 0x7f3ff819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ffe200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ffe200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8114150 con 0x7f3ff8109140 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ffe200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8113fd0 con 0x7f3ff8104010 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ffe200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81142d0 con 0x7f3ff81049c0 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55276/0 (socket says 172.21.15.59:55276) 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7e00640 1 -- 172.21.15.59:0/2798373802 learned_addr learned my addr 172.21.15.59:0/2798373802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 0x7f3ff819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 0x7f3ff819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 -- 172.21.15.59:0/2798373802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 msgr2=0x7f3ff819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 0x7f3ff819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 -- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 msgr2=0x7f3ff819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.446+0000 7f3ff7400640 1 -- 172.21.15.59:0/2798373802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff819f5a0 con 0x7f3ff8104010 2024-09-05T12:11:50.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 0x7f3ff819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3ff7400640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 0x7f3ff819a5f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3fefe00640 1 -- 172.21.15.59:0/2798373802 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800cd80 con 0x7f3ff8104010 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3fefe00640 1 -- 172.21.15.59:0/2798373802 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe80035f0 con 0x7f3ff8104010 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff819f890 con 0x7f3ff8104010 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3fefe00640 1 -- 172.21.15.59:0/2798373802 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8010690 con 0x7f3ff8104010 2024-09-05T12:11:50.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.447+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff81a7170 con 0x7f3ff8104010 2024-09-05T12:11:50.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.450+0000 7f3fefe00640 1 -- 172.21.15.59:0/2798373802 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3fe8003b30 con 0x7f3ff8104010 2024-09-05T12:11:50.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 Processor -- start 2024-09-05T12:11:50.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 -- start start 2024-09-05T12:11:50.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 0x55bbb97a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.451+0000 7f4000c0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbb970e3e0 con 0x55bbb97a7b90 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f4000c0e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bbb96b5d80 con 0x55bbb97b2d70 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f4000c0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbb970a3a0 con 0x55bbb97a84b0 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57500/0 (socket says 172.21.15.59:57500) 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 -- 172.21.15.59:0/98728169 learned_addr learned my addr 172.21.15.59:0/98728169 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 0x55bbb97a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 -- 172.21.15.59:0/98728169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 msgr2=0x55bbb97b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 -- 172.21.15.59:0/98728169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 msgr2=0x55bbb97a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 0x55bbb97a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7400640 1 -- 172.21.15.59:0/98728169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb96d4ae0 con 0x55bbb97a84b0 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.452+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 0x55bbb97a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3fefe00640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 0x7f3fdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3ff7400640 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80d5a90 tx=0x7f3fe80d5f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c512d8b79c75496 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3fefe00640 1 -- 172.21.15.59:0/2798373802 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3fe80cf790 con 0x7f3ff8104010 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 0x7f3fdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3fe3e00640 1 -- 172.21.15.59:0/98728169 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8097e20 con 0x55bbb97a84b0 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3fe3e00640 1 -- 172.21.15.59:0/98728169 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8009450 con 0x55bbb97a84b0 2024-09-05T12:11:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.453+0000 7f3fe3e00640 1 -- 172.21.15.59:0/98728169 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe809b4e0 con 0x55bbb97a84b0 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 -- 172.21.15.59:0/98728169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 msgr2=0x55bbb97b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 0x7f3fdc0a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005cc0 tx=0x7f3ff0005c50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80d5a90 tx=0x7f3fe80d5f50 comp rx=0 tx=0).stop 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 -- 172.21.15.59:0/98728169 shutdown_connections 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a84b0 0x55bbb97b2830 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 --2- 172.21.15.59:0/98728169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a7b90 0x55bbb97a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.454+0000 7f4000c0e740 1 -- 172.21.15.59:0/98728169 >> 172.21.15.59:0/98728169 conn(0x55bbb97a4120 msgr2=0x55bbb97a4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.455+0000 7f4000c0e740 1 -- 172.21.15.59:0/98728169 shutdown_connections 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.455+0000 7f4000c0e740 1 -- 172.21.15.59:0/98728169 wait complete. 2024-09-05T12:11:50.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.455+0000 7f4000c0e740 1 -- start start 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.455+0000 7f4000c0e740 1 Processor -- start 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bbb97b59e0 con 0x55bbb97a84b0 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bbb96c90f0 con 0x55bbb97b2d70 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbb97b5b60 con 0x55bbb97a7b90 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f4000c0e740 10 client.0 _ll_register_callbacks cb 0x55bbb96ba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57502/0 (socket says 172.21.15.59:57502) 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7400640 1 -- 172.21.15.59:0/791339528 learned_addr learned my addr 172.21.15.59:0/791339528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:50.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7400640 1 -- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 msgr2=0x55bbb97b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.456+0000 7f3ff7400640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3ff7400640 1 -- 172.21.15.59:0/791339528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 msgr2=0x55bbb97b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3ff7400640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3ff7400640 1 -- 172.21.15.59:0/791339528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb947f3c0 con 0x55bbb97a7b90 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3ff7400640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe80d5390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c2ed2c7fb91af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe809b6c0 con 0x55bbb97a7b90 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3fe80037a0 con 0x55bbb97a7b90 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.457+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8009450 con 0x55bbb97a7b90 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3fe809b950 con 0x55bbb97a7b90 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 10 client.27544 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:50.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bbb97b59e0 con 0x55bbb97a7b90 2024-09-05T12:11:50.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8009450 con 0x55bbb97a7b90 2024-09-05T12:11:50.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3fe80ba320 con 0x55bbb97a7b90 2024-09-05T12:11:50.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 10 client.27544 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:50.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 10 client.27544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:50.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 20 client.27544 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb97c2890 0x55bbb97c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 4 client.27544 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.458+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb971b950 con 0x55bbb97c2890 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.459+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb97c2890 0x55bbb97c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.459+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb97c2890 0x55bbb97c4c80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817b805ba0643d4f server_cookie=6737e25eeeffbe95 in_seq=0 out_seq=0 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.459+0000 7f3fe0c00640 10 client.27544 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f3fe0c00640 1 -- 172.21.15.59:0/791339528 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbb971b950 con 0x55bbb97c2890 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f3fe0c00640 10 client.27544 handle_command_reply: tid=0 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 client.27544 shutdown 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 msgr2=0x55bbb97afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe80d5390 comp rx=0 tx=0).stop 2024-09-05T12:11:50.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 shutdown_connections 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb97c2890 0x55bbb97c4c80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 --2- 172.21.15.59:0/791339528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97afcf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f3fe0c00640 0 client.27544 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.460+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 >> 172.21.15.59:0/791339528 conn(0x55bbb97a4120 msgr2=0x55bbb97a9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.461+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 shutdown_connections 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.461+0000 7f4000c0e740 1 -- 172.21.15.59:0/791339528 wait complete. 2024-09-05T12:11:50.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.461+0000 7f4000c0e740 20 client.27544 trim_cache size 0 max 16384 2024-09-05T12:11:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 Processor -- start 2024-09-05T12:11:50.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 -- start start 2024-09-05T12:11:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81af0d0 con 0x55bbb97a84b0 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81b1950 con 0x55bbb97b2d70 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.479+0000 7f4000c0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81af740 con 0x55bbb97a7b90 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57508/0 (socket says 172.21.15.59:57508) 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 -- 172.21.15.59:0/1044694083 learned_addr learned my addr 172.21.15.59:0/1044694083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 -- 172.21.15.59:0/1044694083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 msgr2=0x55bbb97b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 -- 172.21.15.59:0/1044694083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 msgr2=0x55bbb97b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 -- 172.21.15.59:0/1044694083 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb97a9110 con 0x55bbb97a7b90 2024-09-05T12:11:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3ff7400640 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c6a0 tx=0x7f3fe809a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d6f4cdef6beb71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.480+0000 7f3fe3400640 1 -- 172.21.15.59:0/1044694083 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe809b6c0 con 0x55bbb97a7b90 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f3fe3400640 1 -- 172.21.15.59:0/1044694083 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8009430 con 0x55bbb97a7b90 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f3fe3400640 1 -- 172.21.15.59:0/1044694083 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8097e20 con 0x55bbb97a7b90 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 -- 172.21.15.59:0/1044694083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 msgr2=0x55bbb97ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c6a0 tx=0x7f3fe809a090 comp rx=0 tx=0).stop 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 -- 172.21.15.59:0/1044694083 shutdown_connections 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97b2d70 0x55bbb97b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97a84b0 0x55bbb97b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 --2- 172.21.15.59:0/1044694083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97ae5c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.481+0000 7f4000c0e740 1 -- 172.21.15.59:0/1044694083 >> 172.21.15.59:0/1044694083 conn(0x55bbb97a4120 msgr2=0x55bbb97afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.482+0000 7f4000c0e740 1 -- 172.21.15.59:0/1044694083 shutdown_connections 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.482+0000 7f4000c0e740 1 -- 172.21.15.59:0/1044694083 wait complete. 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.482+0000 7f4000c0e740 1 -- start start 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.482+0000 7f4000c0e740 1 Processor -- start 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 0x55bbb97b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 0x55bbb97b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81afe10 con 0x55bbb97b2d70 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81b1280 con 0x55bbb97a84b0 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81b04e0 con 0x55bbb97a7b90 2024-09-05T12:11:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f4000c0e740 10 client.0 _ll_register_callbacks cb 0x7f3fe80d58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 0x55bbb97b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57520/0 (socket says 172.21.15.59:57520) 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7400640 1 -- 172.21.15.59:0/3326969405 learned_addr learned my addr 172.21.15.59:0/3326969405 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 0x55bbb97b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7e00640 1 -- 172.21.15.59:0/3326969405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 msgr2=0x55bbb97b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.483+0000 7f3ff7e00640 1 -- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 msgr2=0x55bbb97b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 0x55bbb97b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3ff7e00640 1 -- 172.21.15.59:0/3326969405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb982c420 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3ff6a00640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 0x55bbb97b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3ff7e00640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 0x55bbb97b7630 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009dc0 tx=0x7f3ff00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd55bd119a9d1cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0003c40 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.484+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3ff0003de0 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8009430 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0004050 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f4000c0e740 10 client.17625 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bbb982d710 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8009430 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.485+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3ff00319c0 con 0x55bbb97b2d70 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 10 client.17625 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:50.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 20 client.17625 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:50.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb982dee0 0x55bbb9830300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:50.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 4 client.17625 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:11:50.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbb9715e60 con 0x55bbb982dee0 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.486+0000 7f3ff7400640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb982dee0 0x55bbb9830300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f3ff7400640 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb982dee0 0x55bbb9830300 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb3b4b22b6c10d1 server_cookie=8edbefa1d73588e9 in_seq=0 out_seq=0 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f3fe0c00640 10 client.17625 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f3fe0c00640 1 -- 172.21.15.59:0/3326969405 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbb9715e60 con 0x55bbb982dee0 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f3fe0c00640 10 client.17625 handle_command_reply: tid=0 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 client.17625 shutdown 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 msgr2=0x55bbb97b7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 0x55bbb97b7630 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009dc0 tx=0x7f3ff00095c0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 shutdown_connections 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55bbb982dee0 0x55bbb9830300 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bbb97b2d70 0x55bbb97b7630 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bbb97a84b0 0x55bbb97b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 --2- 172.21.15.59:0/3326969405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb97a7b90 0x55bbb97b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 >> 172.21.15.59:0/3326969405 conn(0x55bbb97a4120 msgr2=0x55bbb97b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f3fe0c00640 0 client.17625 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 shutdown_connections 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.487+0000 7f4000c0e740 1 -- 172.21.15.59:0/3326969405 wait complete. 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f4000c0e740 20 client.17625 trim_cache size 0 max 16384 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:11:50.491 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 msgr2=0x7f3fdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 0x7f3fdc0a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005cc0 tx=0x7f3ff0005c50 comp rx=0 tx=0).stop 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 msgr2=0x7f3ff819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 0x7f3ff819a5f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 shutdown_connections 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3fdc0a2b80 0x7f3fdc0a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ff8109140 0x7f3ff819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff81049c0 0x7f3ff819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 --2- 172.21.15.59:0/2798373802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ff8104010 0x7f3ff819a5f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.488+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 >> 172.21.15.59:0/2798373802 conn(0x7f3ff80ffa50 msgr2=0x7f3ff8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.489+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 shutdown_connections 2024-09-05T12:11:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:50.489+0000 7f3ffe200640 1 -- 172.21.15.59:0/2798373802 wait complete. 2024-09-05T12:11:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:50 smithi089 ceph-mon[34499]: pgmap v819: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:50 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:11:51.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:50 smithi059 ceph-mon[30025]: pgmap v819: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:51.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:50 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:11:51.234 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:51.266 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:11:51.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:11:51.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:50 smithi189 ceph-mon[34772]: pgmap v819: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-05T12:11:51.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:50 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:11:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 -- 172.21.15.59:0/973582767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4103ff0 msgr2=0x7f7fc41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 --2- 172.21.15.59:0/973582767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4103ff0 0x7f7fc41043d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009900 tx=0x7f7fb402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 -- 172.21.15.59:0/973582767 shutdown_connections 2024-09-05T12:11:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 --2- 172.21.15.59:0/973582767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc4109120 0x7f7fc4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 --2- 172.21.15.59:0/973582767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc41049a0 0x7f7fc41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 --2- 172.21.15.59:0/973582767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4103ff0 0x7f7fc41043d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 -- 172.21.15.59:0/973582767 >> 172.21.15.59:0/973582767 conn(0x7f7fc40ffa90 msgr2=0x7f7fc4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 -- 172.21.15.59:0/973582767 shutdown_connections 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.818+0000 7f7fcae00640 1 -- 172.21.15.59:0/973582767 wait complete. 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.819+0000 7f7fcae00640 1 Processor -- start 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.819+0000 7f7fcae00640 1 -- start start 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 0x7f7fc419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4109120 0x7f7fc419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4114140 con 0x7f7fc4109120 2024-09-05T12:11:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113fc0 con 0x7f7fc41049a0 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fcae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41142c0 con 0x7f7fc4103ff0 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 0x7f7fc419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4109120 0x7f7fc419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46338/0 (socket says 172.21.15.59:46338) 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc3400640 1 -- 172.21.15.59:0/798392533 learned_addr learned my addr 172.21.15.59:0/798392533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.820+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 0x7f7fc419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43632/0 (socket says 172.21.15.59:43632) 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 -- 172.21.15.59:0/798392533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 msgr2=0x7f7fc419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 0x7f7fc419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 -- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4109120 msgr2=0x7f7fc419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4109120 0x7f7fc419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 -- 172.21.15.59:0/798392533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc419f5b0 con 0x7f7fc41049a0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc3400640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e9f0 tx=0x7f7fb000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc0c00640 1 -- 172.21.15.59:0/798392533 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000cd80 con 0x7f7fc41049a0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc0c00640 1 -- 172.21.15.59:0/798392533 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0005500 con 0x7f7fc41049a0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc419f8a0 con 0x7f7fc41049a0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fc0c00640 1 -- 172.21.15.59:0/798392533 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0010690 con 0x7f7fc41049a0 2024-09-05T12:11:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.821+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a7180 con 0x7f7fc41049a0 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.824+0000 7f7fcd7bd740 1 Processor -- start 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.824+0000 7f7fcd7bd740 1 -- start start 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7dcd0 0x55fd52b88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b8a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fd52ae3530 con 0x55fd52b7dcd0 2024-09-05T12:11:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fd52a8a860 con 0x55fd52b88590 2024-09-05T12:11:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd52adf4f0 con 0x55fd52b7d3b0 2024-09-05T12:11:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc0c00640 1 -- 172.21.15.59:0/798392533 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fb00046c0 con 0x7f7fc41049a0 2024-09-05T12:11:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7dcd0 0x55fd52b88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43644/0 (socket says 172.21.15.59:43644) 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc8e00640 1 -- 172.21.15.59:0/2157549708 learned_addr learned my addr 172.21.15.59:0/2157549708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b8a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc8e00640 1 -- 172.21.15.59:0/2157549708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 msgr2=0x55fd52b8a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.825+0000 7f7fc8e00640 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b8a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fc8e00640 1 -- 172.21.15.59:0/2157549708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7dcd0 msgr2=0x55fd52b88050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fc8e00640 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7dcd0 0x55fd52b88050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fc8e00640 1 -- 172.21.15.59:0/2157549708 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd52854b90 con 0x55fd52b7d3b0 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b8a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fc8e00640 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009f50 tx=0x7f7fbc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56e47dc40aa1711 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fb9600640 1 -- 172.21.15.59:0/2157549708 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc004d20 con 0x55fd52b7d3b0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fb9600640 1 -- 172.21.15.59:0/2157549708 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc004050 con 0x55fd52b7d3b0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.826+0000 7f7fb9600640 1 -- 172.21.15.59:0/2157549708 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc005280 con 0x55fd52b7d3b0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fc0c00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 0x7f7fa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fc0c00640 1 -- 172.21.15.59:0/798392533 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7fb00cf830 con 0x7f7fc41049a0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 0x7f7fa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157549708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 msgr2=0x55fd52b7d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009f50 tx=0x7f7fbc0118d0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157549708 shutdown_connections 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b8a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7dcd0 0x55fd52b88050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157549708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b7d790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.827+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157549708 >> 172.21.15.59:0/2157549708 conn(0x55fd52b79940 msgr2=0x55fd52b79d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.828+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157549708 shutdown_connections 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.828+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157549708 wait complete. 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.828+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 0x7f7fa80a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009230 tx=0x7f7fb40058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.828+0000 7f7fcd7bd740 1 -- start start 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.828+0000 7f7fcd7bd740 1 Processor -- start 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b86140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fd52b8afb0 con 0x55fd52b7d3b0 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fd52a9de60 con 0x55fd52b88590 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd52b8b130 con 0x55fd52b7dcd0 2024-09-05T12:11:51.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fcd7bd740 10 client.0 _ll_register_callbacks cb 0x55fd52aa2cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58868/0 (socket says 172.21.15.59:58868) 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc3e00640 1 -- 172.21.15.59:0/3718384970 learned_addr learned my addr 172.21.15.59:0/3718384970 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc3400640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc8e00640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b86140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.829+0000 7f7fc3400640 1 -- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 msgr2=0x55fd52b86140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3400640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b86140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3400640 1 -- 172.21.15.59:0/3718384970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 msgr2=0x55fd52b856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3400640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3400640 1 -- 172.21.15.59:0/3718384970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd52aa9990 con 0x55fd52b7dcd0 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fc3400640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b85c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ccf70 tx=0x7f7fb009b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d067e7116b01f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fae000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00a4660 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc004050 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7fb009b0d0 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7fb009bc80 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fcd7bd740 10 client.27562 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.830+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fd52b96de0 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc004050 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7fb00b4060 con 0x55fd52b7dcd0 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 10 client.27562 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 10 client.27562 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 20 client.27562 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52b981f0 0x55fd52b9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 4 client.27562 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd52a86310 con 0x55fd52b981f0 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.831+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52b981f0 0x55fd52b9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.832+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52b981f0 0x55fd52b9a5e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12cf1851e94f52a6 server_cookie=68b1c85dbced7729 in_seq=0 out_seq=0 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.832+0000 7f7fae000640 10 client.27562 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fae000640 1 -- 172.21.15.59:0/3718384970 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd52a86310 con 0x55fd52b981f0 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fae000640 10 client.27562 handle_command_reply: tid=0 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 client.27562 shutdown 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 msgr2=0x55fd52b85c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b85c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ccf70 tx=0x7f7fb009b7c0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 shutdown_connections 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b88590 0x55fd52b86140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b85c00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52b981f0 0x55fd52b9a5e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/3718384970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b7d3b0 0x55fd52b856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fae000640 0 client.27562 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 >> 172.21.15.59:0/3718384970 conn(0x55fd52b79940 msgr2=0x55fd52b78c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 shutdown_connections 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/3718384970 wait complete. 2024-09-05T12:11:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.833+0000 7f7fcd7bd740 20 client.27562 trim_cache size 0 max 16384 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 Processor -- start 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 -- start start 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b84590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 0x55fd52b9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52ba03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41af080 con 0x55fd52b88590 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b1900 con 0x55fd52b7dcd0 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.852+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41af6f0 con 0x55fd52b7d3b0 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b84590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 0x55fd52b9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b84590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43664/0 (socket says 172.21.15.59:43664) 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2603467033 learned_addr learned my addr 172.21.15.59:0/2603467033 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc8e00640 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52ba03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 -- 172.21.15.59:0/2603467033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 msgr2=0x55fd52b84590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b84590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 -- 172.21.15.59:0/2603467033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 msgr2=0x55fd52ba03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52ba03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.853+0000 7f7fc3400640 1 -- 172.21.15.59:0/2603467033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd52b9a950 con 0x55fd52b7dcd0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.854+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 0x55fd52b9fe80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55fd52b86c70 tx=0x7f7fb009a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695203436962304a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.854+0000 7f7fb8c00640 1 -- 172.21.15.59:0/2603467033 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb009b5e0 con 0x55fd52b7dcd0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.854+0000 7f7fb8c00640 1 -- 172.21.15.59:0/2603467033 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb00a48f0 con 0x55fd52b7dcd0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.854+0000 7f7fb8c00640 1 -- 172.21.15.59:0/2603467033 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb009ab40 con 0x55fd52b7dcd0 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2603467033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 msgr2=0x55fd52b9fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 0x55fd52b9fe80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55fd52b86c70 tx=0x7f7fb009a190 comp rx=0 tx=0).stop 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2603467033 shutdown_connections 2024-09-05T12:11:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52ba03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7dcd0 0x55fd52b9fe80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2603467033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7d3b0 0x55fd52b84590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2603467033 >> 172.21.15.59:0/2603467033 conn(0x55fd52b79940 msgr2=0x55fd52b83e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2603467033 shutdown_connections 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2603467033 wait complete. 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.855+0000 7f7fcd7bd740 1 -- start start 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 Processor -- start 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52b8d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41afdc0 con 0x55fd52b88590 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b1230 con 0x55fd52b7d3b0 2024-09-05T12:11:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b0490 con 0x55fd52b7dcd0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.856+0000 7f7fcd7bd740 10 client.0 _ll_register_callbacks cb 0x7f7fb000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52b8d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46390/0 (socket says 172.21.15.59:46390) 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2157420466 learned_addr learned my addr 172.21.15.59:0/2157420466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2157420466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 msgr2=0x55fd52b908c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b908c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2157420466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 msgr2=0x55fd52b8d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52b8d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 -- 172.21.15.59:0/2157420466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd52b8d890 con 0x55fd52b7d3b0 2024-09-05T12:11:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fc3e00640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4004770 tx=0x7f7fb4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a35064b958ecda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.857+0000 7f7fae000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4035040 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb00a48f0 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7fb4003b80 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7fb4066050 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fcd7bd740 10 client.27644 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.858+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fd52c03530 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb00a48f0 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7fb405d500 con 0x55fd52b7d3b0 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 10 client.27644 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 10 client.27644 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 20 client.27644 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52c03ca0 0x55fd52c060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 4 client.27644 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-05T12:11:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x55fd52af0c60 con 0x55fd52c03ca0 2024-09-05T12:11:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.859+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52c03ca0 0x55fd52c060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.860+0000 7f7fc3400640 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52c03ca0 0x55fd52c060c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318fbf42626d06dc server_cookie=bd0eeb2fc8ac5afe in_seq=0 out_seq=0 2024-09-05T12:11:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.860+0000 7f7fae000640 10 client.27644 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.868+0000 7f7fae000640 1 -- 172.21.15.59:0/2157420466 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fd52af0c60 con 0x55fd52c03ca0 2024-09-05T12:11:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.868+0000 7f7fae000640 10 client.27644 handle_command_reply: tid=0 2024-09-05T12:11:51.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.868+0000 7f7fcd7bd740 1 client.27644 shutdown 2024-09-05T12:11:51.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.868+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 msgr2=0x55fd52b90380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.868+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4004770 tx=0x7f7fb4009260 comp rx=0 tx=0).stop 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 shutdown_connections 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55fd52c03ca0 0x55fd52c060c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fd52b88590 0x55fd52b8d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd52b7dcd0 0x55fd52b908c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 --2- 172.21.15.59:0/2157420466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fd52b7d3b0 0x55fd52b90380 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 >> 172.21.15.59:0/2157420466 conn(0x55fd52b79940 msgr2=0x55fd52b846a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fae000640 0 client.27644 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 shutdown_connections 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 1 -- 172.21.15.59:0/2157420466 wait complete. 2024-09-05T12:11:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.869+0000 7f7fcd7bd740 20 client.27644 trim_cache size 0 max 16384 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 46, 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "6925392e": { 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 46, 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:11:51.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 134.5, 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:51.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 msgr2=0x7f7fa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 0x7f7fa80a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009230 tx=0x7f7fb40058d0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 msgr2=0x7f7fc419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e9f0 tx=0x7f7fb000edf0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 shutdown_connections 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7fa80a2b80 0x7f7fa80a5040 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fc4109120 0x7f7fc419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fc41049a0 0x7f7fc419ab40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 --2- 172.21.15.59:0/798392533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc4103ff0 0x7f7fc419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 >> 172.21.15.59:0/798392533 conn(0x7f7fc40ffa90 msgr2=0x7f7fc4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 shutdown_connections 2024-09-05T12:11:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:51.870+0000 7f7fcae00640 1 -- 172.21.15.59:0/798392533 wait complete. 2024-09-05T12:11:52.100 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:52.633 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6925392e', quiesce took 5.9 seconds. Will release after: 8.6 2024-09-05T12:11:53.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:52 smithi059 ceph-mon[30025]: pgmap v820: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:53.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:52 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 -- 172.21.15.59:0/4065283020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf0074ee0 msgr2=0x7f3cf010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/4065283020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf0074ee0 0x7f3cf010dd20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009900 tx=0x7f3cd802a480 comp rx=0 tx=0).stop 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 -- 172.21.15.59:0/4065283020 shutdown_connections 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/4065283020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 0x7f3cf0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/4065283020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf0074ee0 0x7f3cf010dd20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/4065283020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf00745c0 0x7f3cf00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 -- 172.21.15.59:0/4065283020 >> 172.21.15.59:0/4065283020 conn(0x7f3cf00fd740 msgr2=0x7f3cf00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 -- 172.21.15.59:0/4065283020 shutdown_connections 2024-09-05T12:11:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.152+0000 7f3cf5c00640 1 -- 172.21.15.59:0/4065283020 wait complete. 2024-09-05T12:11:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.153+0000 7f3cf5c00640 1 Processor -- start 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.153+0000 7f3cf5c00640 1 -- start start 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf0074ee0 0x7f3cf019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 0x7f3cf019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0114000 con 0x7f3cf0074ee0 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0113e80 con 0x7f3cf010e260 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0114180 con 0x7f3cf00745c0 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3ce7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf0074ee0 0x7f3cf019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43696/0 (socket says 172.21.15.59:43696) 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 -- 172.21.15.59:0/2354189966 learned_addr learned my addr 172.21.15.59:0/2354189966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cf5200640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 0x7f3cf019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 -- 172.21.15.59:0/2354189966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 msgr2=0x7f3cf019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 0x7f3cf019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 -- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf0074ee0 msgr2=0x7f3cf019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf0074ee0 0x7f3cf019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.154+0000 7f3cefe00640 1 -- 172.21.15.59:0/2354189966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf019f450 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3cefe00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9f0 tx=0x7f3ce000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce000cd80 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0010910 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0018990 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf019f740 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.155+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01a7020 con 0x7f3cf00745c0 2024-09-05T12:11:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.156+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf00749a0 con 0x7f3cf00745c0 2024-09-05T12:11:53.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.158+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ce0010ac0 con 0x7f3cf00745c0 2024-09-05T12:11:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.160+0000 7f3ced600640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 0x7f3cd40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.160+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3ce00cf720 con 0x7f3cf00745c0 2024-09-05T12:11:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.160+0000 7f3ce7400640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 0x7f3cd40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:53.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.161+0000 7f3ce7400640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 0x7f3cd40a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8000c00 tx=0x7f3cd8005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:53.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.163+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ce009bbc0 con 0x7f3cf00745c0 2024-09-05T12:11:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:52 smithi089 ceph-mon[34499]: pgmap v820: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:53.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:52 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:11:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:52 smithi189 ceph-mon[34772]: pgmap v820: 129 pgs: 129 active+clean; 7.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:11:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:52 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:11:53.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.413+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cf019b800 con 0x7f3cf00745c0 2024-09-05T12:11:53.416 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:11:53.416 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:11:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.414+0000 7f3ced600640 1 -- 172.21.15.59:0/2354189966 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f3ce009fec0 con 0x7f3cf00745c0 2024-09-05T12:11:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 msgr2=0x7f3cd40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 0x7f3cd40a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8000c00 tx=0x7f3cd8005b50 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 msgr2=0x7f3cf019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9f0 tx=0x7f3ce000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 shutdown_connections 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f3cd40a2c00 0x7f3cd40a50c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cf010e260 0x7f3cf019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cf0074ee0 0x7f3cf019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 --2- 172.21.15.59:0/2354189966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf00745c0 0x7f3cf019a4a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.417+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 >> 172.21.15.59:0/2354189966 conn(0x7f3cf00fd740 msgr2=0x7f3cf00fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.418+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 shutdown_connections 2024-09-05T12:11:53.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:53.418+0000 7f3cf5c00640 1 -- 172.21.15.59:0/2354189966 wait complete. 2024-09-05T12:11:54.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:53 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2354189966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:54.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:11:54.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:11:54.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:53 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:11:54.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:11:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:53 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2354189966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:11:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:11:54.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:53 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:11:54.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:53 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2354189966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:11:54.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:11:54.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:11:54.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:53 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:11:54.533 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 -- 172.21.15.59:0/2942890633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 msgr2=0x7fbba8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 --2- 172.21.15.59:0/2942890633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba8108830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009900 tx=0x7fbb9402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 -- 172.21.15.59:0/2942890633 shutdown_connections 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 --2- 172.21.15.59:0/2942890633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 0x7fbba8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 --2- 172.21.15.59:0/2942890633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba8108830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 --2- 172.21.15.59:0/2942890633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 -- 172.21.15.59:0/2942890633 >> 172.21.15.59:0/2942890633 conn(0x7fbba80ff910 msgr2=0x7fbba8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 -- 172.21.15.59:0/2942890633 shutdown_connections 2024-09-05T12:11:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.071+0000 7fbbae600640 1 -- 172.21.15.59:0/2942890633 wait complete. 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 Processor -- start 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 -- start start 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 0x7fbba819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8114010 con 0x7fbba8108f60 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113e90 con 0x7fbba81047e0 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.073+0000 7fbbae600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8114190 con 0x7fbba8103e30 2024-09-05T12:11:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbbad200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbbad200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43714/0 (socket says 172.21.15.59:43714) 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46412/0 (socket says 172.21.15.59:46412) 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbbad200640 1 -- 172.21.15.59:0/1333698461 learned_addr learned my addr 172.21.15.59:0/1333698461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbbadc00640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 0x7fbba819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 -- 172.21.15.59:0/1333698461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 msgr2=0x7fbba819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 -- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 msgr2=0x7fbba819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 0x7fbba819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.074+0000 7fbba7e00640 1 -- 172.21.15.59:0/1333698461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba819f420 con 0x7fbba81047e0 2024-09-05T12:11:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbba7e00640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb9402a910 tx=0x7fbb94003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbba5600640 1 -- 172.21.15.59:0/1333698461 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9403f070 con 0x7fbba81047e0 2024-09-05T12:11:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbba5600640 1 -- 172.21.15.59:0/1333698461 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb94031c90 con 0x7fbba81047e0 2024-09-05T12:11:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbba5600640 1 -- 172.21.15.59:0/1333698461 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94035030 con 0x7fbba81047e0 2024-09-05T12:11:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba819f6b0 con 0x7fbba81047e0 2024-09-05T12:11:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.075+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a6ff0 con 0x7fbba81047e0 2024-09-05T12:11:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.078+0000 7fbba5600640 1 -- 172.21.15.59:0/1333698461 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb94042070 con 0x7fbba81047e0 2024-09-05T12:11:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.078+0000 7fbbb2689740 1 Processor -- start 2024-09-05T12:11:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 -- start start 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e076799410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e0766f23e0 con 0x55e07678be40 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e076699d80 con 0x55e076797020 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbb2689740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e0766ee3a0 con 0x55e07678c760 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbadc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbadc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58930/0 (socket says 172.21.15.59:58930) 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbadc00640 1 -- 172.21.15.59:0/4078602823 learned_addr learned my addr 172.21.15.59:0/4078602823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.079+0000 7fbbad200640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbba7e00640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e076799410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 -- 172.21.15.59:0/4078602823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 msgr2=0x55e076796ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076796ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 -- 172.21.15.59:0/4078602823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 msgr2=0x55e076799410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e076799410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 -- 172.21.15.59:0/4078602823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0766b8ae0 con 0x55e07678be40 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbba7e00640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e076799410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbba5600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 0x7fbb900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbba5600640 1 -- 172.21.15.59:0/1333698461 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb940efda0 con 0x7fbba81047e0 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbadc00640 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009f50 tx=0x7fbba00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29454d7664e45b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.080+0000 7fbbad200640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 0x7fbb900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbb9a000640 1 -- 172.21.15.59:0/4078602823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004d20 con 0x55e07678be40 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbb9a000640 1 -- 172.21.15.59:0/4078602823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba0004050 con 0x55e07678be40 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbb9a000640 1 -- 172.21.15.59:0/4078602823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0005280 con 0x55e07678be40 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbbb2689740 1 -- 172.21.15.59:0/4078602823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 msgr2=0x55e07678c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbbb2689740 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009f50 tx=0x7fbba00118d0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.081+0000 7fbbad200640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 0x7fbb900a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009640 tx=0x7fbb9c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 -- 172.21.15.59:0/4078602823 shutdown_connections 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e076799410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 --2- 172.21.15.59:0/4078602823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678c220 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 -- 172.21.15.59:0/4078602823 >> 172.21.15.59:0/4078602823 conn(0x55e0767883d0 msgr2=0x55e0767887b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 -- 172.21.15.59:0/4078602823 shutdown_connections 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 -- 172.21.15.59:0/4078602823 wait complete. 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 -- start start 2024-09-05T12:11:55.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.082+0000 7fbbb2689740 1 Processor -- start 2024-09-05T12:11:55.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e07678be40 0x55e076793fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e076797020 0x55e076794a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e076799bf0 con 0x55e07678c760 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e0766ad0f0 con 0x55e07678be40 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e076799d70 con 0x55e076797020 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbb2689740 10 client.0 _ll_register_callbacks cb 0x55e076694bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58946/0 (socket says 172.21.15.59:58946) 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbba7e00640 1 -- 172.21.15.59:0/335148730 learned_addr learned my addr 172.21.15.59:0/335148730 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbadc00640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e076797020 0x55e076794a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.083+0000 7fbbad200640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e07678be40 0x55e076793fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 -- 172.21.15.59:0/335148730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e076797020 msgr2=0x55e076794a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e076797020 0x55e076794a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 -- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e07678be40 msgr2=0x55e076793fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e07678be40 0x55e076793fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 -- 172.21.15.59:0/335148730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0764633c0 con 0x55e07678c760 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbba7e00640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b8c10 tx=0x7fbb94031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f3a056ab9f2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbb8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94049070 con 0x55e07678c760 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb940b88b0 con 0x55e07678c760 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba0004050 con 0x55e07678c760 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb940d5050 con 0x55e07678c760 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbbb2689740 10 client.17661 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:55.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.084+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0767a5cf0 con 0x55e07678c760 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba0004050 con 0x55e07678c760 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb940dc0b0 con 0x55e07678c760 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 10 client.17661 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 20 client.17661 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0767a70e0 0x55e0767a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 4 client.17661 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0766ff950 con 0x55e0767a70e0 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.085+0000 7fbbad200640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0767a70e0 0x55e0767a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.086+0000 7fbbad200640 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0767a70e0 0x55e0767a9530 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce548e1ee937917 server_cookie=78b2ca19611b3bec in_seq=0 out_seq=0 2024-09-05T12:11:55.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.086+0000 7fbb8ea00640 10 client.17661 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:55.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbb8ea00640 1 -- 172.21.15.59:0/335148730 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0766ff950 con 0x55e0767a70e0 2024-09-05T12:11:55.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbb8ea00640 10 client.17661 handle_command_reply: tid=0 2024-09-05T12:11:55.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 client.17661 shutdown 2024-09-05T12:11:55.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 msgr2=0x55e076794520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b8c10 tx=0x7fbb94031490 comp rx=0 tx=0).stop 2024-09-05T12:11:55.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 shutdown_connections 2024-09-05T12:11:55.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e076797020 0x55e076794a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0767a70e0 0x55e0767a9530 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678c760 0x55e076794520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 --2- 172.21.15.59:0/335148730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e07678be40 0x55e076793fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbb8ea00640 0 client.17661 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 >> 172.21.15.59:0/335148730 conn(0x55e0767883d0 msgr2=0x55e07678e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 shutdown_connections 2024-09-05T12:11:55.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.087+0000 7fbbb2689740 1 -- 172.21.15.59:0/335148730 wait complete. 2024-09-05T12:11:55.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.088+0000 7fbbb2689740 20 client.17661 trim_cache size 0 max 16384 2024-09-05T12:11:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.101+0000 7fbbb2689740 1 Processor -- start 2024-09-05T12:11:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.101+0000 7fbbb2689740 1 -- start start 2024-09-05T12:11:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076795f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e0767aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81aeec0 con 0x55e07678be40 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81b1740 con 0x55e076797020 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbb2689740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81af530 con 0x55e07678c760 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbadc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e0767aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58950/0 (socket says 172.21.15.59:58950) 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 -- 172.21.15.59:0/2015977175 learned_addr learned my addr 172.21.15.59:0/2015977175 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbba7e00640 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076795f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 -- 172.21.15.59:0/2015977175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 msgr2=0x55e076795f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076795f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 -- 172.21.15.59:0/2015977175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 msgr2=0x55e0767aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e0767aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.102+0000 7fbbad200640 1 -- 172.21.15.59:0/2015977175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0767a9bb0 con 0x55e07678be40 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.103+0000 7fbbadc00640 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e0767aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.103+0000 7fbbad200640 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c01d680 tx=0x7fbb9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5974502560a3bdb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.103+0000 7fbb99600640 1 -- 172.21.15.59:0/2015977175 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c005020 con 0x55e07678be40 2024-09-05T12:11:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.103+0000 7fbb99600640 1 -- 172.21.15.59:0/2015977175 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c0047a0 con 0x55e07678be40 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.103+0000 7fbb99600640 1 -- 172.21.15.59:0/2015977175 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c02a670 con 0x55e07678be40 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- 172.21.15.59:0/2015977175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 msgr2=0x55e07678e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c01d680 tx=0x7fbb9c019000 comp rx=0 tx=0).stop 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- 172.21.15.59:0/2015977175 shutdown_connections 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e0767aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e076795f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 --2- 172.21.15.59:0/2015977175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07678e8a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- 172.21.15.59:0/2015977175 >> 172.21.15.59:0/2015977175 conn(0x55e0767883d0 msgr2=0x55e076795840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- 172.21.15.59:0/2015977175 shutdown_connections 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- 172.21.15.59:0/2015977175 wait complete. 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.104+0000 7fbbb2689740 1 -- start start 2024-09-05T12:11:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 Processor -- start 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e07679b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e07679b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81afc00 con 0x55e07678be40 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81b1070 con 0x55e076797020 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81b02d0 con 0x55e07678c760 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbb2689740 10 client.0 _ll_register_callbacks cb 0x7fbb94009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbad200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbad200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58954/0 (socket says 172.21.15.59:58954) 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.105+0000 7fbbad200640 1 -- 172.21.15.59:0/735507232 learned_addr learned my addr 172.21.15.59:0/735507232 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e07679b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbbadc00640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e07679b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 -- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 msgr2=0x55e07679b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e07679b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 -- 172.21.15.59:0/735507232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 msgr2=0x55e07679af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 -- 172.21.15.59:0/735507232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e076811010 con 0x55e07678c760 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbbad200640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbba7e00640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e07679b4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006010 tx=0x7fbb940f0020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c912e2e64994a739 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbb8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:55.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.106+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94049070 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c0047a0 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb940c4970 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb940bbba0 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 10 client.27592 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0768122d0 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c0047a0 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb940bc4a0 con 0x55e07678c760 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 10 client.27592 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 20 client.27592 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.107+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0768128b0 0x55e076814ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.108+0000 7fbbb2689740 4 client.27592 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.108+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0766f9e60 con 0x55e0768128b0 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.108+0000 7fbbad200640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0768128b0 0x55e076814ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.108+0000 7fbbad200640 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0768128b0 0x55e076814ca0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a37dbaaa9ac9f8 server_cookie=848f6be735aeb94a in_seq=0 out_seq=0 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.108+0000 7fbb8ea00640 10 client.27592 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbb8ea00640 1 -- 172.21.15.59:0/735507232 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0766f9e60 con 0x55e0768128b0 2024-09-05T12:11:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbb8ea00640 10 client.27592 handle_command_reply: tid=0 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 client.27592 shutdown 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 msgr2=0x55e07679b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e07679b4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006010 tx=0x7fbb940f0020 comp rx=0 tx=0).stop 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 shutdown_connections 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e076797020 0x55e07679b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07678c760 0x55e07679b4a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e0768128b0 0x55e076814ca0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 --2- 172.21.15.59:0/735507232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e07678be40 0x55e07679af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 >> 172.21.15.59:0/735507232 conn(0x55e0767883d0 msgr2=0x55e076796060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbb8ea00640 0 client.27592 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 shutdown_connections 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 1 -- 172.21.15.59:0/735507232 wait complete. 2024-09-05T12:11:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.109+0000 7fbbb2689740 20 client.27592 trim_cache size 0 max 16384 2024-09-05T12:11:55.117 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:11:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 msgr2=0x7fbb900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 0x7fbb900a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009640 tx=0x7fbb9c005f90 comp rx=0 tx=0).stop 2024-09-05T12:11:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 msgr2=0x7fbba819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb9402a910 tx=0x7fbb94003b20 comp rx=0 tx=0).stop 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 shutdown_connections 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb900a2b80 0x7fbb900a5040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba8108f60 0x7fbba819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbba81047e0 0x7fbba819a9b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 --2- 172.21.15.59:0/1333698461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbba8103e30 0x7fbba819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 >> 172.21.15.59:0/1333698461 conn(0x7fbba80ff910 msgr2=0x7fbba8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 shutdown_connections 2024-09-05T12:11:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:55.110+0000 7fbbae600640 1 -- 172.21.15.59:0/1333698461 wait complete. 2024-09-05T12:11:55.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:54 smithi089 ceph-mon[34499]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:55.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:54 smithi059 ceph-mon[30025]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:55.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:54 smithi189 ceph-mon[34772]: pgmap v821: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:11:56.806 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:11:56.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:11:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:56 smithi089 ceph-mon[34499]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:57.196 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:57.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:56 smithi059 ceph-mon[30025]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:56 smithi189 ceph-mon[34772]: pgmap v822: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 -- 172.21.15.59:0/4198059053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690074e30 msgr2=0x7f669010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 --2- 172.21.15.59:0/4198059053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690074e30 0x7f669010df10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802a480 comp rx=0 tx=0).stop 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 -- 172.21.15.59:0/4198059053 shutdown_connections 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 --2- 172.21.15.59:0/4198059053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f669010e450 0x7f6690110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 --2- 172.21.15.59:0/4198059053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6690074e30 0x7f669010df10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 --2- 172.21.15.59:0/4198059053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f66900748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 -- 172.21.15.59:0/4198059053 >> 172.21.15.59:0/4198059053 conn(0x7f66900fd8b0 msgr2=0x7f66900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 -- 172.21.15.59:0/4198059053 shutdown_connections 2024-09-05T12:11:57.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.774+0000 7f6694a00640 1 -- 172.21.15.59:0/4198059053 wait complete. 2024-09-05T12:11:57.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 Processor -- start 2024-09-05T12:11:57.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 -- start start 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6690074e30 0x7f669019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 0x7f669019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.775+0000 7f6694a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901141c0 con 0x7f669010e450 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f6694a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6690114040 con 0x7f6690074510 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f6694a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6690114340 con 0x7f6690074e30 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46484/0 (socket says 172.21.15.59:46484) 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 -- 172.21.15.59:0/3822065313 learned_addr learned my addr 172.21.15.59:0/3822065313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668e800640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 0x7f669019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f6685400640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6690074e30 0x7f669019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 -- 172.21.15.59:0/3822065313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6690074e30 msgr2=0x7f669019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6690074e30 0x7f669019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 -- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 msgr2=0x7f669019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 0x7f669019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668de00640 1 -- 172.21.15.59:0/3822065313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669019e6b0 con 0x7f6690074510 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.776+0000 7f668e800640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 0x7f669019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f668de00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668000e9f0 tx=0x7f668000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f6687400640 1 -- 172.21.15.59:0/3822065313 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668000cd80 con 0x7f6690074510 2024-09-05T12:11:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f6687400640 1 -- 172.21.15.59:0/3822065313 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6680010910 con 0x7f6690074510 2024-09-05T12:11:57.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669019e9a0 con 0x7f6690074510 2024-09-05T12:11:57.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f6687400640 1 -- 172.21.15.59:0/3822065313 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680018990 con 0x7f6690074510 2024-09-05T12:11:57.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.777+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66901a7310 con 0x7f6690074510 2024-09-05T12:11:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.780+0000 7f669741b740 1 Processor -- start 2024-09-05T12:11:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.780+0000 7f6687400640 1 -- 172.21.15.59:0/3822065313 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6680010ac0 con 0x7f6690074510 2024-09-05T12:11:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.780+0000 7f669741b740 1 -- start start 2024-09-05T12:11:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b68ca0 0x55c856b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c856ac4290 con 0x55c856b5dac0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c856a6bb40 con 0x55c856b5e3e0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f669741b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c856ac0250 con 0x55c856b68ca0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58980/0 (socket says 172.21.15.59:58980) 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 -- 172.21.15.59:0/2392329951 learned_addr learned my addr 172.21.15.59:0/2392329951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f6685400640 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b68ca0 0x55c856b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 -- 172.21.15.59:0/2392329951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b68ca0 msgr2=0x55c856b6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b68ca0 0x55c856b6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 -- 172.21.15.59:0/2392329951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 msgr2=0x55c856b68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.781+0000 7f668e800640 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f668e800640 1 -- 172.21.15.59:0/2392329951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8568356a0 con 0x55c856b5dac0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f668e800640 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fc0 tx=0x7f6688010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e25116bb11a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f6687400640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 0x7f66740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f667f000640 1 -- 172.21.15.59:0/2392329951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688011700 con 0x55c856b5dac0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f667f000640 1 -- 172.21.15.59:0/2392329951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688011d80 con 0x55c856b5dac0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f667f000640 1 -- 172.21.15.59:0/2392329951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668801a380 con 0x55c856b5dac0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f6687400640 1 -- 172.21.15.59:0/3822065313 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f66800cf770 con 0x7f6690074510 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.782+0000 7f668de00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 0x7f66740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 -- 172.21.15.59:0/2392329951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 msgr2=0x55c856b5dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fc0 tx=0x7f6688010860 comp rx=0 tx=0).stop 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 -- 172.21.15.59:0/2392329951 shutdown_connections 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f668de00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 0x7f66740a50c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f668009b8c0 tx=0x7f66800d5550 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b68ca0 0x55c856b6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 --2- 172.21.15.59:0/2392329951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b5dea0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 -- 172.21.15.59:0/2392329951 >> 172.21.15.59:0/2392329951 conn(0x55c856b5a050 msgr2=0x55c856b5a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 -- 172.21.15.59:0/2392329951 shutdown_connections 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.783+0000 7f669741b740 1 -- 172.21.15.59:0/2392329951 wait complete. 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 -- start start 2024-09-05T12:11:57.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 Processor -- start 2024-09-05T12:11:57.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c856b6b910 con 0x55c856b68ca0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c856a7efa0 con 0x55c856b5e3e0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c856b6ba90 con 0x55c856b5dac0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.784+0000 7f669741b740 10 client.0 _ll_register_callbacks cb 0x55c8567d2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f668e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f668e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46498/0 (socket says 172.21.15.59:46498) 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 -- 172.21.15.59:0/4055632650 learned_addr learned my addr 172.21.15.59:0/4055632650 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 -- 172.21.15.59:0/4055632650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 msgr2=0x55c856b68610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b68610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 -- 172.21.15.59:0/4055632650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 msgr2=0x55c856b65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b65e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 -- 172.21.15.59:0/4055632650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c856b66650 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f668e800640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6685400640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f6678004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89eb645fd26a3b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.785+0000 7f6673400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035040 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6688011d80 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6678005450 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f667804f050 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f669741b740 10 client.27680 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c856b77850 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.786+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688011d80 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6678003a00 con 0x55c856b5e3e0 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 10 client.27680 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:57.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 20 client.27680 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856b78d10 0x55c856b7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 4 client.27680 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c856ad1800 con 0x55c856b78d10 2024-09-05T12:11:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.787+0000 7f668e800640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856b78d10 0x55c856b7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.788+0000 7f668e800640 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856b78d10 0x55c856b7b150 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8875f7e24ff4a4d8 server_cookie=8f7b9cd8bb007ecd in_seq=0 out_seq=0 2024-09-05T12:11:57.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.788+0000 7f6673400640 10 client.27680 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:57.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.788+0000 7f6673400640 1 -- 172.21.15.59:0/4055632650 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c856ad1800 con 0x55c856b78d10 2024-09-05T12:11:57.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.788+0000 7f6673400640 10 client.27680 handle_command_reply: tid=0 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 client.27680 shutdown 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 msgr2=0x55c856b658c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f6678004440 comp rx=0 tx=0).stop 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 shutdown_connections 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856b78d10 0x55c856b7b150 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b658c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 --2- 172.21.15.59:0/4055632650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b68610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f6673400640 0 client.27680 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 >> 172.21.15.59:0/4055632650 conn(0x55c856b5a050 msgr2=0x55c856b591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 shutdown_connections 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 1 -- 172.21.15.59:0/4055632650 wait complete. 2024-09-05T12:11:57.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.789+0000 7f669741b740 20 client.27680 trim_cache size 0 max 16384 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.807+0000 7f669741b740 1 Processor -- start 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.807+0000 7f669741b740 1 -- start start 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.807+0000 7f669741b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f669741b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f669741b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b80f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f669741b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901af080 con 0x55c856b68ca0 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f669741b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b1900 con 0x55c856b5e3e0 2024-09-05T12:11:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f669741b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66901af6f0 con 0x55c856b5dac0 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b80f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f6685400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43812/0 (socket says 172.21.15.59:43812) 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 -- 172.21.15.59:0/1649323386 learned_addr learned my addr 172.21.15.59:0/1649323386 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f6685400640 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 -- 172.21.15.59:0/1649323386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 msgr2=0x55c856b80a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 -- 172.21.15.59:0/1649323386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 msgr2=0x55c856b80f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b80f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f668e800640 1 -- 172.21.15.59:0/1649323386 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c856b7b7d0 con 0x55c856b5dac0 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.808+0000 7f6685400640 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.809+0000 7f668e800640 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f668801c9c0 tx=0x7f6688010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ee0875b1233a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.809+0000 7f667e600640 1 -- 172.21.15.59:0/1649323386 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688026040 con 0x55c856b5dac0 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.809+0000 7f667e600640 1 -- 172.21.15.59:0/1649323386 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688031070 con 0x55c856b5dac0 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.809+0000 7f667e600640 1 -- 172.21.15.59:0/1649323386 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688011500 con 0x55c856b5dac0 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- 172.21.15.59:0/1649323386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 msgr2=0x55c856b648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f668801c9c0 tx=0x7f6688010fd0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- 172.21.15.59:0/1649323386 shutdown_connections 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b68ca0 0x55c856b80f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b5e3e0 0x55c856b80a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 --2- 172.21.15.59:0/1649323386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5dac0 0x55c856b648a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- 172.21.15.59:0/1649323386 >> 172.21.15.59:0/1649323386 conn(0x55c856b5a050 msgr2=0x55c856b64190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- 172.21.15.59:0/1649323386 shutdown_connections 2024-09-05T12:11:57.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- 172.21.15.59:0/1649323386 wait complete. 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 -- start start 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.810+0000 7f669741b740 1 Processor -- start 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b62910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b68ca0 0x55c856b63390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66901afdc0 con 0x55c856b5dac0 2024-09-05T12:11:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b1230 con 0x55c856b68ca0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66901b0490 con 0x55c856b5e3e0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f669741b740 10 client.0 _ll_register_callbacks cb 0x55c856b66820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f668e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b62910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f6685400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f6685400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43814/0 (socket says 172.21.15.59:43814) 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f6685400640 1 -- 172.21.15.59:0/4184804009 learned_addr learned my addr 172.21.15.59:0/4184804009 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.811+0000 7f668e800640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b68ca0 0x55c856b63390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 -- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b68ca0 msgr2=0x55c856b63390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b68ca0 0x55c856b63390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 -- 172.21.15.59:0/4184804009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 msgr2=0x55c856b62910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b62910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 -- 172.21.15.59:0/4184804009 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c856be2a40 con 0x55c856b5e3e0 2024-09-05T12:11:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f668e800640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b62910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6685400640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f667802a9a0 tx=0x7f66780041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b2a228169e160b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6673400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035070 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6678005460 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.812+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6688031070 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6678031880 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 10 client.27610 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c856be3d60 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688031070 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f667805b030 con 0x55c856b5e3e0 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 10 client.27610 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 10 client.27610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 20 client.27610 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856be44d0 0x55c856be68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 4 client.27610 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:11:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.813+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c856acbd10 con 0x55c856be44d0 2024-09-05T12:11:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.814+0000 7f668e800640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856be44d0 0x55c856be68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.814+0000 7f668e800640 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856be44d0 0x55c856be68f0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d083fc41d21d95 server_cookie=3ada609bc29080c3 in_seq=0 out_seq=0 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.814+0000 7f6673400640 10 client.27610 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.815+0000 7f6673400640 1 -- 172.21.15.59:0/4184804009 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c856acbd10 con 0x55c856be44d0 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.815+0000 7f6673400640 10 client.27610 handle_command_reply: tid=0 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.815+0000 7f669741b740 1 client.27610 shutdown 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 msgr2=0x55c856b62e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f667802a9a0 tx=0x7f66780041a0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 shutdown_connections 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c856b68ca0 0x55c856b63390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c856b5e3e0 0x55c856b62e50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c856be44d0 0x55c856be68f0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 --2- 172.21.15.59:0/4184804009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c856b5dac0 0x55c856b62910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 >> 172.21.15.59:0/4184804009 conn(0x55c856b5a050 msgr2=0x55c856b649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f6673400640 0 client.27610 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 shutdown_connections 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 1 -- 172.21.15.59:0/4184804009 wait complete. 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.816+0000 7f669741b740 20 client.27610 trim_cache size 0 max 16384 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:57.820 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:11:57.821 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "43b9e75a-1e31-4fc7-81cd-65e6fe8da6c0", 2024-09-05T12:11:57.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:11:57.821 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:57.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 msgr2=0x7f66740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 0x7f66740a50c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f668009b8c0 tx=0x7f66800d5550 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 msgr2=0x7f669019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668000e9f0 tx=0x7f668000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 shutdown_connections 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f66740a2c00 0x7f66740a50c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f669010e450 0x7f669019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6690074e30 0x7f669019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 --2- 172.21.15.59:0/3822065313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6690074510 0x7f669019a760 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668000e9f0 tx=0x7f668000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 >> 172.21.15.59:0/3822065313 conn(0x7f66900fd8b0 msgr2=0x7f66900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 shutdown_connections 2024-09-05T12:11:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:57.817+0000 7f6694a00640 1 -- 172.21.15.59:0/3822065313 wait complete. 2024-09-05T12:11:58.553 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:11:58.932 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:11:59.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:58 smithi059 ceph-mon[30025]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:58 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:11:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:58 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:58 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:11:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:11:58 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:58 smithi189 ceph-mon[34772]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:58 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:11:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:58 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:58 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:11:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:11:58 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:58 smithi089 ceph-mon[34499]: pgmap v823: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:11:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:58 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:11:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:58 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:58 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:11:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:11:58 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:11:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 -- 172.21.15.59:0/3287375207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10076720 msgr2=0x7fea10076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 --2- 172.21.15.59:0/3287375207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10076720 0x7fea10076b00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fea00009900 tx=0x7fea0002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 -- 172.21.15.59:0/3287375207 shutdown_connections 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 --2- 172.21.15.59:0/3287375207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea10114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 --2- 172.21.15.59:0/3287375207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10077040 0x7fea101124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 --2- 172.21.15.59:0/3287375207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10076720 0x7fea10076b00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 -- 172.21.15.59:0/3287375207 >> 172.21.15.59:0/3287375207 conn(0x7fea100ffa90 msgr2=0x7fea10101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 -- 172.21.15.59:0/3287375207 shutdown_connections 2024-09-05T12:11:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.487+0000 7fea16e00640 1 -- 172.21.15.59:0/3287375207 wait complete. 2024-09-05T12:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 Processor -- start 2024-09-05T12:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 -- start start 2024-09-05T12:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 0x7fea101a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10077040 0x7fea101a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea1019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea10118610 con 0x7fea10077040 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea10118490 con 0x7fea10076720 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.488+0000 7fea16e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea10118790 con 0x7fea10112a30 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea14e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea1019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea14e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea1019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43842/0 (socket says 172.21.15.59:43842) 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea14e00640 1 -- 172.21.15.59:0/1943785877 learned_addr learned my addr 172.21.15.59:0/1943785877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 0x7fea101a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0f400640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10077040 0x7fea101a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 -- 172.21.15.59:0/1943785877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 msgr2=0x7fea1019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea1019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 -- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10077040 msgr2=0x7fea101a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10077040 0x7fea101a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.489+0000 7fea0fe00640 1 -- 172.21.15.59:0/1943785877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1019efd0 con 0x7fea10076720 2024-09-05T12:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 0x7fea101a46e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fea0002a910 tx=0x7fea00003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea0cc00640 1 -- 172.21.15.59:0/1943785877 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0003f070 con 0x7fea10076720 2024-09-05T12:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1019f260 con 0x7fea10076720 2024-09-05T12:11:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea0cc00640 1 -- 172.21.15.59:0/1943785877 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea00031c90 con 0x7fea10076720 2024-09-05T12:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea0cc00640 1 -- 172.21.15.59:0/1943785877 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00035030 con 0x7fea10076720 2024-09-05T12:11:59.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.490+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101ab5b0 con 0x7fea10076720 2024-09-05T12:11:59.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.493+0000 7fea19a69740 1 Processor -- start 2024-09-05T12:11:59.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.493+0000 7fea19a69740 1 -- start start 2024-09-05T12:11:59.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.493+0000 7fea19a69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.493+0000 7fea19a69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b3f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea19a69740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea19a69740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571c2a9b3e0 con 0x5571c2b3fd70 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea19a69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5571c2a42d80 con 0x5571c2b34b90 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea19a69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571c2a973a0 con 0x5571c2b354b0 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea0cc00640 1 -- 172.21.15.59:0/1943785877 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea00042070 con 0x7fea10076720 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46546/0 (socket says 172.21.15.59:46546) 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59046/0 (socket says 172.21.15.59:59046) 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 -- 172.21.15.59:0/867366436 learned_addr learned my addr 172.21.15.59:0/867366436 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea0fe00640 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b3f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 -- 172.21.15.59:0/867366436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 msgr2=0x5571c2b3f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b3f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 -- 172.21.15.59:0/867366436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 msgr2=0x5571c2b42160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea14e00640 1 -- 172.21.15.59:0/867366436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c2a61ae0 con 0x5571c2b34b90 2024-09-05T12:11:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.494+0000 7fea0f400640 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.495+0000 7fea14e00640 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fea08009fb0 tx=0x7fea080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6ccb2dc16ba72f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.495+0000 7fea05600640 1 -- 172.21.15.59:0/867366436 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08004b20 con 0x5571c2b34b90 2024-09-05T12:11:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.495+0000 7fea05600640 1 -- 172.21.15.59:0/867366436 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08004cc0 con 0x5571c2b34b90 2024-09-05T12:11:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.495+0000 7fea05600640 1 -- 172.21.15.59:0/867366436 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08005480 con 0x5571c2b34b90 2024-09-05T12:11:59.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 -- 172.21.15.59:0/867366436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 msgr2=0x5571c2b34f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fea08009fb0 tx=0x7fea080118d0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea0cc00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 -- 172.21.15.59:0/867366436 shutdown_connections 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b42160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea0cc00640 1 -- 172.21.15.59:0/1943785877 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea000efa30 con 0x7fea10076720 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b3f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 --2- 172.21.15.59:0/867366436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b34f70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 -- 172.21.15.59:0/867366436 >> 172.21.15.59:0/867366436 conn(0x5571c2b31120 msgr2=0x5571c2b31500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea14e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 -- 172.21.15.59:0/867366436 shutdown_connections 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.496+0000 7fea19a69740 1 -- 172.21.15.59:0/867366436 wait complete. 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.497+0000 7fea19a69740 1 -- start start 2024-09-05T12:11:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.497+0000 7fea14e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea08009dc0 tx=0x7fea08005bd0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.497+0000 7fea19a69740 1 Processor -- start 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 0x5571c2b3d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571c2b429e0 con 0x5571c2b3fd70 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5571c2a560f0 con 0x5571c2b354b0 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571c2b42b60 con 0x5571c2b34b90 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea19a69740 10 client.0 _ll_register_callbacks cb 0x5571c2a479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43848/0 (socket says 172.21.15.59:43848) 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 -- 172.21.15.59:0/1624492263 learned_addr learned my addr 172.21.15.59:0/1624492263 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0f400640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 0x5571c2b3d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea14e00640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 -- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 msgr2=0x5571c2b3d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 0x5571c2b3d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 -- 172.21.15.59:0/1624492263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 msgr2=0x5571c2b3d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b3d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0fe00640 1 -- 172.21.15.59:0/1624492263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c280c3c0 con 0x5571c2b34b90 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea14e00640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.498+0000 7fea0f400640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 0x5571c2b3d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fea0fe00640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea000b88a0 tx=0x7fea00031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e23ee04d9378f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fe9fa000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00049070 con 0x5571c2b34b90 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea000b8540 con 0x5571c2b34b90 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea08004cc0 con 0x5571c2b34b90 2024-09-05T12:11:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fea000bbed0 con 0x5571c2b34b90 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fea19a69740 10 client.27622 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.499+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571c2b429e0 con 0x5571c2b34b90 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08004cc0 con 0x5571c2b34b90 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fea000c4600 con 0x5571c2b34b90 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 10 client.27622 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 10 client.27622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 20 client.27622 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2b4fa80 0x5571c2b51f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 4 client.27622 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c2aa8950 con 0x5571c2b4fa80 2024-09-05T12:11:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.500+0000 7fea0f400640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2b4fa80 0x5571c2b51f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.501+0000 7fea0f400640 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2b4fa80 0x5571c2b51f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1714c54dccf84be0 server_cookie=8f5d2a021d6f892d in_seq=0 out_seq=0 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.501+0000 7fe9fa000640 10 client.27622 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fe9fa000640 1 -- 172.21.15.59:0/1624492263 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c2aa8950 con 0x5571c2b4fa80 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fe9fa000640 10 client.27622 handle_command_reply: tid=0 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 client.27622 shutdown 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 msgr2=0x5571c2b3cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea000b88a0 tx=0x7fea00031490 comp rx=0 tx=0).stop 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 shutdown_connections 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2b4fa80 0x5571c2b51f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b354b0 0x5571c2b3d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 --2- 172.21.15.59:0/1624492263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b34b90 0x5571c2b3cc20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fe9fa000640 0 client.27622 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.502+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 >> 172.21.15.59:0/1624492263 conn(0x5571c2b31120 msgr2=0x5571c2b36e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.503+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 shutdown_connections 2024-09-05T12:11:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.503+0000 7fea19a69740 1 -- 172.21.15.59:0/1624492263 wait complete. 2024-09-05T12:11:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.503+0000 7fea19a69740 20 client.27622 trim_cache size 0 max 16384 2024-09-05T12:11:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 Processor -- start 2024-09-05T12:11:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 -- start start 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 0x5571c2b577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b3fd70 0x5571c2b57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b3500 con 0x5571c2b354b0 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b5d80 con 0x5571c2b34b90 2024-09-05T12:11:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea19a69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b3b70 con 0x5571c2b3fd70 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 0x5571c2b577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 0x5571c2b577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59052/0 (socket says 172.21.15.59:59052) 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea0f400640 1 -- 172.21.15.59:0/2167728084 learned_addr learned my addr 172.21.15.59:0/2167728084 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea0fe00640 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.521+0000 7fea14e00640 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b3fd70 0x5571c2b57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 -- 172.21.15.59:0/2167728084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b3fd70 msgr2=0x5571c2b57d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b3fd70 0x5571c2b57d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 -- 172.21.15.59:0/2167728084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 msgr2=0x5571c2b577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 0x5571c2b577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 -- 172.21.15.59:0/2167728084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c2b31db0 con 0x5571c2b34b90 2024-09-05T12:11:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea0fe00640 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b36cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea000ba180 tx=0x7fea0002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f708ac5758b66753 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea04c00640 1 -- 172.21.15.59:0/2167728084 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00049070 con 0x5571c2b34b90 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea04c00640 1 -- 172.21.15.59:0/2167728084 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000c4a90 con 0x5571c2b34b90 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.522+0000 7fea04c00640 1 -- 172.21.15.59:0/2167728084 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000c4600 con 0x5571c2b34b90 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 -- 172.21.15.59:0/2167728084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 msgr2=0x5571c2b36cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b36cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea000ba180 tx=0x7fea0002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 -- 172.21.15.59:0/2167728084 shutdown_connections 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b3fd70 0x5571c2b57d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b354b0 0x5571c2b577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 --2- 172.21.15.59:0/2167728084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b36cf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 -- 172.21.15.59:0/2167728084 >> 172.21.15.59:0/2167728084 conn(0x5571c2b31120 msgr2=0x5571c2b365e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 -- 172.21.15.59:0/2167728084 shutdown_connections 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.523+0000 7fea19a69740 1 -- 172.21.15.59:0/2167728084 wait complete. 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 -- start start 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 Processor -- start 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b4240 con 0x5571c2b3fd70 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b56b0 con 0x5571c2b34b90 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b4910 con 0x5571c2b354b0 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.524+0000 7fea19a69740 10 client.0 _ll_register_callbacks cb 0x7fea000b8170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:11:59.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 -- 172.21.15.59:0/17098737 learned_addr learned my addr 172.21.15.59:0/17098737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0f400640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea14e00640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 -- 172.21.15.59:0/17098737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 msgr2=0x5571c2b39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 -- 172.21.15.59:0/17098737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 msgr2=0x5571c2b39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 -- 172.21.15.59:0/17098737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c2b3a460 con 0x5571c2b34b90 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fea0fe00640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea000c4a00 tx=0x7fea0003b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd2fafcf56e6127 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fe9fa000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:11:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.525+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00049070 con 0x5571c2b34b90 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fea000bb470 con 0x5571c2b34b90 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea000c4a90 con 0x5571c2b34b90 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fea000bba10 con 0x5571c2b34b90 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 10 client.27704 fetch_fsmap learned FSMap version 21 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571c2b3a460 con 0x5571c2b34b90 2024-09-05T12:11:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000c4a90 con 0x5571c2b34b90 2024-09-05T12:11:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fea000bae00 con 0x5571c2b34b90 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 10 client.27704 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 20 client.27704 populate_metadata read hostname 'smithi059' 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2bba970 0x5571c2bbcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 4 client.27704 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.526+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571c2aa2e60 con 0x5571c2bba970 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fea0f400640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2bba970 0x5571c2bbcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fea0f400640 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2bba970 0x5571c2bbcd60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ede7bdde5d9d48 server_cookie=a47668f3a1f1a6c6 in_seq=0 out_seq=0 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fe9fa000640 10 client.27704 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fe9fa000640 1 -- 172.21.15.59:0/17098737 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5571c2aa2e60 con 0x5571c2bba970 2024-09-05T12:11:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fe9fa000640 10 client.27704 handle_command_reply: tid=0 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fea19a69740 1 client.27704 shutdown 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 msgr2=0x5571c2b483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.527+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea000c4a00 tx=0x7fea0003b690 comp rx=0 tx=0).stop 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 shutdown_connections 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5571c2bba970 0x5571c2bbcd60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c2b3fd70 0x5571c2b39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571c2b354b0 0x5571c2b39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 --2- 172.21.15.59:0/17098737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5571c2b34b90 0x5571c2b483d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fe9fa000640 0 client.27704 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 >> 172.21.15.59:0/17098737 conn(0x5571c2b31120 msgr2=0x5571c2b301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 shutdown_connections 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 1 -- 172.21.15.59:0/17098737 wait complete. 2024-09-05T12:11:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.528+0000 7fea19a69740 20 client.27704 trim_cache size 0 max 16384 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "43b9e75a-1e31-4fc7-81cd-65e6fe8da6c0": { 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "43b9e75a-1e31-4fc7-81cd-65e6fe8da6c0", 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:59.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 msgr2=0x7fe9f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 0x7fe9f40a5040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea08009dc0 tx=0x7fea08005bd0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 msgr2=0x7fea101a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 0x7fea101a46e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fea0002a910 tx=0x7fea00003b20 comp rx=0 tx=0).stop 2024-09-05T12:11:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 shutdown_connections 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9f40a2b80 0x7fe9f40a5040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea10112a30 0x7fea1019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea10077040 0x7fea101a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 --2- 172.21.15.59:0/1943785877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea10076720 0x7fea101a46e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 >> 172.21.15.59:0/1943785877 conn(0x7fea100ffa90 msgr2=0x7fea10101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 shutdown_connections 2024-09-05T12:11:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:11:59.529+0000 7fea16e00640 1 -- 172.21.15.59:0/1943785877 wait complete. 2024-09-05T12:12:00.308 INFO:tasks.cephfs.filesystem:scrub status for tag:43b9e75a-1e31-4fc7-81cd-65e6fe8da6c0 - {'path': '/', 'tag': '43b9e75a-1e31-4fc7-81cd-65e6fe8da6c0', 'options': 'recursive,force'} 2024-09-05T12:12:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:00 smithi059 ceph-mon[30025]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 4 op/s 2024-09-05T12:12:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:00 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:12:01.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:00 smithi189 ceph-mon[34772]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 4 op/s 2024-09-05T12:12:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:00 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:12:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:00 smithi089 ceph-mon[34499]: pgmap v824: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 4 op/s 2024-09-05T12:12:01.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:00 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:12:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 -- 172.21.15.59:0/1979078336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8101df0 msgr2=0x7faca81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 --2- 172.21.15.59:0/1979078336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8101df0 0x7faca81021d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac98009900 tx=0x7fac9802a410 comp rx=0 tx=0).stop 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 -- 172.21.15.59:0/1979078336 shutdown_connections 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 --2- 172.21.15.59:0/1979078336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8106f20 0x7faca810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 --2- 172.21.15.59:0/1979078336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca81027a0 0x7faca81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 --2- 172.21.15.59:0/1979078336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca8101df0 0x7faca81021d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 -- 172.21.15.59:0/1979078336 >> 172.21.15.59:0/1979078336 conn(0x7faca80fd8b0 msgr2=0x7faca80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 -- 172.21.15.59:0/1979078336 shutdown_connections 2024-09-05T12:12:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.193+0000 7facaf000640 1 -- 172.21.15.59:0/1979078336 wait complete. 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.194+0000 7facaf000640 1 Processor -- start 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.194+0000 7facaf000640 1 -- start start 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca81027a0 0x7faca8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca8106f20 0x7faca8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faca8111f80 con 0x7faca81027a0 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faca8111e00 con 0x7faca8101df0 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facaf000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faca8112100 con 0x7faca8106f20 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7faca7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7faca7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:48444/0 (socket says 172.21.15.59:48444) 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7faca7e00640 1 -- 172.21.15.59:0/2601203110 learned_addr learned my addr 172.21.15.59:0/2601203110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7faca7400640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca81027a0 0x7faca8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.195+0000 7facad000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca8106f20 0x7faca8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 -- 172.21.15.59:0/2601203110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca8106f20 msgr2=0x7faca8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca8106f20 0x7faca8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 -- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca81027a0 msgr2=0x7faca8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca81027a0 0x7faca8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 -- 172.21.15.59:0/2601203110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a68e0 con 0x7faca8101df0 2024-09-05T12:12:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca7e00640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac98006010 tx=0x7fac98004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9803f030 con 0x7faca8101df0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac98004440 con 0x7faca8101df0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a6b70 con 0x7faca8101df0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035070 con 0x7faca8101df0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.196+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81a70a0 con 0x7faca8101df0 2024-09-05T12:12:02.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.197+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca81021d0 con 0x7faca8101df0 2024-09-05T12:12:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.199+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac98042030 con 0x7faca8101df0 2024-09-05T12:12:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.201+0000 7faca4c00640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 0x7fac8c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.201+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fac980ef770 con 0x7faca8101df0 2024-09-05T12:12:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.201+0000 7faca7400640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 0x7fac8c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.202+0000 7faca7400640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 0x7fac8c0a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faca8075f80 tx=0x7fac94005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.205+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac980bbc10 con 0x7faca8101df0 2024-09-05T12:12:02.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.425+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca810a100 con 0x7faca8101df0 2024-09-05T12:12:02.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.426+0000 7faca4c00640 1 -- 172.21.15.59:0/2601203110 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fac980bff10 con 0x7faca8101df0 2024-09-05T12:12:02.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:02.428 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:02.428 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 msgr2=0x7fac8c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:02.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 0x7fac8c0a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faca8075f80 tx=0x7fac94005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:02.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 msgr2=0x7faca80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:02.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac98006010 tx=0x7fac98004040 comp rx=0 tx=0).stop 2024-09-05T12:12:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 shutdown_connections 2024-09-05T12:12:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.428+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fac8c0a2c00 0x7fac8c0a50c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faca8106f20 0x7faca8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faca81027a0 0x7faca8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 --2- 172.21.15.59:0/2601203110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faca8101df0 0x7faca80744f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 >> 172.21.15.59:0/2601203110 conn(0x7faca80fd8b0 msgr2=0x7faca80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 shutdown_connections 2024-09-05T12:12:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:02.429+0000 7facaf000640 1 -- 172.21.15.59:0/2601203110 wait complete. 2024-09-05T12:12:03.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:02 smithi059 ceph-mon[30025]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-09-05T12:12:03.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:02 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2601203110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:03.192 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 6925392e --release --await' 2024-09-05T12:12:03.192 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 6925392e --release --await 2024-09-05T12:12:03.236 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:02 smithi089 ceph-mon[34499]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-09-05T12:12:03.236 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:02 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2601203110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:02 smithi189 ceph-mon[34772]: pgmap v825: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 5 op/s 2024-09-05T12:12:03.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:02 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2601203110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:03.580 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:04.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.135+0000 7fe5c4a00640 1 -- 172.21.15.59:0/3662981885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0109120 msgr2=0x7fe5c0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.135+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/3662981885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0109120 0x7fe5c0110ca0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009ca0 tx=0x7fe5b802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.135+0000 7fe5c4a00640 1 -- 172.21.15.59:0/3662981885 shutdown_connections 2024-09-05T12:12:04.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/3662981885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0109120 0x7fe5c0110ca0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/3662981885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c01049a0 0x7fe5c01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/3662981885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c0103ff0 0x7fe5c01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 -- 172.21.15.59:0/3662981885 >> 172.21.15.59:0/3662981885 conn(0x7fe5c00ffa90 msgr2=0x7fe5c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 -- 172.21.15.59:0/3662981885 shutdown_connections 2024-09-05T12:12:04.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.136+0000 7fe5c4a00640 1 -- 172.21.15.59:0/3662981885 wait complete. 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 Processor -- start 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 -- start start 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 0x7fe5c019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 0x7fe5c019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c0109120 0x7fe5c019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0114150 con 0x7fe5c0103ff0 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0113fd0 con 0x7fe5c01049a0 2024-09-05T12:12:04.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.137+0000 7fe5c4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01142d0 con 0x7fe5c0109120 2024-09-05T12:12:04.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 0x7fe5c019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 0x7fe5c019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33028/0 (socket says 172.21.15.59:33028) 2024-09-05T12:12:04.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5bde00640 1 -- 172.21.15.59:0/2835344131 learned_addr learned my addr 172.21.15.59:0/2835344131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:04.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 0x7fe5c019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5be800640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c0109120 0x7fe5c019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 -- 172.21.15.59:0/2835344131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c0109120 msgr2=0x7fe5c019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c0109120 0x7fe5c019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 -- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 msgr2=0x7fe5c019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 0x7fe5c019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 -- 172.21.15.59:0/2835344131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c019f570 con 0x7fe5c01049a0 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.138+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 0x7fe5c019ab00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9f0 tx=0x7fe5a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.139+0000 7fe5b6a00640 1 -- 172.21.15.59:0/2835344131 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800ccc0 con 0x7fe5c01049a0 2024-09-05T12:12:04.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.139+0000 7fe5b6a00640 1 -- 172.21.15.59:0/2835344131 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a8005020 con 0x7fe5c01049a0 2024-09-05T12:12:04.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.139+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c019f860 con 0x7fe5c01049a0 2024-09-05T12:12:04.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.139+0000 7fe5b6a00640 1 -- 172.21.15.59:0/2835344131 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80049e0 con 0x7fe5c01049a0 2024-09-05T12:12:04.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.139+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c01a7140 con 0x7fe5c01049a0 2024-09-05T12:12:04.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.142+0000 7fe5b6a00640 1 -- 172.21.15.59:0/2835344131 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5a8005560 con 0x7fe5c01049a0 2024-09-05T12:12:04.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.142+0000 7fe5c73d0740 1 Processor -- start 2024-09-05T12:12:04.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.142+0000 7fe5c73d0740 1 -- start start 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d31534650 con 0x564d315ce310 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564d314dbf10 con 0x564d315cec30 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d31530610 con 0x564d315d94f0 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5be800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:48472/0 (socket says 172.21.15.59:48472) 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 -- 172.21.15.59:0/2758376123 learned_addr learned my addr 172.21.15.59:0/2758376123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5b6a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 0x7fe5a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 -- 172.21.15.59:0/2758376123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 msgr2=0x564d315db8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315db8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 -- 172.21.15.59:0/2758376123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 msgr2=0x564d315ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5bde00640 1 -- 172.21.15.59:0/2758376123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d31243580 con 0x564d315cec30 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.143+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5b6a00640 1 -- 172.21.15.59:0/2835344131 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5a80cf300 con 0x7fe5c01049a0 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5be800640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 0x7fe5a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5bde00640 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009dc0 tx=0x7fe5b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b785e632918a181 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5af000640 1 -- 172.21.15.59:0/2758376123 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001e070 con 0x564d315cec30 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5af000640 1 -- 172.21.15.59:0/2758376123 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0019b10 con 0x564d315cec30 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.144+0000 7fe5af000640 1 -- 172.21.15.59:0/2758376123 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000eaa0 con 0x564d315cec30 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- 172.21.15.59:0/2758376123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 msgr2=0x564d315d8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5be800640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 0x7fe5a40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8004af0 tx=0x7fe5b80095c0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009dc0 tx=0x7fe5b0011810 comp rx=0 tx=0).stop 2024-09-05T12:12:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- 172.21.15.59:0/2758376123 shutdown_connections 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315d8fb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/2758376123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- 172.21.15.59:0/2758376123 >> 172.21.15.59:0/2758376123 conn(0x564d315ca8a0 msgr2=0x564d315cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- 172.21.15.59:0/2758376123 shutdown_connections 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- 172.21.15.59:0/2758376123 wait complete. 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.145+0000 7fe5c73d0740 1 -- start start 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 Processor -- start 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315d6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d315dc0c0 con 0x564d315ce310 2024-09-05T12:12:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564d314ef350 con 0x564d315d94f0 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d315dc240 con 0x564d315cec30 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.146+0000 7fe5c73d0740 10 client.0 _ll_register_callbacks cb 0x564d31512290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33052/0 (socket says 172.21.15.59:33052) 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 -- 172.21.15.59:0/3056110936 learned_addr learned my addr 172.21.15.59:0/3056110936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5be800640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315d6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 -- 172.21.15.59:0/3056110936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 msgr2=0x564d315d6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315d6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 -- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 msgr2=0x564d315d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 -- 172.21.15.59:0/3056110936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d3130d960 con 0x564d315ce310 2024-09-05T12:12:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5bde00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000e680 tx=0x7fe5b00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66478eb42eb70d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.147+0000 7fe5a3400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0027070 con 0x564d315ce310 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5b0004450 con 0x564d315ce310 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0019b10 con 0x564d315ce310 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0033050 con 0x564d315ce310 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5c73d0740 10 client.17718 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:04.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564d315e7cf0 con 0x564d315ce310 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0019b10 con 0x564d315ce310 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.148+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe5b000ecf0 con 0x564d315ce310 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 10 client.17718 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 10 client.17718 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 20 client.17718 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d315e9040 0x564d315eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 4 client.17718 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d31541bc0 con 0x564d315e9040 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.149+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d315e9040 0x564d315eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.150+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d315e9040 0x564d315eb430 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a841ae7c3f4dbea server_cookie=9628830b22c6e7bf in_seq=0 out_seq=0 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.150+0000 7fe5a3400640 10 client.17718 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.150+0000 7fe5a3400640 1 -- 172.21.15.59:0/3056110936 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d31541bc0 con 0x564d315e9040 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.150+0000 7fe5a3400640 10 client.17718 handle_command_reply: tid=0 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.150+0000 7fe5c73d0740 1 client.17718 shutdown 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 msgr2=0x564d315d63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000e680 tx=0x7fe5b00095c0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 shutdown_connections 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315d6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d315e9040 0x564d315eb430 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/3056110936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d63f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 >> 172.21.15.59:0/3056110936 conn(0x564d315ca8a0 msgr2=0x564d315c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5a3400640 0 client.17718 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:04.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 shutdown_connections 2024-09-05T12:12:04.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 1 -- 172.21.15.59:0/3056110936 wait complete. 2024-09-05T12:12:04.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.151+0000 7fe5c73d0740 20 client.17718 trim_cache size 0 max 16384 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 Processor -- start 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 -- start start 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01af0b0 con 0x564d315ce310 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01b1930 con 0x564d315cec30 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01af720 con 0x564d315d94f0 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.169+0000 7fe5bde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33060/0 (socket says 172.21.15.59:33060) 2024-09-05T12:12:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 -- 172.21.15.59:0/891639493 learned_addr learned my addr 172.21.15.59:0/891639493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5be800640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 -- 172.21.15.59:0/891639493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 msgr2=0x564d315f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 -- 172.21.15.59:0/891639493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 msgr2=0x564d315c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 -- 172.21.15.59:0/891639493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d315cf890 con 0x564d315ce310 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5b7e00640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5bde00640 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1b08f4b567bd5e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.170+0000 7fe5ae600640 1 -- 172.21.15.59:0/891639493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0027070 con 0x564d315ce310 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5ae600640 1 -- 172.21.15.59:0/891639493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b00039e0 con 0x564d315ce310 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5ae600640 1 -- 172.21.15.59:0/891639493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001e040 con 0x564d315ce310 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 -- 172.21.15.59:0/891639493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 msgr2=0x564d315d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0009c90 comp rx=0 tx=0).stop 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 -- 172.21.15.59:0/891639493 shutdown_connections 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315d94f0 0x564d315f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315cec30 0x564d315c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/891639493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315ce310 0x564d315d0d70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.171+0000 7fe5c73d0740 1 -- 172.21.15.59:0/891639493 >> 172.21.15.59:0/891639493 conn(0x564d315ca8a0 msgr2=0x564d315d83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.172+0000 7fe5c73d0740 1 -- 172.21.15.59:0/891639493 shutdown_connections 2024-09-05T12:12:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.172+0000 7fe5c73d0740 1 -- 172.21.15.59:0/891639493 wait complete. 2024-09-05T12:12:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.172+0000 7fe5c73d0740 1 -- start start 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.172+0000 7fe5c73d0740 1 Processor -- start 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01afdf0 con 0x564d315dd6b0 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01b1260 con 0x564d315d94f0 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c01b04c0 con 0x564d315cec30 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5c73d0740 10 client.0 _ll_register_callbacks cb 0x7fe5b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5be800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5be800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33076/0 (socket says 172.21.15.59:33076) 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5be800640 1 -- 172.21.15.59:0/264709807 learned_addr learned my addr 172.21.15.59:0/264709807 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.173+0000 7fe5bde00640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 -- 172.21.15.59:0/264709807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 msgr2=0x564d315e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 -- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 msgr2=0x564d315dcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315dcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 -- 172.21.15.59:0/264709807 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d31653080 con 0x564d315dd6b0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5be800640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009ca0 tx=0x7fe5b802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdf91ff76596461 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5a3400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b802fc60 con 0x564d315dd6b0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe5b802fe00 con 0x564d315dd6b0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.174+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b00039e0 con 0x564d315dd6b0 2024-09-05T12:12:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe5b8003970 con 0x564d315dd6b0 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 10 client.17724 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564d316540d0 con 0x564d315dd6b0 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b00039e0 con 0x564d315dd6b0 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe5b8052070 con 0x564d315dd6b0 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 10 client.17724 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 10 client.17724 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 20 client.17724 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d31654850 0x564d31656c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 4 client.17724 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6925392e", "release": true, "await": true}] 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.175+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6925392e", "release": true, "await": true}) -- 0x564d3153c0d0 con 0x564d31654850 2024-09-05T12:12:04.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.176+0000 7fe5bde00640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d31654850 0x564d31656c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:04.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.176+0000 7fe5bde00640 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d31654850 0x564d31656c70 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff53b98fa62c2baa server_cookie=513830443af21990 in_seq=0 out_seq=0 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.176+0000 7fe5a3400640 10 client.17724 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.177+0000 7fe5a3400640 1 -- 172.21.15.59:0/264709807 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564d3153c0d0 con 0x564d31654850 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.177+0000 7fe5a3400640 10 client.17724 handle_command_reply: tid=0 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 client.17724 shutdown 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 msgr2=0x564d31652b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009ca0 tx=0x7fe5b802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 shutdown_connections 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x564d31654850 0x564d31656c70 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d315dd6b0 0x564d31652b40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564d315d94f0 0x564d315dcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 --2- 172.21.15.59:0/264709807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d315cec30 0x564d315e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5a3400640 0 client.17724 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:04.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 >> 172.21.15.59:0/264709807 conn(0x564d315ca8a0 msgr2=0x564d315d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 shutdown_connections 2024-09-05T12:12:04.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 1 -- 172.21.15.59:0/264709807 wait complete. 2024-09-05T12:12:04.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.178+0000 7fe5c73d0740 20 client.17724 trim_cache size 0 max 16384 2024-09-05T12:12:04.182 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 48, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "6925392e": { 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 48, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:12:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 134.5, 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:04.184 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 msgr2=0x7fe5a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 0x7fe5a40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8004af0 tx=0x7fe5b80095c0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 msgr2=0x7fe5c019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 0x7fe5c019ab00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9f0 tx=0x7fe5a800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 shutdown_connections 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe5a40a2c00 0x7fe5a40a50c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5c0109120 0x7fe5c019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5c01049a0 0x7fe5c019ab00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 --2- 172.21.15.59:0/2835344131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5c0103ff0 0x7fe5c019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:04.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 >> 172.21.15.59:0/2835344131 conn(0x7fe5c00ffa90 msgr2=0x7fe5c0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:04.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 shutdown_connections 2024-09-05T12:12:04.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:04.179+0000 7fe5c4a00640 1 -- 172.21.15.59:0/2835344131 wait complete. 2024-09-05T12:12:04.952 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6925392e', total seconds elapsed: 18.2 2024-09-05T12:12:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:04 smithi059 ceph-mon[30025]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T12:12:05.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:04 smithi189 ceph-mon[34772]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T12:12:05.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:04 smithi089 ceph-mon[34499]: pgmap v826: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 KiB/s wr, 5 op/s 2024-09-05T12:12:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:06 smithi059 ceph-mon[30025]: pgmap v827: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:07.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:06 smithi189 ceph-mon[34772]: pgmap v827: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:07.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:06 smithi089 ceph-mon[34499]: pgmap v827: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:08 smithi059 ceph-mon[30025]: pgmap v828: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-05T12:12:09.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:08 smithi189 ceph-mon[34772]: pgmap v828: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-05T12:12:09.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:08 smithi089 ceph-mon[34499]: pgmap v828: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-09-05T12:12:11.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:10 smithi059 ceph-mon[30025]: pgmap v829: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T12:12:11.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:10 smithi189 ceph-mon[34772]: pgmap v829: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T12:12:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:10 smithi089 ceph-mon[34499]: pgmap v829: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-09-05T12:12:13.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:12 smithi189 ceph-mon[34772]: pgmap v830: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:12:13.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:12 smithi089 ceph-mon[34499]: pgmap v830: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:12:13.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:12 smithi059 ceph-mon[30025]: pgmap v830: 129 pgs: 129 active+clean; 7.3 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:499/600: open fd = 3 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:500/600: open fd = 3 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:501/600: open fd = 3 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.273 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:502/600: open fd = 3 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:503/600: open fd = 3 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:504/600: open fd = 3 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.274 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:505/600: open fd = 3 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:506/600: open fd = 3 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:507/600: open fd = 3 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:508/600: open fd = 3 2024-09-05T12:12:15.275 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:509/600: open fd = 3 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:510/600: open fd = 3 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:511/600: open fd = 3 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.276 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:512/600: open fd = 3 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:513/600: open fd = 3 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:514/600: open fd = 3 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.277 INFO:tasks.workunit.client.0.smithi059.stdout:515/600: open fd = 3 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:516/600: open fd = 3 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:517/600: open fd = 3 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:518/600: open fd = 3 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.278 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:519/600: open fd = 3 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:520/600: open fd = 3 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:521/600: open fd = 3 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.279 INFO:tasks.workunit.client.0.smithi059.stdout:522/600: open fd = 3 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:523/600: open fd = 3 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:524/600: open fd = 3 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:525/600: open fd = 3 2024-09-05T12:12:15.280 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:526/600: open fd = 3 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:527/600: open fd = 3 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:528/600: open fd = 3 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.281 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:529/600: open fd = 3 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:530/600: open fd = 3 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:531/600: open fd = 3 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:532/600: open fd = 3 2024-09-05T12:12:15.282 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:533/600: open fd = 3 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:534/600: open fd = 3 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:535/600: open fd = 3 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.283 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:536/600: open fd = 3 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:537/600: open fd = 3 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:538/600: open fd = 3 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.284 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:539/600: open fd = 3 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:540/600: open fd = 3 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:541/600: open fd = 3 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.285 INFO:tasks.workunit.client.0.smithi059.stdout:542/600: open fd = 3 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:543/600: open fd = 3 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:544/600: open fd = 3 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:545/600: open fd = 3 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.286 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:546/600: open fd = 3 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:547/600: open fd = 3 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:548/600: open fd = 3 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.287 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:549/600: open fd = 3 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:550/600: open fd = 3 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:551/600: open fd = 3 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.288 INFO:tasks.workunit.client.0.smithi059.stdout:552/600: open fd = 3 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:553/600: open fd = 3 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:554/600: open fd = 3 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:555/600: open fd = 3 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.289 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:556/600: open fd = 3 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:557/600: open fd = 3 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:558/600: open fd = 3 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.290 INFO:tasks.workunit.client.0.smithi059.stdout:559/600: open fd = 3 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:560/600: open fd = 3 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:12:15.291 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:12:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:14 smithi189 ceph-mon[34772]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:12:15.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:14 smithi089 ceph-mon[34499]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:12:15.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:14 smithi059 ceph-mon[30025]: pgmap v831: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-09-05T12:12:17.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:16 smithi189 ceph-mon[34772]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T12:12:17.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:16 smithi089 ceph-mon[34499]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T12:12:17.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:16 smithi059 ceph-mon[30025]: pgmap v832: 129 pgs: 129 active+clean; 7.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-09-05T12:12:19.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:18 smithi189 ceph-mon[34772]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:12:19.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:18 smithi089 ceph-mon[34499]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:12:19.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:18 smithi059 ceph-mon[30025]: pgmap v833: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:12:20.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:20 smithi189 ceph-mon[34772]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:20.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:20 smithi089 ceph-mon[34499]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:20.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:20 smithi059 ceph-mon[30025]: pgmap v834: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:22.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:22 smithi189 ceph-mon[34772]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:22.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:22 smithi089 ceph-mon[34499]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:22.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:22 smithi059 ceph-mon[30025]: pgmap v835: 129 pgs: 129 active+clean; 7.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:24.510 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:24 smithi059 ceph-mon[30025]: pgmap v836: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:24.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:24 smithi189 ceph-mon[34772]: pgmap v836: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:24.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:24 smithi089 ceph-mon[34499]: pgmap v836: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:26.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:26 smithi059 ceph-mon[30025]: pgmap v837: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:26.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:26 smithi189 ceph-mon[34772]: pgmap v837: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:26.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:26 smithi089 ceph-mon[34499]: pgmap v837: 129 pgs: 129 active+clean; 7.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:28.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:28 smithi189 ceph-mon[34772]: pgmap v838: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:12:28.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:28 smithi089 ceph-mon[34499]: pgmap v838: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:12:28.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:28 smithi059 ceph-mon[30025]: pgmap v838: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-05T12:12:30.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:12:30.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:30 smithi189 ceph-mon[34772]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:30.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:30 smithi089 ceph-mon[34499]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:30.704 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:30.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:30 smithi059 ceph-mon[30025]: pgmap v839: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 -- 172.21.15.59:0/1563088056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c84074590 msgr2=0x7f6c84074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 --2- 172.21.15.59:0/1563088056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c84074590 0x7f6c84074970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009860 tx=0x7f6c7402a3d0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 -- 172.21.15.59:0/1563088056 shutdown_connections 2024-09-05T12:12:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 --2- 172.21.15.59:0/1563088056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c841129d0 0x7f6c84114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 --2- 172.21.15.59:0/1563088056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074eb0 0x7f6c84112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 --2- 172.21.15.59:0/1563088056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c84074590 0x7f6c84074970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 -- 172.21.15.59:0/1563088056 >> 172.21.15.59:0/1563088056 conn(0x7f6c840fd6e0 msgr2=0x7f6c840ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 -- 172.21.15.59:0/1563088056 shutdown_connections 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.281+0000 7f6c89200640 1 -- 172.21.15.59:0/1563088056 wait complete. 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.282+0000 7f6c89200640 1 Processor -- start 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.282+0000 7f6c89200640 1 -- start start 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074590 0x7f6c841a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c841129d0 0x7f6c8419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841183e0 con 0x7f6c84074eb0 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84118260 con 0x7f6c841129d0 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c89200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84118560 con 0x7f6c84074590 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45984/0 (socket says 172.21.15.59:45984) 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2079487312 learned_addr learned my addr 172.21.15.59:0/2079487312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c83400640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074590 0x7f6c841a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.283+0000 7f6c83e00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c841129d0 0x7f6c8419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2079487312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074590 msgr2=0x7f6c841a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074590 0x7f6c841a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2079487312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c841129d0 msgr2=0x7f6c8419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c841129d0 0x7f6c8419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2079487312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8419efc0 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9c0 tx=0x7f6c6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c80c00640 1 -- 172.21.15.59:0/2079487312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00ccc0 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c80c00640 1 -- 172.21.15.59:0/2079487312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c005500 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c80c00640 1 -- 172.21.15.59:0/2079487312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c010430 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8419f2b0 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.284+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841ab600 con 0x7f6c84074eb0 2024-09-05T12:12:31.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 Processor -- start 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 -- start start 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c80c00640 1 -- 172.21.15.59:0/2079487312 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c6c0046c0 con 0x7f6c84074eb0 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 0x55e72eb7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb82320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e72eadb3e0 con 0x55e72eb74d50 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e72ea82d80 con 0x55e72eb7ff30 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.288+0000 7f6c8d419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e72ead73a0 con 0x55e72eb75670 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45992/0 (socket says 172.21.15.59:45992) 2024-09-05T12:12:31.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 0x55e72eb7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb82320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83e00640 1 -- 172.21.15.59:0/1641291285 learned_addr learned my addr 172.21.15.59:0/1641291285 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 -- 172.21.15.59:0/1641291285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 msgr2=0x55e72eb82320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb82320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 -- 172.21.15.59:0/1641291285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 msgr2=0x55e72eb75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83400640 1 -- 172.21.15.59:0/1641291285 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72eaa1ae0 con 0x55e72eb75670 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.289+0000 7f6c83e00640 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c83400640 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 0x55e72eb7f9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009830 tx=0x7f6c74030ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded061680a1816c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c78c00640 1 -- 172.21.15.59:0/1641291285 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74004030 con 0x55e72eb75670 2024-09-05T12:12:31.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c78c00640 1 -- 172.21.15.59:0/1641291285 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c74041030 con 0x55e72eb75670 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c78c00640 1 -- 172.21.15.59:0/1641291285 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74005360 con 0x55e72eb75670 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c80c00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 0x7f6c680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c80c00640 1 -- 172.21.15.59:0/2079487312 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c6c0cf700 con 0x7f6c84074eb0 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.290+0000 7f6c83e00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 0x7f6c680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- 172.21.15.59:0/1641291285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 msgr2=0x55e72eb7f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 0x55e72eb7f9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009830 tx=0x7f6c74030ab0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- 172.21.15.59:0/1641291285 shutdown_connections 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb82320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb75670 0x55e72eb7f9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 --2- 172.21.15.59:0/1641291285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- 172.21.15.59:0/1641291285 >> 172.21.15.59:0/1641291285 conn(0x55e72eb712e0 msgr2=0x55e72eb716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- 172.21.15.59:0/1641291285 shutdown_connections 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c83e00640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 0x7f6c680a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009fd0 tx=0x7f6c7c009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- 172.21.15.59:0/1641291285 wait complete. 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.291+0000 7f6c8d419740 1 -- start start 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 Processor -- start 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb7ff30 0x55e72eb7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e72eb82b40 con 0x55e72eb74d50 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e72ea960f0 con 0x55e72eb75670 2024-09-05T12:12:31.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e72eb82cc0 con 0x55e72eb7ff30 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c8d419740 10 client.0 _ll_register_callbacks cb 0x55e72ea879e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46000/0 (socket says 172.21.15.59:46000) 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c83400640 1 -- 172.21.15.59:0/3521552345 learned_addr learned my addr 172.21.15.59:0/3521552345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.292+0000 7f6c83e00640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb7ff30 0x55e72eb7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 -- 172.21.15.59:0/3521552345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb7ff30 msgr2=0x55e72eb7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb7ff30 0x55e72eb7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 -- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 msgr2=0x55e72eb7d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb7d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 -- 172.21.15.59:0/3521552345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72e84c3c0 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c83400640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6c74004650 tx=0x7f6c740095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791de0744c8043a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c71600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74055060 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c740303a0 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.293+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c74041030 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c740305a0 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 10 client.17736 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e72eb8e780 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c74041030 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c7404f070 con 0x55e72eb74d50 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 10 client.17736 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:31.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 20 client.17736 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72eb8fad0 0x55e72eb91ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 4 client.17736 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.294+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e72eae8950 con 0x55e72eb8fad0 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.295+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72eb8fad0 0x55e72eb91ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.295+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72eb8fad0 0x55e72eb91ec0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b4ca7e13e46f25 server_cookie=69b5f321aa0420dc in_seq=0 out_seq=0 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.295+0000 7f6c71600640 10 client.17736 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c71600640 1 -- 172.21.15.59:0/3521552345 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e72eae8950 con 0x55e72eb8fad0 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c71600640 10 client.17736 handle_command_reply: tid=0 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 client.17736 shutdown 2024-09-05T12:12:31.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 msgr2=0x55e72eb7cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6c74004650 tx=0x7f6c740095b0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 shutdown_connections 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb7ff30 0x55e72eb7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb7d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72eb8fad0 0x55e72eb91ec0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 --2- 172.21.15.59:0/3521552345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb74d50 0x55e72eb7cd80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 >> 172.21.15.59:0/3521552345 conn(0x55e72eb712e0 msgr2=0x55e72eb77010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.296+0000 7f6c71600640 0 client.17736 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.297+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 shutdown_connections 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.297+0000 7f6c8d419740 1 -- 172.21.15.59:0/3521552345 wait complete. 2024-09-05T12:12:31.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.297+0000 7f6c8d419740 20 client.17736 trim_cache size 0 max 16384 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 Processor -- start 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 -- start start 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb79fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb7e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b32e0 con 0x55e72eb75670 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b5b60 con 0x55e72eb7ff30 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c8d419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b3950 con 0x55e72eb74d50 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46004/0 (socket says 172.21.15.59:46004) 2024-09-05T12:12:31.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c7aa00640 1 -- 172.21.15.59:0/484380467 learned_addr learned my addr 172.21.15.59:0/484380467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c83400640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb79fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.315+0000 7f6c83e00640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb7e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 -- 172.21.15.59:0/484380467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 msgr2=0x55e72eb79fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb79fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 -- 172.21.15.59:0/484380467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 msgr2=0x55e72eb7e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb7e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 -- 172.21.15.59:0/484380467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72eb72470 con 0x55e72eb75670 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c83400640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb79fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c83e00640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb7e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c09ba80 tx=0x7f6c6c0d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c6d01833f568fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c73e00640 1 -- 172.21.15.59:0/484380467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09b630 con 0x55e72eb75670 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c73e00640 1 -- 172.21.15.59:0/484380467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c09c1f0 con 0x55e72eb75670 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.316+0000 7f6c73e00640 1 -- 172.21.15.59:0/484380467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09b450 con 0x55e72eb75670 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- 172.21.15.59:0/484380467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 msgr2=0x55e72eb7a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c09ba80 tx=0x7f6c6c0d5530 comp rx=0 tx=0).stop 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- 172.21.15.59:0/484380467 shutdown_connections 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb7ff30 0x55e72eb7e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb75670 0x55e72eb7a520 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 --2- 172.21.15.59:0/484380467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb79fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- 172.21.15.59:0/484380467 >> 172.21.15.59:0/484380467 conn(0x55e72eb712e0 msgr2=0x55e72eb76fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- 172.21.15.59:0/484380467 shutdown_connections 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- 172.21.15.59:0/484380467 wait complete. 2024-09-05T12:12:31.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.317+0000 7f6c8d419740 1 -- start start 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 Processor -- start 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b4020 con 0x55e72eb7ff30 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b5490 con 0x55e72eb75670 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b46f0 con 0x55e72eb74d50 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.318+0000 7f6c8d419740 10 client.0 _ll_register_callbacks cb 0x7f6c740096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46008/0 (socket says 172.21.15.59:46008) 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83e00640 1 -- 172.21.15.59:0/2837480928 learned_addr learned my addr 172.21.15.59:0/2837480928 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83400640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2837480928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 msgr2=0x55e72eb95720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb95720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2837480928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 msgr2=0x55e72eb91090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 -- 172.21.15.59:0/2837480928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72eb918c0 con 0x55e72eb75670 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83400640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb95720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c83e00640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:31.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c7aa00640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb90aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55e72eb7e140 tx=0x7f6c6c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6778e3d6c5b5ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c71600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.319+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09b630 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c09c1f0 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6c6c098160 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c0a4760 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 10 client.27746 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e72ebfa4f0 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c09c1f0 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c6c0a42d0 con 0x55e72eb75670 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 10 client.27746 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:31.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 10 client.27746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 20 client.27746 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72ebfab60 0x55e72ebfcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 4 client.27746 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.320+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e72eae2e60 con 0x55e72ebfab60 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c83400640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72ebfab60 0x55e72ebfcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c83400640 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72ebfab60 0x55e72ebfcf50 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1cbb875a44b3c9f server_cookie=393370bf5b9153d4 in_seq=0 out_seq=0 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c71600640 10 client.27746 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c71600640 1 -- 172.21.15.59:0/2837480928 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e72eae2e60 con 0x55e72ebfab60 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c71600640 10 client.27746 handle_command_reply: tid=0 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c8d419740 1 client.27746 shutdown 2024-09-05T12:12:31.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 msgr2=0x55e72eb90aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.321+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb90aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55e72eb7e140 tx=0x7f6c6c009f90 comp rx=0 tx=0).stop 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 shutdown_connections 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55e72ebfab60 0x55e72ebfcf50 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e72eb7ff30 0x55e72eb91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c71600640 0 client.27746 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e72eb75670 0x55e72eb90aa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 --2- 172.21.15.59:0/2837480928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e72eb74d50 0x55e72eb95720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 >> 172.21.15.59:0/2837480928 conn(0x55e72eb712e0 msgr2=0x55e72eb7cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.449 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:31.449 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:12:31.449 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:12:31.449 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:31.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 shutdown_connections 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 1 -- 172.21.15.59:0/2837480928 wait complete. 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.322+0000 7f6c8d419740 20 client.27746 trim_cache size 0 max 16384 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 msgr2=0x7f6c680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 0x7f6c680a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009fd0 tx=0x7f6c7c009500 comp rx=0 tx=0).stop 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 msgr2=0x7f6c841a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e9c0 tx=0x7f6c6c00ee80 comp rx=0 tx=0).stop 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 shutdown_connections 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f6c680a2c00 0x7f6c680a50c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c841129d0 0x7f6c8419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c84074eb0 0x7f6c841a4bc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 --2- 172.21.15.59:0/2079487312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c84074590 0x7f6c841a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:31.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 >> 172.21.15.59:0/2079487312 conn(0x7f6c840fd6e0 msgr2=0x7f6c840fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:31.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 shutdown_connections 2024-09-05T12:12:31.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:31.323+0000 7f6c89200640 1 -- 172.21.15.59:0/2079487312 wait complete. 2024-09-05T12:12:31.882 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:12:31.882 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.0 2024-09-05T12:12:31.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:32.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:12:32.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:32.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:32 smithi059 ceph-mon[30025]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:32.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:32 smithi189 ceph-mon[34772]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:32.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:32 smithi089 ceph-mon[34499]: pgmap v840: 129 pgs: 129 active+clean; 7.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:33.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 -- 172.21.15.59:0/1227958706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501019c0 msgr2=0x7fca50101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 --2- 172.21.15.59:0/1227958706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501019c0 0x7fca50101e20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009930 tx=0x7fca3c02a420 comp rx=0 tx=0).stop 2024-09-05T12:12:33.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 -- 172.21.15.59:0/1227958706 shutdown_connections 2024-09-05T12:12:33.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 --2- 172.21.15.59:0/1227958706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 0x7fca5010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 --2- 172.21.15.59:0/1227958706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501019c0 0x7fca50101e20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 --2- 172.21.15.59:0/1227958706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501079c0 0x7fca50107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 -- 172.21.15.59:0/1227958706 >> 172.21.15.59:0/1227958706 conn(0x7fca500fd740 msgr2=0x7fca500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 -- 172.21.15.59:0/1227958706 shutdown_connections 2024-09-05T12:12:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.176+0000 7fca4fe00640 1 -- 172.21.15.59:0/1227958706 wait complete. 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.177+0000 7fca4fe00640 1 Processor -- start 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.177+0000 7fca4fe00640 1 -- start start 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 0x7fca50198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca5019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111dc0 con 0x7fca501079c0 2024-09-05T12:12:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111c40 con 0x7fca50102360 2024-09-05T12:12:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111f40 con 0x7fca501019c0 2024-09-05T12:12:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca5019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55264/0 (socket says 172.21.15.59:55264) 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4ea00640 1 -- 172.21.15.59:0/2433684944 learned_addr learned my addr 172.21.15.59:0/2433684944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4e000640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 0x7fca50198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4f400640 1 -- 172.21.15.59:0/2433684944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 msgr2=0x7fca50198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.178+0000 7fca4f400640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4f400640 1 -- 172.21.15.59:0/2433684944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 msgr2=0x7fca50198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4f400640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 0x7fca50198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4f400640 1 -- 172.21.15.59:0/2433684944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5019d2d0 con 0x7fca501079c0 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4ea00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4f400640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca5019cbf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca4800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800ce20 con 0x7fca501079c0 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca48005020 con 0x7fca501079c0 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5019d5c0 con 0x7fca501079c0 2024-09-05T12:12:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48010720 con 0x7fca501079c0 2024-09-05T12:12:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.179+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a4ea0 con 0x7fca501079c0 2024-09-05T12:12:33.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.181+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca50101e20 con 0x7fca501079c0 2024-09-05T12:12:33.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.182+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca48005540 con 0x7fca501079c0 2024-09-05T12:12:33.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.184+0000 7fca54a00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.184+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fca480cfb30 con 0x7fca501079c0 2024-09-05T12:12:33.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.184+0000 7fca4ea00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.185+0000 7fca4ea00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fca40011490 tx=0x7fca40005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:33.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.190+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca4809c620 con 0x7fca501079c0 2024-09-05T12:12:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.421+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca5002d7c0 con 0x7fca501079c0 2024-09-05T12:12:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.422+0000 7fca54a00640 1 -- 172.21.15.59:0/2433684944 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fca4809bf20 con 0x7fca501079c0 2024-09-05T12:12:33.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:33.425 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 msgr2=0x7fca340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fca40011490 tx=0x7fca40005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 msgr2=0x7fca5019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca5019cbf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca4800ef10 comp rx=0 tx=0).stop 2024-09-05T12:12:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 shutdown_connections 2024-09-05T12:12:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fca340a2c00 0x7fca340a50c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca501079c0 0x7fca5019cbf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca50102360 0x7fca50198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 --2- 172.21.15.59:0/2433684944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca501019c0 0x7fca50198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 >> 172.21.15.59:0/2433684944 conn(0x7fca500fd740 msgr2=0x7fca500fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 shutdown_connections 2024-09-05T12:12:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.425+0000 7fca4fe00640 1 -- 172.21.15.59:0/2433684944 wait complete. 2024-09-05T12:12:33.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 -- 172.21.15.59:0/1731242757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894112a50 msgr2=0x7f5894114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 --2- 172.21.15.59:0/1731242757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894112a50 0x7f5894114e40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f588c009ca0 tx=0x7f588c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 -- 172.21.15.59:0/1731242757 shutdown_connections 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 --2- 172.21.15.59:0/1731242757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894112a50 0x7f5894114e40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 --2- 172.21.15.59:0/1731242757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894077040 0x7f5894112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 --2- 172.21.15.59:0/1731242757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894076720 0x7f5894076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 -- 172.21.15.59:0/1731242757 >> 172.21.15.59:0/1731242757 conn(0x7f58940ffa50 msgr2=0x7f5894101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 -- 172.21.15.59:0/1731242757 shutdown_connections 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.938+0000 7f589c600640 1 -- 172.21.15.59:0/1731242757 wait complete. 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.939+0000 7f589c600640 1 Processor -- start 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.939+0000 7f589c600640 1 -- start start 2024-09-05T12:12:33.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894076720 0x7f58941a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 0x7f589419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5894118590 con 0x7f5894077040 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5894118410 con 0x7f5894112a50 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589c600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5894118710 con 0x7f5894076720 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46064/0 (socket says 172.21.15.59:46064) 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899200640 1 -- 172.21.15.59:0/3580221781 learned_addr learned my addr 172.21.15.59:0/3580221781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899c00640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894076720 0x7f58941a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f589a600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 0x7f589419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899200640 1 -- 172.21.15.59:0/3580221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894076720 msgr2=0x7f58941a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.940+0000 7f5899200640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894076720 0x7f58941a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5899200640 1 -- 172.21.15.59:0/3580221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 msgr2=0x7f589419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5899200640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 0x7f589419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5899200640 1 -- 172.21.15.59:0/3580221781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589419efa0 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f589a600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 0x7f589419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5899200640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f588000d950 tx=0x7f588000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880017030 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589419f290 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5880004020 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58800047a0 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.941+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58941ab580 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.942+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5894076b00 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.944+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58800041d0 con 0x7f5894077040 2024-09-05T12:12:33.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.946+0000 7f5892000640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 0x7f587c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:33.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.946+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f58800ced00 con 0x7f5894077040 2024-09-05T12:12:33.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.946+0000 7f5899c00640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 0x7f587c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:33.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.947+0000 7f5899c00640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 0x7f587c0a50c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f589419f9b0 tx=0x7f588800b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:33.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:33.950+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f588009b7f0 con 0x7f5894077040 2024-09-05T12:12:34.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.175+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58941ab8e0 con 0x7f5894077040 2024-09-05T12:12:34.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.176+0000 7f5892000640 1 -- 172.21.15.59:0/3580221781 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f588009b0f0 con 0x7f5894077040 2024-09-05T12:12:34.178 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:34.182 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 msgr2=0x7f587c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 0x7f587c0a50c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f589419f9b0 tx=0x7f588800b390 comp rx=0 tx=0).stop 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 msgr2=0x7f58941a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f588000d950 tx=0x7f588000de10 comp rx=0 tx=0).stop 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 shutdown_connections 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f587c0a2c00 0x7f587c0a50c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5894112a50 0x7f589419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:34.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5894077040 0x7f58941a4bf0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:34.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 --2- 172.21.15.59:0/3580221781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5894076720 0x7f58941a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:34.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 >> 172.21.15.59:0/3580221781 conn(0x7f58940ffa50 msgr2=0x7f5894101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:34.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 shutdown_connections 2024-09-05T12:12:34.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:34.179+0000 7f589c600640 1 -- 172.21.15.59:0/3580221781 wait complete. 2024-09-05T12:12:34.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:34 smithi059 ceph-mon[30025]: pgmap v841: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:34.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2433684944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:34.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:34 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3580221781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:34 smithi189 ceph-mon[34772]: pgmap v841: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2433684944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:34.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:34 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3580221781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:34.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0' 2024-09-05T12:12:34.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0 2024-09-05T12:12:34.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:34 smithi089 ceph-mon[34499]: pgmap v841: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-09-05T12:12:34.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2433684944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:34.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:34 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3580221781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:35.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:12:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 -- 172.21.15.59:0/534188812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388102790 msgr2=0x7fa3881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 --2- 172.21.15.59:0/534188812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388102790 0x7fa3881067e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa378009900 tx=0x7fa37802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 -- 172.21.15.59:0/534188812 shutdown_connections 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 --2- 172.21.15.59:0/534188812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 --2- 172.21.15.59:0/534188812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388102790 0x7fa3881067e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 --2- 172.21.15.59:0/534188812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388101de0 0x7fa3881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.024+0000 7fa390800640 1 -- 172.21.15.59:0/534188812 >> 172.21.15.59:0/534188812 conn(0x7fa3880fd8e0 msgr2=0x7fa3880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.025+0000 7fa390800640 1 -- 172.21.15.59:0/534188812 shutdown_connections 2024-09-05T12:12:36.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.025+0000 7fa390800640 1 -- 172.21.15.59:0/534188812 wait complete. 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 Processor -- start 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 -- start start 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388102790 0x7fa38819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa388111fc0 con 0x7fa388101de0 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa388111e40 con 0x7fa388102790 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa390800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa388112140 con 0x7fa388106f10 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.026+0000 7fa38de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46080/0 (socket says 172.21.15.59:46080) 2024-09-05T12:12:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 -- 172.21.15.59:0/1344787387 learned_addr learned my addr 172.21.15.59:0/1344787387 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38d400640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388102790 0x7fa38819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38e800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 -- 172.21.15.59:0/1344787387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 msgr2=0x7fa38819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 -- 172.21.15.59:0/1344787387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388102790 msgr2=0x7fa38819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388102790 0x7fa38819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 -- 172.21.15.59:0/1344787387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38819f5e0 con 0x7fa388101de0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38e800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.027+0000 7fa38de00640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00ea60 tx=0x7fa37c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.028+0000 7fa382a00640 1 -- 172.21.15.59:0/1344787387 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00cde0 con 0x7fa388101de0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.028+0000 7fa382a00640 1 -- 172.21.15.59:0/1344787387 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c0035f0 con 0x7fa388101de0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.028+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38819f8d0 con 0x7fa388101de0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.028+0000 7fa382a00640 1 -- 172.21.15.59:0/1344787387 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c010670 con 0x7fa388101de0 2024-09-05T12:12:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.028+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a71b0 con 0x7fa388101de0 2024-09-05T12:12:36.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.031+0000 7fa382a00640 1 -- 172.21.15.59:0/1344787387 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa37c0037a0 con 0x7fa388101de0 2024-09-05T12:12:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.032+0000 7fa3933c7740 1 Processor -- start 2024-09-05T12:12:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.032+0000 7fa382a00640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 0x7fa3700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 -- start start 2024-09-05T12:12:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa382a00640 1 -- 172.21.15.59:0/1344787387 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa37c0cf730 con 0x7fa388101de0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa38e800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 0x7fa3700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 0x55b13001c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 0x55b13002f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13002fda0 0x55b130032190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b12ff82530 con 0x55b13001cbf0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b12ff29860 con 0x55b13002fda0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.033+0000 7fa3933c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b12ff7e4f0 con 0x55b13001c2d0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 0x55b13002f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 0x55b13001c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 0x55b13002f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46092/0 (socket says 172.21.15.59:46092) 2024-09-05T12:12:36.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38d400640 1 -- 172.21.15.59:0/2296926563 learned_addr learned my addr 172.21.15.59:0/2296926563 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38e800640 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13002fda0 0x55b130032190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 -- 172.21.15.59:0/2296926563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13002fda0 msgr2=0x55b130032190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13002fda0 0x55b130032190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 -- 172.21.15.59:0/2296926563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 msgr2=0x55b13002f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 0x55b13002f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 -- 172.21.15.59:0/2296926563 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12fcf3b90 con 0x55b13001c2d0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38e800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 0x7fa3700a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa384005fd0 tx=0x7fa384009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.034+0000 7fa38de00640 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 0x55b13001c6b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ccdc0 tx=0x7fa37c09b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fcae969661ad409 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.035+0000 7fa377000640 1 -- 172.21.15.59:0/2296926563 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c09fed0 con 0x55b13001c2d0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.035+0000 7fa377000640 1 -- 172.21.15.59:0/2296926563 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c09ce50 con 0x55b13001c2d0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.035+0000 7fa377000640 1 -- 172.21.15.59:0/2296926563 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0cf390 con 0x55b13001c2d0 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.035+0000 7fa3933c7740 1 -- 172.21.15.59:0/2296926563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 msgr2=0x55b13001c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.035+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 0x55b13001c6b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ccdc0 tx=0x7fa37c09b720 comp rx=0 tx=0).stop 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 -- 172.21.15.59:0/2296926563 shutdown_connections 2024-09-05T12:12:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13002fda0 0x55b130032190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13001cbf0 0x55b13002f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2296926563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001c2d0 0x55b13001c6b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 -- 172.21.15.59:0/2296926563 >> 172.21.15.59:0/2296926563 conn(0x55b130018940 msgr2=0x55b130018d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 -- 172.21.15.59:0/2296926563 shutdown_connections 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 -- 172.21.15.59:0/2296926563 wait complete. 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 -- start start 2024-09-05T12:12:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.036+0000 7fa3933c7740 1 Processor -- start 2024-09-05T12:12:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b13002cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b13002d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b130032970 con 0x55b13002fda0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b12ff3ce60 con 0x55b13001c2d0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b130032af0 con 0x55b13001cbf0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa3933c7740 10 client.0 _ll_register_callbacks cb 0x55b12ff41cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa38d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa38d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55322/0 (socket says 172.21.15.59:55322) 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa38d400640 1 -- 172.21.15.59:0/2491753957 learned_addr learned my addr 172.21.15.59:0/2491753957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa38de00640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b13002cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.037+0000 7fa38e800640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b13002d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 -- 172.21.15.59:0/2491753957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 msgr2=0x55b13002d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 -- 172.21.15.59:0/2491753957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 msgr2=0x55b13002d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b13002d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 -- 172.21.15.59:0/2491753957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12ff48990 con 0x55b13001c2d0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38d400640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38e800640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b13002d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa38de00640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b13002cd80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ccdc0 tx=0x7fa37c0a4860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2557457ff3cb409a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.038+0000 7fa36b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0ba050 con 0x55b13001c2d0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa37c09ce50 con 0x55b13001c2d0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa37c0cf390 con 0x55b13001c2d0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa37c09fed0 con 0x55b13001c2d0 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa3933c7740 10 client.27770 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.039+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b13003e6b0 con 0x55b13001c2d0 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c09ce50 con 0x55b13001c2d0 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa37c0cf070 con 0x55b13001c2d0 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 10 client.27770 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 10 client.27770 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 20 client.27770 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b13003fa00 0x55b130041df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 4 client.27770 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12ff25310 con 0x55b13003fa00 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.040+0000 7fa38d400640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b13003fa00 0x55b130041df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.041+0000 7fa38d400640 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b13003fa00 0x55b130041df0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c7ef6d315356d8 server_cookie=4130128471602f9e in_seq=0 out_seq=0 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.041+0000 7fa36b400640 10 client.27770 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa36b400640 1 -- 172.21.15.59:0/2491753957 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b12ff25310 con 0x55b13003fa00 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa36b400640 10 client.27770 handle_command_reply: tid=0 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 client.27770 shutdown 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 msgr2=0x55b13002cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b13002cd80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0ccdc0 tx=0x7fa37c0a4860 comp rx=0 tx=0).stop 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 shutdown_connections 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b13003fa00 0x55b130041df0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b13002d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b13002d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 --2- 172.21.15.59:0/2491753957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b13002cd80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 >> 172.21.15.59:0/2491753957 conn(0x55b130018940 msgr2=0x55b130017c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa36b400640 0 client.27770 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 shutdown_connections 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.042+0000 7fa3933c7740 1 -- 172.21.15.59:0/2491753957 wait complete. 2024-09-05T12:12:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.043+0000 7fa3933c7740 20 client.27770 trim_cache size 0 max 16384 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 Processor -- start 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 -- start start 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b130045360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881aad40 con 0x55b13002fda0 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881ad580 con 0x55b13001c2d0 2024-09-05T12:12:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.072+0000 7fa3933c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881ab390 con 0x55b13001cbf0 2024-09-05T12:12:36.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:53958/0 (socket says 172.21.15.59:53958) 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 -- 172.21.15.59:0/4243646816 learned_addr learned my addr 172.21.15.59:0/4243646816 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38e800640 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b130045360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38d400640 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 -- 172.21.15.59:0/4243646816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 msgr2=0x55b130044e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 -- 172.21.15.59:0/4243646816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 msgr2=0x55b130045360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b130045360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.073+0000 7fa38de00640 1 -- 172.21.15.59:0/4243646816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1300183e0 con 0x55b13001c2d0 2024-09-05T12:12:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.074+0000 7fa38de00640 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa37c09b820 tx=0x7fa37c003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c6f2ba01b91cec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.074+0000 7fa376600640 1 -- 172.21.15.59:0/4243646816 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c003bf0 con 0x55b13001c2d0 2024-09-05T12:12:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.074+0000 7fa376600640 1 -- 172.21.15.59:0/4243646816 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c09ce50 con 0x55b13001c2d0 2024-09-05T12:12:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.074+0000 7fa376600640 1 -- 172.21.15.59:0/4243646816 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c097a20 con 0x55b13001c2d0 2024-09-05T12:12:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- 172.21.15.59:0/4243646816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 msgr2=0x55b130017a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa37c09b820 tx=0x7fa37c003f30 comp rx=0 tx=0).stop 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- 172.21.15.59:0/4243646816 shutdown_connections 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b130045360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 --2- 172.21.15.59:0/4243646816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130017a90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- 172.21.15.59:0/4243646816 >> 172.21.15.59:0/4243646816 conn(0x55b130018940 msgr2=0x55b13001e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- 172.21.15.59:0/4243646816 shutdown_connections 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- 172.21.15.59:0/4243646816 wait complete. 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.075+0000 7fa3933c7740 1 -- start start 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 Processor -- start 2024-09-05T12:12:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130044a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b1300a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881aba40 con 0x55b13002fda0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881aceb0 con 0x55b13001c2d0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881ac110 con 0x55b13001cbf0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa3933c7740 10 client.0 _ll_register_callbacks cb 0x7fa3840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.076+0000 7fa38e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b1300a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b1300a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46110/0 (socket says 172.21.15.59:46110) 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38e800640 1 -- 172.21.15.59:0/1376030874 learned_addr learned my addr 172.21.15.59:0/1376030874 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38de00640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130044a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 -- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 msgr2=0x55b130044a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130044a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 -- 172.21.15.59:0/1376030874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 msgr2=0x55b1300a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b1300a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 -- 172.21.15.59:0/1376030874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1300a1d60 con 0x55b13001cbf0 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa38d400640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa378004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102bc9f014fe819d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.077+0000 7fa36b400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378035040 con 0x55b13001cbf0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa37c09ce50 con 0x55b13001cbf0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa3780054d0 con 0x55b13001cbf0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa378057050 con 0x55b13001cbf0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa3933c7740 10 client.27700 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.078+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b1300aa2d0 con 0x55b13001cbf0 2024-09-05T12:12:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c09ce50 con 0x55b13001cbf0 2024-09-05T12:12:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa3780319f0 con 0x55b13001cbf0 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 10 client.27700 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 10 client.27700 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 20 client.27700 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1300aaa10 0x55b1300ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 4 client.27700 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}] 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}) -- 0x55b12ff8fc60 con 0x55b1300aaa10 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.079+0000 7fa38de00640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1300aaa10 0x55b1300ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.080+0000 7fa38de00640 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1300aaa10 0x55b1300ace30 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a378a0d7fb32cf2 server_cookie=fd3d543495d54c38 in_seq=0 out_seq=0 2024-09-05T12:12:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.080+0000 7fa36b400640 10 client.27700 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.088+0000 7fa36b400640 1 -- 172.21.15.59:0/1376030874 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b12ff8fc60 con 0x55b1300aaa10 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.088+0000 7fa36b400640 10 client.27700 handle_command_reply: tid=0 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.088+0000 7fa3933c7740 1 client.27700 shutdown 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.088+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 msgr2=0x55b130044fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.088+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa378004530 comp rx=0 tx=0).stop 2024-09-05T12:12:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 shutdown_connections 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55b1300aaa10 0x55b1300ace30 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b13002fda0 0x55b1300a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b13001cbf0 0x55b130044fb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 --2- 172.21.15.59:0/1376030874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b13001c2d0 0x55b130044a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa36b400640 0 client.27700 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 >> 172.21.15.59:0/1376030874 conn(0x55b130018940 msgr2=0x55b13001dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 shutdown_connections 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 1 -- 172.21.15.59:0/1376030874 wait complete. 2024-09-05T12:12:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.089+0000 7fa3933c7740 20 client.27700 trim_cache size 0 max 16384 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 50, 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "3587fa16": { 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 50, 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:12:36.092 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 140.0, 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:36.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 msgr2=0x7fa3700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 0x7fa3700a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa384005fd0 tx=0x7fa384009500 comp rx=0 tx=0).stop 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 msgr2=0x7fa38819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00ea60 tx=0x7fa37c00ef20 comp rx=0 tx=0).stop 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 shutdown_connections 2024-09-05T12:12:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa3700a2b80 0x7fa3700a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa388106f10 0x7fa38819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa388102790 0x7fa38819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 --2- 172.21.15.59:0/1344787387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa388101de0 0x7fa38819a630 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 >> 172.21.15.59:0/1344787387 conn(0x7fa3880fd8e0 msgr2=0x7fa3880fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 shutdown_connections 2024-09-05T12:12:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.090+0000 7fa390800640 1 -- 172.21.15.59:0/1344787387 wait complete. 2024-09-05T12:12:36.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:36 smithi189 ceph-mon[34772]: pgmap v842: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:36.613 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:36 smithi059 ceph-mon[30025]: pgmap v842: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:36.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:36 smithi089 ceph-mon[34499]: pgmap v842: 129 pgs: 129 active+clean; 7.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:36.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 -- 172.21.15.59:0/2674532250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434074590 msgr2=0x7f2434074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 --2- 172.21.15.59:0/2674532250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434074590 0x7f2434074970 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2424009900 tx=0x7f242402a480 comp rx=0 tx=0).stop 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 -- 172.21.15.59:0/2674532250 shutdown_connections 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 --2- 172.21.15.59:0/2674532250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434112b90 0x7f2434114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 --2- 172.21.15.59:0/2674532250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074eb0 0x7f2434112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 --2- 172.21.15.59:0/2674532250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434074590 0x7f2434074970 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 -- 172.21.15.59:0/2674532250 >> 172.21.15.59:0/2674532250 conn(0x7f24340fd8c0 msgr2=0x7f24340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 -- 172.21.15.59:0/2674532250 shutdown_connections 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.865+0000 7f243a000640 1 -- 172.21.15.59:0/2674532250 wait complete. 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.866+0000 7f243a000640 1 Processor -- start 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 -- start start 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074590 0x7f24341a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 0x7f24341a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2434118570 con 0x7f2434112b90 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24341183f0 con 0x7f2434074590 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f243a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24341186f0 con 0x7f2434074eb0 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46122/0 (socket says 172.21.15.59:46122) 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f2433e00640 1 -- 172.21.15.59:0/366357735 learned_addr learned my addr 172.21.15.59:0/366357735 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f2432a00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 0x7f24341a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.867+0000 7f2433400640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074590 0x7f24341a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 -- 172.21.15.59:0/366357735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 msgr2=0x7f24341a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 0x7f24341a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 -- 172.21.15.59:0/366357735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074590 msgr2=0x7f24341a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074590 0x7f24341a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 -- 172.21.15.59:0/366357735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243419f0a0 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2432a00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 0x7f24341a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f2433e00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f242c00eb40 tx=0x7f242c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f242be00640 1 -- 172.21.15.59:0/366357735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c004b20 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243419f390 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f242be00640 1 -- 172.21.15.59:0/366357735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242c004cc0 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.868+0000 7f242be00640 1 -- 172.21.15.59:0/366357735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c010850 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.869+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24341ab680 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.872+0000 7f243ca84740 1 Processor -- start 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f242be00640 1 -- 172.21.15.59:0/366357735 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f242c005020 con 0x7f2434112b90 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 -- start start 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf25e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf1c43e0 con 0x55dfdf25e590 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf16bd80 con 0x55dfdf25dc70 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.873+0000 7f243ca84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf1c03a0 con 0x55dfdf268e50 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2433400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55364/0 (socket says 172.21.15.59:55364) 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 -- 172.21.15.59:0/3905273219 learned_addr learned my addr 172.21.15.59:0/3905273219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2433400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46130/0 (socket says 172.21.15.59:46130) 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf25e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 -- 172.21.15.59:0/3905273219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 msgr2=0x55dfdf25e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf25e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 -- 172.21.15.59:0/3905273219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 msgr2=0x55dfdf268910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2432a00640 1 -- 172.21.15.59:0/3905273219 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdf18aae0 con 0x55dfdf268e50 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f2433400640 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.874+0000 7f242be00640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 0x7f24180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f2432a00640 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2420011bb0 tx=0x7f24200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d2d2d938da63e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f242be00640 1 -- 172.21.15.59:0/366357735 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f242c0cfbb0 con 0x7f2434112b90 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f2433400640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 0x7f24180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f241fe00640 1 -- 172.21.15.59:0/3905273219 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242000fc50 con 0x55dfdf268e50 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f241fe00640 1 -- 172.21.15.59:0/3905273219 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242000fdf0 con 0x55dfdf268e50 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.875+0000 7f241fe00640 1 -- 172.21.15.59:0/3905273219 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420003a20 con 0x55dfdf268e50 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- 172.21.15.59:0/3905273219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 msgr2=0x55dfdf26b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f2433400640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 0x7f24180a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f243419fab0 tx=0x7f24240058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2420011bb0 tx=0x7f24200046c0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- 172.21.15.59:0/3905273219 shutdown_connections 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf26b240 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25e590 0x55dfdf268910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 --2- 172.21.15.59:0/3905273219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf25e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- 172.21.15.59:0/3905273219 >> 172.21.15.59:0/3905273219 conn(0x55dfdf25a200 msgr2=0x55dfdf25a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- 172.21.15.59:0/3905273219 shutdown_connections 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- 172.21.15.59:0/3905273219 wait complete. 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.876+0000 7f243ca84740 1 -- start start 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 Processor -- start 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 0x55dfdf265da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf2662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf26ba20 con 0x55dfdf268e50 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf17f0f0 con 0x55dfdf25e590 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dfdf26bba0 con 0x55dfdf25dc70 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f243ca84740 10 client.0 _ll_register_callbacks cb 0x55dfdf166bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2433400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 0x55dfdf265da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46144/0 (socket says 172.21.15.59:46144) 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2433400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 0x55dfdf265da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:55376/0 (socket says 172.21.15.59:55376) 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2433e00640 1 -- 172.21.15.59:0/791342634 learned_addr learned my addr 172.21.15.59:0/791342634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.877+0000 7f2432a00640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf2662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 -- 172.21.15.59:0/791342634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 msgr2=0x55dfdf265da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 0x55dfdf265da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 -- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 msgr2=0x55dfdf2662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf2662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 -- 172.21.15.59:0/791342634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdef353c0 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2432a00640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf2662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f2433e00640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f242c098400 tx=0x7f242c09b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb1b42aae9231b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f241cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c09bfa0 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f242c0ba770 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.878+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242000fdf0 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f242c07f050 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 10 client.17784 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dfdf277b20 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242000fdf0 con 0x55dfdf268e50 2024-09-05T12:12:36.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f242c0ba3d0 con 0x55dfdf268e50 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 10 client.17784 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 20 client.17784 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf279030 0x55dfdf27b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 4 client.17784 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.879+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfdf1d1950 con 0x55dfdf279030 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.880+0000 7f2433400640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf279030 0x55dfdf27b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.880+0000 7f2433400640 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf279030 0x55dfdf27b460 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e83331d75164692 server_cookie=e07808774f0b0fc8 in_seq=0 out_seq=0 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.880+0000 7f241cc00640 10 client.17784 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f241cc00640 1 -- 172.21.15.59:0/791342634 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfdf1d1950 con 0x55dfdf279030 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f241cc00640 10 client.17784 handle_command_reply: tid=0 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 client.17784 shutdown 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 msgr2=0x55dfdf266820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f242c098400 tx=0x7f242c09b900 comp rx=0 tx=0).stop 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 shutdown_connections 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf279030 0x55dfdf27b460 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf266820 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf2662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 --2- 172.21.15.59:0/791342634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25dc70 0x55dfdf265da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 >> 172.21.15.59:0/791342634 conn(0x55dfdf25a200 msgr2=0x55dfdf25ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.881+0000 7f241cc00640 0 client.17784 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.882+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 shutdown_connections 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.882+0000 7f243ca84740 1 -- 172.21.15.59:0/791342634 wait complete. 2024-09-05T12:12:36.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.882+0000 7f243ca84740 20 client.17784 trim_cache size 0 max 16384 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.895+0000 7f243ca84740 1 Processor -- start 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.895+0000 7f243ca84740 1 -- start start 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf280d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf281270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b35c0 con 0x55dfdf25dc70 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b5e40 con 0x55dfdf25e590 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f243ca84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b3c30 con 0x55dfdf268e50 2024-09-05T12:12:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46154/0 (socket says 172.21.15.59:46154) 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 -- 172.21.15.59:0/2378052678 learned_addr learned my addr 172.21.15.59:0/2378052678 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2432a00640 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf280d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433e00640 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf281270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 -- 172.21.15.59:0/2378052678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 msgr2=0x55dfdf281270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf281270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 -- 172.21.15.59:0/2378052678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 msgr2=0x55dfdf280d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf280d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.896+0000 7f2433400640 1 -- 172.21.15.59:0/2378052678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdf255770 con 0x55dfdf25dc70 2024-09-05T12:12:36.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.897+0000 7f2433400640 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2424009230 tx=0x7f242403f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f993390685b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.897+0000 7f241f400640 1 -- 172.21.15.59:0/2378052678 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424035040 con 0x55dfdf25dc70 2024-09-05T12:12:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.897+0000 7f241f400640 1 -- 172.21.15.59:0/2378052678 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242405a070 con 0x55dfdf25dc70 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.897+0000 7f241f400640 1 -- 172.21.15.59:0/2378052678 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242403d450 con 0x55dfdf25dc70 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- 172.21.15.59:0/2378052678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 msgr2=0x55dfdf25fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2424009230 tx=0x7f242403f070 comp rx=0 tx=0).stop 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- 172.21.15.59:0/2378052678 shutdown_connections 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf268e50 0x55dfdf281270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25e590 0x55dfdf280d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 --2- 172.21.15.59:0/2378052678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf25dc70 0x55dfdf25fe20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- 172.21.15.59:0/2378052678 >> 172.21.15.59:0/2378052678 conn(0x55dfdf25a200 msgr2=0x55dfdf25f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- 172.21.15.59:0/2378052678 shutdown_connections 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- 172.21.15.59:0/2378052678 wait complete. 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.898+0000 7f243ca84740 1 -- start start 2024-09-05T12:12:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 Processor -- start 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf2714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25e590 0x55dfdf26c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b4300 con 0x55dfdf268e50 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b5770 con 0x55dfdf25dc70 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24341b49d0 con 0x55dfdf25e590 2024-09-05T12:12:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.899+0000 7f243ca84740 10 client.0 _ll_register_callbacks cb 0x7f242c005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf2714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46166/0 (socket says 172.21.15.59:46166) 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2432a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25e590 0x55dfdf26c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 -- 172.21.15.59:0/2901247052 learned_addr learned my addr 172.21.15.59:0/2901247052 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 -- 172.21.15.59:0/2901247052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25e590 msgr2=0x55dfdf26c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25e590 0x55dfdf26c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 -- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 msgr2=0x55dfdf2714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf2714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 -- 172.21.15.59:0/2901247052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdf26d590 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f2433e00640 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f242c000f80 tx=0x7f242c09ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2b82eff2956b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.900+0000 7f241cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c0a4e60 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f242c0ba770 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242405a070 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f242c09bd70 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f243ca84740 10 client.17790 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dfdf2e3cb0 con 0x55dfdf268e50 2024-09-05T12:12:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.901+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242405a070 con 0x55dfdf268e50 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f242c09c2d0 con 0x55dfdf268e50 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 10 client.17790 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 20 client.17790 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf2e4290 0x55dfdf2e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 4 client.17790 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfdf1cbe60 con 0x55dfdf2e4290 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.902+0000 7f2433400640 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf2e4290 0x55dfdf2e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f2433400640 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf2e4290 0x55dfdf2e6680 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b04b9465a54d0 server_cookie=160e78a047e9582c in_seq=0 out_seq=0 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f241cc00640 10 client.17790 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f241cc00640 1 -- 172.21.15.59:0/2901247052 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfdf1cbe60 con 0x55dfdf2e4290 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f241cc00640 10 client.17790 handle_command_reply: tid=0 2024-09-05T12:12:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 client.17790 shutdown 2024-09-05T12:12:36.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 msgr2=0x55dfdf26cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f242c000f80 tx=0x7f242c09ba30 comp rx=0 tx=0).stop 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 shutdown_connections 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55dfdf2e4290 0x55dfdf2e6680 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfdf268e50 0x55dfdf26cdb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdf25e590 0x55dfdf26c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 --2- 172.21.15.59:0/2901247052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dfdf25dc70 0x55dfdf2714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f241cc00640 0 client.17790 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 >> 172.21.15.59:0/2901247052 conn(0x55dfdf25a200 msgr2=0x55dfdf267e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 shutdown_connections 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 1 -- 172.21.15.59:0/2901247052 wait complete. 2024-09-05T12:12:36.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.903+0000 7f243ca84740 20 client.17790 trim_cache size 0 max 16384 2024-09-05T12:12:36.909 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:12:36.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 msgr2=0x7f24180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 0x7f24180a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f243419fab0 tx=0x7f24240058d0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 msgr2=0x7f243419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:36.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f242c00eb40 tx=0x7f242c00c2d0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 shutdown_connections 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f24180a2b80 0x7f24180a5040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2434112b90 0x7f243419e970 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2434074eb0 0x7f24341a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 --2- 172.21.15.59:0/366357735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2434074590 0x7f24341a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 >> 172.21.15.59:0/366357735 conn(0x7f24340fd8c0 msgr2=0x7f24340ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 shutdown_connections 2024-09-05T12:12:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:36.904+0000 7f243a000640 1 -- 172.21.15.59:0/366357735 wait complete. 2024-09-05T12:12:38.121 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3587fa16', quiesce took 6.2 seconds. Will release after: 13.8 2024-09-05T12:12:38.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:38 smithi189 ceph-mon[34772]: pgmap v843: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:12:38.653 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:12:38.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:12:38.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:38 smithi089 ceph-mon[34499]: pgmap v843: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:12:38.707 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:38 smithi059 ceph-mon[30025]: pgmap v843: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-05T12:12:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.648+0000 7f0148a00640 1 -- 172.21.15.59:0/2161287898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0144074620 msgr2=0x7f0144074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.648+0000 7f0148a00640 1 --2- 172.21.15.59:0/2161287898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0144074620 0x7f0144074a00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f013402a480 comp rx=0 tx=0).stop 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 -- 172.21.15.59:0/2161287898 shutdown_connections 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 --2- 172.21.15.59:0/2161287898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f014410e3c0 0x7f0144110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 --2- 172.21.15.59:0/2161287898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074f40 0x7f014410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 --2- 172.21.15.59:0/2161287898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0144074620 0x7f0144074a00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 -- 172.21.15.59:0/2161287898 >> 172.21.15.59:0/2161287898 conn(0x7f01440fd8c0 msgr2=0x7f01440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 -- 172.21.15.59:0/2161287898 shutdown_connections 2024-09-05T12:12:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.649+0000 7f0148a00640 1 -- 172.21.15.59:0/2161287898 wait complete. 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 Processor -- start 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 -- start start 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 0x7f014419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0144074f40 0x7f014419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 0x7f014419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0144114170 con 0x7f014410e3c0 2024-09-05T12:12:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0144113ff0 con 0x7f0144074f40 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0148a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01441142f0 con 0x7f0144074620 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 0x7f014419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 0x7f014419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46184/0 (socket says 172.21.15.59:46184) 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0141e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 0x7f014419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0142800640 1 -- 172.21.15.59:0/505257505 learned_addr learned my addr 172.21.15.59:0/505257505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.650+0000 7f0141400640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0144074f40 0x7f014419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 -- 172.21.15.59:0/505257505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0144074f40 msgr2=0x7f014419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0144074f40 0x7f014419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 -- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 msgr2=0x7f014419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 0x7f014419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 -- 172.21.15.59:0/505257505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014419f6c0 con 0x7f0144074620 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f0141e00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 0x7f014419a710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0134005940 tx=0x7f01340040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.651+0000 7f013aa00640 1 -- 172.21.15.59:0/505257505 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013403f030 con 0x7f0144074620 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.652+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014419f950 con 0x7f0144074620 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.652+0000 7f013aa00640 1 -- 172.21.15.59:0/505257505 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01340044c0 con 0x7f0144074620 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.652+0000 7f013aa00640 1 -- 172.21.15.59:0/505257505 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134035070 con 0x7f0144074620 2024-09-05T12:12:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.652+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01441a7290 con 0x7f0144074620 2024-09-05T12:12:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 Processor -- start 2024-09-05T12:12:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f013aa00640 1 -- 172.21.15.59:0/505257505 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0134042030 con 0x7f0144074620 2024-09-05T12:12:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 -- start start 2024-09-05T12:12:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565324a4f290 con 0x565324af3ca0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5653249f6b40 con 0x565324ae8ac0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.655+0000 7f014b3ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565324a4b250 con 0x565324ae93e0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0141e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:54040/0 (socket says 172.21.15.59:54040) 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 -- 172.21.15.59:0/1837644226 learned_addr learned my addr 172.21.15.59:0/1837644226 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0141400640 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 -- 172.21.15.59:0/1837644226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 msgr2=0x565324af3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 -- 172.21.15.59:0/1837644226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 msgr2=0x565324af6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 -- 172.21.15.59:0/1837644226 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653247c06a0 con 0x565324ae8ac0 2024-09-05T12:12:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0141400640 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.656+0000 7f0142800640 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f013c011470 tx=0x7f013c011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdeb7d23671f2d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f013aa00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 0x7f01280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f012f000640 1 -- 172.21.15.59:0/1837644226 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004b20 con 0x565324ae8ac0 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f013aa00640 1 -- 172.21.15.59:0/505257505 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f01340ef730 con 0x7f0144074620 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f012f000640 1 -- 172.21.15.59:0/1837644226 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c004cc0 con 0x565324ae8ac0 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f012f000640 1 -- 172.21.15.59:0/1837644226 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0053a0 con 0x565324ae8ac0 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.657+0000 7f0141e00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 0x7f01280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- 172.21.15.59:0/1837644226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 msgr2=0x565324ae8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f0141e00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 0x7f01280a50c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f013402a9a0 tx=0x7f01340f5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f013c011470 tx=0x7f013c011930 comp rx=0 tx=0).stop 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- 172.21.15.59:0/1837644226 shutdown_connections 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1837644226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324ae8ea0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- 172.21.15.59:0/1837644226 >> 172.21.15.59:0/1837644226 conn(0x565324ae5050 msgr2=0x565324ae5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- 172.21.15.59:0/1837644226 shutdown_connections 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- 172.21.15.59:0/1837644226 wait complete. 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.658+0000 7f014b3ea740 1 -- start start 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 Processor -- start 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae8ac0 0x565324af3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 0x565324af0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565324af6910 con 0x565324ae93e0 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x565324a09fa0 con 0x565324af3ca0 2024-09-05T12:12:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565324af6a90 con 0x565324ae8ac0 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f014b3ea740 10 client.0 _ll_register_callbacks cb 0x56532475dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.659+0000 7f0141400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0141400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46208/0 (socket says 172.21.15.59:46208) 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0141400640 1 -- 172.21.15.59:0/1223228709 learned_addr learned my addr 172.21.15.59:0/1223228709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 0x565324af0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae8ac0 0x565324af3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 -- 172.21.15.59:0/1223228709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae8ac0 msgr2=0x565324af3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae8ac0 0x565324af3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 -- 172.21.15.59:0/1223228709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 msgr2=0x565324af0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 -- 172.21.15.59:0/1223228709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565324af1550 con 0x565324af3ca0 2024-09-05T12:12:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0141400640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0142800640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 0x565324af0d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f013c01b040 tx=0x7f013c00ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7050a9033dcf1f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.660+0000 7f0123400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c02d060 con 0x565324af3ca0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013c004cc0 con 0x565324af3ca0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f013c016040 con 0x565324af3ca0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f013c034050 con 0x565324af3ca0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f014b3ea740 10 client.27806 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565324b02800 con 0x565324af3ca0 2024-09-05T12:12:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c004cc0 con 0x565324af3ca0 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.661+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f013c028450 con 0x565324af3ca0 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 10 client.27806 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 10 client.27806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 20 client.27806 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b03bb0 0x565324b05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 4 client.27806 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565324a5c800 con 0x565324b03bb0 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f0141400640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b03bb0 0x565324b05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f0141400640 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b03bb0 0x565324b05fa0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a02a40ddeee982 server_cookie=b22599fb3cacf920 in_seq=0 out_seq=0 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.662+0000 7f0123400640 10 client.27806 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.663+0000 7f0123400640 1 -- 172.21.15.59:0/1223228709 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565324a5c800 con 0x565324b03bb0 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.663+0000 7f0123400640 10 client.27806 handle_command_reply: tid=0 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.663+0000 7f014b3ea740 1 client.27806 shutdown 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.663+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 msgr2=0x565324af0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.663+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 0x565324af0d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f013c01b040 tx=0x7f013c00ff60 comp rx=0 tx=0).stop 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 shutdown_connections 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324af3ca0 0x565324af0d70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b03bb0 0x565324b05fa0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae93e0 0x565324af0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 --2- 172.21.15.59:0/1223228709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae8ac0 0x565324af3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f0123400640 0 client.27806 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 >> 172.21.15.59:0/1223228709 conn(0x565324ae5050 msgr2=0x565324aead30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 shutdown_connections 2024-09-05T12:12:39.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 1 -- 172.21.15.59:0/1223228709 wait complete. 2024-09-05T12:12:39.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.664+0000 7f014b3ea740 20 client.27806 trim_cache size 0 max 16384 2024-09-05T12:12:39.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 Processor -- start 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 -- start start 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 0x565324b0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324af3ca0 0x565324b0bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01441af150 con 0x565324ae8ac0 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01441b19d0 con 0x565324ae93e0 2024-09-05T12:12:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.682+0000 7f014b3ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01441af7c0 con 0x565324af3ca0 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0141400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 0x565324b0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46220/0 (socket says 172.21.15.59:46220) 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 -- 172.21.15.59:0/4255693358 learned_addr learned my addr 172.21.15.59:0/4255693358 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324af3ca0 0x565324b0bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 -- 172.21.15.59:0/4255693358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324af3ca0 msgr2=0x565324b0bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324af3ca0 0x565324b0bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 -- 172.21.15.59:0/4255693358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 msgr2=0x565324b0b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 0x565324b0b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0142800640 1 -- 172.21.15.59:0/4255693358 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565324aeed80 con 0x565324ae8ac0 2024-09-05T12:12:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.683+0000 7f0141400640 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 0x565324b0b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f0142800640 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f013c002230 tx=0x7f013c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7d4e97ef753cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f012e600640 1 -- 172.21.15.59:0/4255693358 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c02d030 con 0x565324ae8ac0 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f012e600640 1 -- 172.21.15.59:0/4255693358 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c028bc0 con 0x565324ae8ac0 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f012e600640 1 -- 172.21.15.59:0/4255693358 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c028650 con 0x565324ae8ac0 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f014b3ea740 1 -- 172.21.15.59:0/4255693358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 msgr2=0x565324af27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.684+0000 7f014b3ea740 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f013c002230 tx=0x7f013c0096c0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 -- 172.21.15.59:0/4255693358 shutdown_connections 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324af3ca0 0x565324b0bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae93e0 0x565324b0b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 --2- 172.21.15.59:0/4255693358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324ae8ac0 0x565324af27a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 -- 172.21.15.59:0/4255693358 >> 172.21.15.59:0/4255693358 conn(0x565324ae5050 msgr2=0x565324af2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 -- 172.21.15.59:0/4255693358 shutdown_connections 2024-09-05T12:12:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 -- 172.21.15.59:0/4255693358 wait complete. 2024-09-05T12:12:39.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 -- start start 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.685+0000 7f014b3ea740 1 Processor -- start 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324afc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01441afe90 con 0x565324af3ca0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01441b1300 con 0x565324ae8ac0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01441b0560 con 0x565324ae93e0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f014b3ea740 10 client.0 _ll_register_callbacks cb 0x7f013c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f0141400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f0142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46226/0 (socket says 172.21.15.59:46226) 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f0142800640 1 -- 172.21.15.59:0/194780767 learned_addr learned my addr 172.21.15.59:0/194780767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.686+0000 7f0142800640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324afc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 -- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 msgr2=0x565324afc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324afc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 -- 172.21.15.59:0/194780767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 msgr2=0x565324af7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 -- 172.21.15.59:0/194780767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565324af8490 con 0x565324ae93e0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0142800640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0141400640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af7770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x565324b06620 tx=0x7f0130012d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d4eb31e5e6e76c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.687+0000 7f0123400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300218e0 con 0x565324ae93e0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013c028bc0 con 0x565324ae93e0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0130021a80 con 0x565324ae93e0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0130035050 con 0x565324ae93e0 2024-09-05T12:12:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f014b3ea740 10 client.27736 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565324b6e690 con 0x565324ae93e0 2024-09-05T12:12:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.688+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c028bc0 con 0x565324ae93e0 2024-09-05T12:12:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0130028410 con 0x565324ae93e0 2024-09-05T12:12:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 10 client.27736 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 10 client.27736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 20 client.27736 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b6edb0 0x565324b711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 4 client.27736 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565324a56d10 con 0x565324b6edb0 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.689+0000 7f0142800640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b6edb0 0x565324b711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.690+0000 7f0142800640 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b6edb0 0x565324b711a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f9da162cd3f91a server_cookie=31978965611d5e02 in_seq=0 out_seq=0 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.690+0000 7f0123400640 10 client.27736 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f0123400640 1 -- 172.21.15.59:0/194780767 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565324a56d10 con 0x565324b6edb0 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f0123400640 10 client.27736 handle_command_reply: tid=0 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 client.27736 shutdown 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 msgr2=0x565324af7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af7770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x565324b06620 tx=0x7f0130012d90 comp rx=0 tx=0).stop 2024-09-05T12:12:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 shutdown_connections 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x565324b6edb0 0x565324b711a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f0123400640 0 client.27736 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565324af3ca0 0x565324af7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565324ae93e0 0x565324af7770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 --2- 172.21.15.59:0/194780767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565324ae8ac0 0x565324afc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 >> 172.21.15.59:0/194780767 conn(0x565324ae5050 msgr2=0x565324af28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 shutdown_connections 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 1 -- 172.21.15.59:0/194780767 wait complete. 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.691+0000 7f014b3ea740 20 client.27736 trim_cache size 0 max 16384 2024-09-05T12:12:39.695 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:39.696 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:12:39.696 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "b378d0dd-0d26-4367-ad5d-d3856f3099a2", 2024-09-05T12:12:39.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:12:39.696 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:39.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 msgr2=0x7f01280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 0x7f01280a50c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f013402a9a0 tx=0x7f01340f5530 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 msgr2=0x7f014419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 0x7f014419a710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0134005940 tx=0x7f01340040c0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 shutdown_connections 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f01280a2c00 0x7f01280a50c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f014410e3c0 0x7f014419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0144074f40 0x7f014419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 --2- 172.21.15.59:0/505257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0144074620 0x7f014419a710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 >> 172.21.15.59:0/505257505 conn(0x7f01440fd8c0 msgr2=0x7f01440fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 shutdown_connections 2024-09-05T12:12:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:39.692+0000 7f0148a00640 1 -- 172.21.15.59:0/505257505 wait complete. 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: pgmap v844: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:40 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:12:40.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:12:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: pgmap v844: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:40.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:12:40.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:12:40.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:40 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: pgmap v844: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:40.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:40 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:12:40.801 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 -- 172.21.15.59:0/3905721322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 msgr2=0x7fbb281089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3905721322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb281089f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1002a410 comp rx=0 tx=0).stop 2024-09-05T12:12:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 -- 172.21.15.59:0/3905721322 shutdown_connections 2024-09-05T12:12:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3905721322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 0x7fbb28110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3905721322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb281089f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3905721322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 -- 172.21.15.59:0/3905721322 >> 172.21.15.59:0/3905721322 conn(0x7fbb280ffa90 msgr2=0x7fbb28101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 -- 172.21.15.59:0/3905721322 shutdown_connections 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.395+0000 7fbb2f000640 1 -- 172.21.15.59:0/3905721322 wait complete. 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.396+0000 7fbb2f000640 1 Processor -- start 2024-09-05T12:12:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.396+0000 7fbb2f000640 1 -- start start 2024-09-05T12:12:41.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.396+0000 7fbb2f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.396+0000 7fbb2f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb2819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.396+0000 7fbb2f000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 0x7fbb2819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb2f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28114140 con 0x7fbb281049a0 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb2f000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28113fc0 con 0x7fbb28109120 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb2f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281142c0 con 0x7fbb28103ff0 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb2819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36348/0 (socket says 172.21.15.59:36348) 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 -- 172.21.15.59:0/3536916644 learned_addr learned my addr 172.21.15.59:0/3536916644 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb2d000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 0x7fbb2819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 -- 172.21.15.59:0/3536916644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 msgr2=0x7fbb2819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 0x7fbb2819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 -- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 msgr2=0x7fbb2819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb2819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.397+0000 7fbb27e00640 1 -- 172.21.15.59:0/3536916644 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2819f5d0 con 0x7fbb28103ff0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e9f0 tx=0x7fbb1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb25600640 1 -- 172.21.15.59:0/3536916644 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800cd80 con 0x7fbb28103ff0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb25600640 1 -- 172.21.15.59:0/3536916644 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb18005500 con 0x7fbb28103ff0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2819f8c0 con 0x7fbb28103ff0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb25600640 1 -- 172.21.15.59:0/3536916644 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18010690 con 0x7fbb28103ff0 2024-09-05T12:12:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.398+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281a71a0 con 0x7fbb28103ff0 2024-09-05T12:12:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.401+0000 7fbb25600640 1 -- 172.21.15.59:0/3536916644 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb180046c0 con 0x7fbb28103ff0 2024-09-05T12:12:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.401+0000 7fbb31bd8740 1 Processor -- start 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.401+0000 7fbb31bd8740 1 -- start start 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5600563e0 con 0x55a5600efb90 2024-09-05T12:12:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a55fffdd80 con 0x55a5600fad70 2024-09-05T12:12:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5600523a0 con 0x55a5600f04b0 2024-09-05T12:12:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44728/0 (socket says 172.21.15.59:44728) 2024-09-05T12:12:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 -- 172.21.15.59:0/3700811791 learned_addr learned my addr 172.21.15.59:0/3700811791 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 -- 172.21.15.59:0/3700811791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 msgr2=0x55a5600fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 -- 172.21.15.59:0/3700811791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 msgr2=0x55a5600fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb2d000640 1 -- 172.21.15.59:0/3700811791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a56001cae0 con 0x55a5600efb90 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.402+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb2d000640 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb20010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2fd07f3386a29e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb25600640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 0x7fbb0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb25600640 1 -- 172.21.15.59:0/3536916644 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb180cf890 con 0x7fbb28103ff0 2024-09-05T12:12:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb1d600640 1 -- 172.21.15.59:0/3700811791 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20011700 con 0x55a5600efb90 2024-09-05T12:12:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 0x7fbb0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb1d600640 1 -- 172.21.15.59:0/3700811791 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb20011d80 con 0x55a5600efb90 2024-09-05T12:12:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.403+0000 7fbb1d600640 1 -- 172.21.15.59:0/3700811791 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20019e40 con 0x55a5600efb90 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 -- 172.21.15.59:0/3700811791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 msgr2=0x55a5600eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb27e00640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 0x7fbb0c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbb180a43c0 tx=0x7fbb180d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb20010860 comp rx=0 tx=0).stop 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 -- 172.21.15.59:0/3700811791 shutdown_connections 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/3700811791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600eff70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 -- 172.21.15.59:0/3700811791 >> 172.21.15.59:0/3700811791 conn(0x55a5600ec120 msgr2=0x55a5600ec500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 -- 172.21.15.59:0/3700811791 shutdown_connections 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.404+0000 7fbb31bd8740 1 -- 172.21.15.59:0/3700811791 wait complete. 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 -- start start 2024-09-05T12:12:41.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 Processor -- start 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600f8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5600fd9e0 con 0x55a5600efb90 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a5600110f0 con 0x55a5600fad70 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5600fdb60 con 0x55a5600f04b0 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.405+0000 7fbb31bd8740 10 client.0 _ll_register_callbacks cb 0x55a5600029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb2d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb2d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44742/0 (socket says 172.21.15.59:44742) 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36364/0 (socket says 172.21.15.59:36364) 2024-09-05T12:12:41.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb2d000640 1 -- 172.21.15.59:0/452093255 learned_addr learned my addr 172.21.15.59:0/452093255 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb2d000640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 -- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 msgr2=0x55a5600f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 -- 172.21.15.59:0/452093255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 msgr2=0x55a5600f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 -- 172.21.15.59:0/452093255 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55fdc73c0 con 0x55a5600f04b0 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb2d000640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.406+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005940 tx=0x7fbb100317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08ab7856404cd4f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:41.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10035040 con 0x55a5600f04b0 2024-09-05T12:12:41.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb20011d80 con 0x55a5600f04b0 2024-09-05T12:12:41.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb10004460 con 0x55a5600f04b0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb1004c050 con 0x55a5600f04b0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb31bd8740 10 client.27748 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a5601096a0 con 0x55a5600f04b0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb20011d80 con 0x55a5600f04b0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.407+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb1003c7d0 con 0x55a5600f04b0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 10 client.27748 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 10 client.27748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 20 client.27748 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a56010aab0 0x55a56010cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 4 client.27748 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a560063950 con 0x55a56010aab0 2024-09-05T12:12:41.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.408+0000 7fbb2d000640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a56010aab0 0x55a56010cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.409+0000 7fbb2d000640 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a56010aab0 0x55a56010cf10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3480a6448245d674 server_cookie=6782a68ac6feb5f4 in_seq=0 out_seq=0 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.409+0000 7fbb16000640 10 client.27748 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.409+0000 7fbb16000640 1 -- 172.21.15.59:0/452093255 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a560063950 con 0x55a56010aab0 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.409+0000 7fbb16000640 10 client.27748 handle_command_reply: tid=0 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.409+0000 7fbb31bd8740 1 client.27748 shutdown 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 msgr2=0x55a5600f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005940 tx=0x7fbb100317e0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 shutdown_connections 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5600f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600f04b0 0x55a5600f81e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a56010aab0 0x55a56010cf10 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/452093255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600efb90 0x55a5600f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 >> 172.21.15.59:0/452093255 conn(0x55a5600ec120 msgr2=0x55a5600f1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb16000640 0 client.27748 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:41.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 shutdown_connections 2024-09-05T12:12:41.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 1 -- 172.21.15.59:0/452093255 wait complete. 2024-09-05T12:12:41.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.410+0000 7fbb31bd8740 20 client.27748 trim_cache size 0 max 16384 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.431+0000 7fbb31bd8740 1 Processor -- start 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.431+0000 7fbb31bd8740 1 -- start start 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.431+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5601127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600fad70 0x55a5600f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281af0e0 con 0x55a5600fad70 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281b1960 con 0x55a5600f04b0 2024-09-05T12:12:41.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281af750 con 0x55a5600efb90 2024-09-05T12:12:41.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56294/0 (socket says 172.21.15.59:56294) 2024-09-05T12:12:41.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 -- 172.21.15.59:0/692395842 learned_addr learned my addr 172.21.15.59:0/692395842 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb2d000640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600fad70 0x55a5600f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb2d000640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5601127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 -- 172.21.15.59:0/692395842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 msgr2=0x55a5601127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5601127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 -- 172.21.15.59:0/692395842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600fad70 msgr2=0x55a5600f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600fad70 0x55a5600f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb1fe00640 1 -- 172.21.15.59:0/692395842 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5600ebbc0 con 0x55a5600f04b0 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.432+0000 7fbb2d000640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5601127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.433+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005c40 tx=0x7fbb1002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3906fe14900aad38 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.433+0000 7fbb1cc00640 1 -- 172.21.15.59:0/692395842 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10054020 con 0x55a5600f04b0 2024-09-05T12:12:41.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.433+0000 7fbb1cc00640 1 -- 172.21.15.59:0/692395842 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10049070 con 0x55a5600f04b0 2024-09-05T12:12:41.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.433+0000 7fbb1cc00640 1 -- 172.21.15.59:0/692395842 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10035040 con 0x55a5600f04b0 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- 172.21.15.59:0/692395842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 msgr2=0x55a560112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005c40 tx=0x7fbb1002a410 comp rx=0 tx=0).stop 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- 172.21.15.59:0/692395842 shutdown_connections 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600fad70 0x55a5600f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600f04b0 0x55a560112d20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/692395842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5601127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- 172.21.15.59:0/692395842 >> 172.21.15.59:0/692395842 conn(0x55a5600ec120 msgr2=0x55a5600f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- 172.21.15.59:0/692395842 shutdown_connections 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- 172.21.15.59:0/692395842 wait complete. 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 -- start start 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.434+0000 7fbb31bd8740 1 Processor -- start 2024-09-05T12:12:41.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5600ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5601747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281afe20 con 0x55a5600f04b0 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281b1290 con 0x55a5600fad70 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281b04f0 con 0x55a5600efb90 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb31bd8740 10 client.0 _ll_register_callbacks cb 0x7fbb10005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44754/0 (socket says 172.21.15.59:44754) 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb1fe00640 1 -- 172.21.15.59:0/1031787347 learned_addr learned my addr 172.21.15.59:0/1031787347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb2d000640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5600ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.435+0000 7fbb2d000640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5601747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 -- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 msgr2=0x55a5601747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5601747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 -- 172.21.15.59:0/1031787347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 msgr2=0x55a5600ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 -- 172.21.15.59:0/1031787347 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a560174cf0 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb2d000640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5600ff160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb200198a0 tx=0x7fbb20010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f2e9c4fbda1855 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb16000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20019e70 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10049070 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.436+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb20003970 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb20003b70 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 10 client.27754 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a560175f10 con 0x55a5600efb90 2024-09-05T12:12:41.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10049070 con 0x55a5600efb90 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb20005070 con 0x55a5600efb90 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 10 client.27754 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 10 client.27754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 20 client.27754 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a5601763a0 0x55a560178790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 4 client.27754 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.437+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a56005de60 con 0x55a5601763a0 2024-09-05T12:12:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a5601763a0 0x55a560178790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:41.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb1fe00640 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a5601763a0 0x55a560178790 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd26526fe97a545 server_cookie=82a76efc72335a67 in_seq=0 out_seq=0 2024-09-05T12:12:41.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb16000640 10 client.27754 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:41.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb16000640 1 -- 172.21.15.59:0/1031787347 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a56005de60 con 0x55a5601763a0 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb16000640 10 client.27754 handle_command_reply: tid=0 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.438+0000 7fbb31bd8740 1 client.27754 shutdown 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 msgr2=0x55a5600ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5600ff160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb200198a0 tx=0x7fbb20010860 comp rx=0 tx=0).stop 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 shutdown_connections 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5600fad70 0x55a5601747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55a5601763a0 0x55a560178790 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5600f04b0 0x55a5600ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 --2- 172.21.15.59:0/1031787347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5600efb90 0x55a5600ff160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb16000640 0 client.27754 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 >> 172.21.15.59:0/1031787347 conn(0x55a5600ec120 msgr2=0x55a5600f15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 shutdown_connections 2024-09-05T12:12:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 1 -- 172.21.15.59:0/1031787347 wait complete. 2024-09-05T12:12:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.439+0000 7fbb31bd8740 20 client.27754 trim_cache size 0 max 16384 2024-09-05T12:12:41.446 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:12:41.446 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:12:41.446 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 msgr2=0x7fbb0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 0x7fbb0c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbb180a43c0 tx=0x7fbb180d5530 comp rx=0 tx=0).stop 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 msgr2=0x7fbb2819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e9f0 tx=0x7fbb1800edf0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 shutdown_connections 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbb0c0a2b80 0x7fbb0c0a5040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28109120 0x7fbb2819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb281049a0 0x7fbb2819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 --2- 172.21.15.59:0/3536916644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb28103ff0 0x7fbb2819a620 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 >> 172.21.15.59:0/3536916644 conn(0x7fbb280ffa90 msgr2=0x7fbb28100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 shutdown_connections 2024-09-05T12:12:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:41.440+0000 7fbb2f000640 1 -- 172.21.15.59:0/3536916644 wait complete. 2024-09-05T12:12:42.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:12:42.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:42.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:42 smithi059 ceph-mon[30025]: pgmap v845: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:42.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:42 smithi189 ceph-mon[34772]: pgmap v845: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:42.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:42 smithi089 ceph-mon[34499]: pgmap v845: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 -- 172.21.15.59:0/4146640497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 msgr2=0x7f7d70114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 --2- 172.21.15.59:0/4146640497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d70114c60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009ca0 tx=0x7f7d6c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 -- 172.21.15.59:0/4146640497 shutdown_connections 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 --2- 172.21.15.59:0/4146640497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d70114c60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 --2- 172.21.15.59:0/4146640497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 0x7f7d70112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 --2- 172.21.15.59:0/4146640497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d70076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 -- 172.21.15.59:0/4146640497 >> 172.21.15.59:0/4146640497 conn(0x7f7d700ff8d0 msgr2=0x7f7d70101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 -- 172.21.15.59:0/4146640497 shutdown_connections 2024-09-05T12:12:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.159+0000 7f7d77200640 1 -- 172.21.15.59:0/4146640497 wait complete. 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.160+0000 7f7d77200640 1 Processor -- start 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.160+0000 7f7d77200640 1 -- start start 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 0x7f7d701a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d7019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70118420 con 0x7f7d70112870 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701182a0 con 0x7f7d70076720 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d77200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701185a0 con 0x7f7d70077040 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:56322/0 (socket says 172.21.15.59:56322) 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75e00640 1 -- 172.21.15.59:0/1589645620 learned_addr learned my addr 172.21.15.59:0/1589645620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75400640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 0x7f7d701a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d76800640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d7019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75e00640 1 -- 172.21.15.59:0/1589645620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 msgr2=0x7f7d701a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.161+0000 7f7d75e00640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 0x7f7d701a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d75e00640 1 -- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 msgr2=0x7f7d7019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d75e00640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d7019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d75e00640 1 -- 172.21.15.59:0/1589645620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7019efc0 con 0x7f7d70076720 2024-09-05T12:12:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d75e00640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400ea60 tx=0x7f7d6400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400cde0 con 0x7f7d70076720 2024-09-05T12:12:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d640035f0 con 0x7f7d70076720 2024-09-05T12:12:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64010670 con 0x7f7d70076720 2024-09-05T12:12:43.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7019f2b0 con 0x7f7d70076720 2024-09-05T12:12:43.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.162+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701ab430 con 0x7f7d70076720 2024-09-05T12:12:43.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.163+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d70077e60 con 0x7f7d70076720 2024-09-05T12:12:43.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.165+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d640037a0 con 0x7f7d70076720 2024-09-05T12:12:43.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.167+0000 7f7d6aa00640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 0x7f7d580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:43.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.167+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7d640cf850 con 0x7f7d70076720 2024-09-05T12:12:43.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.167+0000 7f7d75400640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 0x7f7d580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:43.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.168+0000 7f7d75400640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 0x7f7d580a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d7019ffa0 tx=0x7f7d60005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:43.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.169+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d6409c3d0 con 0x7f7d70076720 2024-09-05T12:12:43.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.389+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d70076b00 con 0x7f7d70076720 2024-09-05T12:12:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.390+0000 7f7d6aa00640 1 -- 172.21.15.59:0/1589645620 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f7d70076b00 con 0x7f7d70076720 2024-09-05T12:12:43.393 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:43.393 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 msgr2=0x7f7d580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 0x7f7d580a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d7019ffa0 tx=0x7f7d60005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 msgr2=0x7f7d701a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400ea60 tx=0x7f7d6400ef20 comp rx=0 tx=0).stop 2024-09-05T12:12:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 shutdown_connections 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f7d580a2b80 0x7f7d580a5040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d70112870 0x7f7d7019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d70077040 0x7f7d701a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 --2- 172.21.15.59:0/1589645620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d70076720 0x7f7d701a4680 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 >> 172.21.15.59:0/1589645620 conn(0x7f7d700ff8d0 msgr2=0x7f7d70101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 shutdown_connections 2024-09-05T12:12:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:43.393+0000 7f7d77200640 1 -- 172.21.15.59:0/1589645620 wait complete. 2024-09-05T12:12:44.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:12:44.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:44 smithi059 ceph-mon[30025]: pgmap v846: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:12:44.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:44 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1589645620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:44.535 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:44.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:44 smithi189 ceph-mon[34772]: pgmap v846: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:12:44.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:44 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1589645620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:44.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:44 smithi089 ceph-mon[34499]: pgmap v846: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-05T12:12:44.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:44 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1589645620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 -- 172.21.15.59:0/2889658854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37441049a0 msgr2=0x7f37441089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 --2- 172.21.15.59:0/2889658854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37441049a0 0x7f37441089f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f372c009900 tx=0x7f372c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 -- 172.21.15.59:0/2889658854 shutdown_connections 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 --2- 172.21.15.59:0/2889658854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3744109120 0x7f3744110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 --2- 172.21.15.59:0/2889658854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37441049a0 0x7f37441089f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 --2- 172.21.15.59:0/2889658854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744103ff0 0x7f37441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 -- 172.21.15.59:0/2889658854 >> 172.21.15.59:0/2889658854 conn(0x7f37440ffa90 msgr2=0x7f3744101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 -- 172.21.15.59:0/2889658854 shutdown_connections 2024-09-05T12:12:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.130+0000 7f3749000640 1 -- 172.21.15.59:0/2889658854 wait complete. 2024-09-05T12:12:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.131+0000 7f3749000640 1 Processor -- start 2024-09-05T12:12:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.131+0000 7f3749000640 1 -- start start 2024-09-05T12:12:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.131+0000 7f3749000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3749000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37441049a0 0x7f374419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3749000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744109120 0x7f374419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3749000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114190 con 0x7f3744103ff0 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3749000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114010 con 0x7f3744109120 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3749000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114310 con 0x7f37441049a0 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44786/0 (socket says 172.21.15.59:44786) 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 -- 172.21.15.59:0/2311380995 learned_addr learned my addr 172.21.15.59:0/2311380995 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3739400640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37441049a0 0x7f374419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3742800640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744109120 0x7f374419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 -- 172.21.15.59:0/2311380995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37441049a0 msgr2=0x7f374419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37441049a0 0x7f374419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 -- 172.21.15.59:0/2311380995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744109120 msgr2=0x7f374419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744109120 0x7f374419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.132+0000 7f3741e00640 1 -- 172.21.15.59:0/2311380995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374419f5a0 con 0x7f3744103ff0 2024-09-05T12:12:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f3741e00640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f373400ea60 tx=0x7f373400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f373b400640 1 -- 172.21.15.59:0/2311380995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373400cde0 con 0x7f3744103ff0 2024-09-05T12:12:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f373b400640 1 -- 172.21.15.59:0/2311380995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37340035f0 con 0x7f3744103ff0 2024-09-05T12:12:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374419f890 con 0x7f3744103ff0 2024-09-05T12:12:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a7170 con 0x7f3744103ff0 2024-09-05T12:12:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.133+0000 7f373b400640 1 -- 172.21.15.59:0/2311380995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734010670 con 0x7f3744103ff0 2024-09-05T12:12:45.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.136+0000 7f374bc66740 1 Processor -- start 2024-09-05T12:12:45.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.136+0000 7f374bc66740 1 -- start start 2024-09-05T12:12:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.136+0000 7f373b400640 1 -- 172.21.15.59:0/2311380995 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37340037a0 con 0x7f3744103ff0 2024-09-05T12:12:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x55841508f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415092240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558414feb3e0 con 0x558415085590 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558414f92d80 con 0x55841508fe50 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f374bc66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558414fe73a0 con 0x558415084c70 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36434/0 (socket says 172.21.15.59:36434) 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f3742800640 1 -- 172.21.15.59:0/4234750545 learned_addr learned my addr 172.21.15.59:0/4234750545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f3741e00640 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x55841508f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.137+0000 7f3739400640 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415092240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.138+0000 7f3742800640 1 -- 172.21.15.59:0/4234750545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 msgr2=0x558415092240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.138+0000 7f3742800640 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415092240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.138+0000 7f3742800640 1 -- 172.21.15.59:0/4234750545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 msgr2=0x55841508f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.138+0000 7f3742800640 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x55841508f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.138+0000 7f3742800640 1 -- 172.21.15.59:0/4234750545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558414fb1ae0 con 0x558415084c70 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f3742800640 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f373c009f50 tx=0x7f373c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c532fadc1733799 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f373b400640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 0x7f37280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f3733400640 1 -- 172.21.15.59:0/4234750545 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c004b20 con 0x558415084c70 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f373b400640 1 -- 172.21.15.59:0/2311380995 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f37340cf6f0 con 0x7f3744103ff0 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f3733400640 1 -- 172.21.15.59:0/4234750545 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f373c004cc0 con 0x558415084c70 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f3733400640 1 -- 172.21.15.59:0/4234750545 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c005020 con 0x558415084c70 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.139+0000 7f3741e00640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 0x7f37280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 -- 172.21.15.59:0/4234750545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 msgr2=0x558415085050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f373c009f50 tx=0x7f373c0118d0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f3741e00640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 0x7f37280a50c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37340cf990 tx=0x7f37340d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 -- 172.21.15.59:0/4234750545 shutdown_connections 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415092240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x55841508f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 --2- 172.21.15.59:0/4234750545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x558415085050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 -- 172.21.15.59:0/4234750545 >> 172.21.15.59:0/4234750545 conn(0x558415081200 msgr2=0x5584150815e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 -- 172.21.15.59:0/4234750545 shutdown_connections 2024-09-05T12:12:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.140+0000 7f374bc66740 1 -- 172.21.15.59:0/4234750545 wait complete. 2024-09-05T12:12:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.141+0000 7f374bc66740 1 -- start start 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.141+0000 7f374bc66740 1 Processor -- start 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.141+0000 7f374bc66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.141+0000 7f374bc66740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 0x55841508d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.141+0000 7f374bc66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55841508fe50 0x55841508d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f374bc66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558415092a20 con 0x55841508fe50 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f374bc66740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558414fa60f0 con 0x558415085590 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f374bc66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558415092ba0 con 0x558415084c70 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f374bc66740 10 client.0 _ll_register_callbacks cb 0x558414f8dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3742800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55841508fe50 0x55841508d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 0x55841508d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36438/0 (socket says 172.21.15.59:36438) 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3742800640 1 -- 172.21.15.59:0/1939293039 learned_addr learned my addr 172.21.15.59:0/1939293039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 -- 172.21.15.59:0/1939293039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 msgr2=0x55841508cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 -- 172.21.15.59:0/1939293039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55841508fe50 msgr2=0x55841508d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55841508fe50 0x55841508d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3739400640 1 -- 172.21.15.59:0/1939293039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558414d5c3c0 con 0x558415085590 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.142+0000 7f3742800640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f3739400640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 0x55841508d270 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f372c02a9e0 tx=0x7f372c004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26399c3d879b4cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f3727e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c035040 con 0x558415085590 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c004cc0 con 0x558415085590 2024-09-05T12:12:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f372c005450 con 0x558415085590 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f372c04f050 con 0x558415085590 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f374bc66740 10 client.27845 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.143+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55841509ea60 con 0x558415085590 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f373c004cc0 con 0x558415085590 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f372c003850 con 0x558415085590 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 10 client.27845 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 10 client.27845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 20 client.27845 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841509ff20 0x5584150a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 4 client.27845 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558414ff8950 con 0x55841509ff20 2024-09-05T12:12:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.144+0000 7f3742800640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841509ff20 0x5584150a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.145+0000 7f3742800640 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841509ff20 0x5584150a2340 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524e171b40671b79 server_cookie=9de4fa29d03783f in_seq=0 out_seq=0 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.145+0000 7f3727e00640 10 client.27845 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f3727e00640 1 -- 172.21.15.59:0/1939293039 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558414ff8950 con 0x55841509ff20 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f3727e00640 10 client.27845 handle_command_reply: tid=0 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 client.27845 shutdown 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 msgr2=0x55841508d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 0x55841508d270 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f372c02a9e0 tx=0x7f372c004390 comp rx=0 tx=0).stop 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 shutdown_connections 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841509ff20 0x5584150a2340 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55841508fe50 0x55841508d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558415085590 0x55841508d270 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 --2- 172.21.15.59:0/1939293039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 >> 172.21.15.59:0/1939293039 conn(0x558415081200 msgr2=0x558415086f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.146+0000 7f3727e00640 0 client.27845 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.147+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 shutdown_connections 2024-09-05T12:12:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.147+0000 7f374bc66740 1 -- 172.21.15.59:0/1939293039 wait complete. 2024-09-05T12:12:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.147+0000 7f374bc66740 20 client.27845 trim_cache size 0 max 16384 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.160+0000 7f374bc66740 1 Processor -- start 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.160+0000 7f374bc66740 1 -- start start 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x5584150a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x5584150a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37441aeff0 con 0x558415085590 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b1870 con 0x55841508fe50 2024-09-05T12:12:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f374bc66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37441af660 con 0x558415084c70 2024-09-05T12:12:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3739400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x5584150a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3739400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x5584150a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44802/0 (socket says 172.21.15.59:44802) 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3739400640 1 -- 172.21.15.59:0/17452421 learned_addr learned my addr 172.21.15.59:0/17452421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36450/0 (socket says 172.21.15.59:36450) 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x5584150a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 -- 172.21.15.59:0/17452421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 msgr2=0x5584150a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x5584150a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 -- 172.21.15.59:0/17452421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 msgr2=0x5584150a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x5584150a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.161+0000 7f3742800640 1 -- 172.21.15.59:0/17452421 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558415081030 con 0x558415084c70 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.162+0000 7f3742800640 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f373c009270 tx=0x7f373c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db0a9e63c02d2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.162+0000 7f3732a00640 1 -- 172.21.15.59:0/17452421 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c004000 con 0x558415084c70 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.162+0000 7f3732a00640 1 -- 172.21.15.59:0/17452421 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f373c004b10 con 0x558415084c70 2024-09-05T12:12:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.162+0000 7f3732a00640 1 -- 172.21.15.59:0/17452421 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0289f0 con 0x558415084c70 2024-09-05T12:12:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- 172.21.15.59:0/17452421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 msgr2=0x55841508b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f373c009270 tx=0x7f373c009500 comp rx=0 tx=0).stop 2024-09-05T12:12:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- 172.21.15.59:0/17452421 shutdown_connections 2024-09-05T12:12:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x5584150a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415085590 0x5584150a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 --2- 172.21.15.59:0/17452421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415084c70 0x55841508b600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- 172.21.15.59:0/17452421 >> 172.21.15.59:0/17452421 conn(0x558415081200 msgr2=0x558415086ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- 172.21.15.59:0/17452421 shutdown_connections 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- 172.21.15.59:0/17452421 wait complete. 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.163+0000 7f374bc66740 1 -- start start 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 Processor -- start 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 0x5584150938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415093e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37441afd30 con 0x558415084c70 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b11a0 con 0x55841508fe50 2024-09-05T12:12:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b0400 con 0x558415085590 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f374bc66740 10 client.0 _ll_register_callbacks cb 0x7f372c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f3742800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f3742800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44808/0 (socket says 172.21.15.59:44808) 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f3742800640 1 -- 172.21.15.59:0/3611160670 learned_addr learned my addr 172.21.15.59:0/3611160670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.164+0000 7f3739400640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 0x5584150938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3742800640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415093e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 -- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 msgr2=0x558415093e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415093e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 -- 172.21.15.59:0/3611160670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 msgr2=0x5584150985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 -- 172.21.15.59:0/3611160670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558415094640 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3742800640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3739400640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 0x5584150938d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f372c009850 tx=0x7f372c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0e324de9da3d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3727e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.165+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c005850 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c004b10 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f372c005460 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f372c062050 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 10 client.27778 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55841510ac40 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f373c004b10 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f372c004aa0 con 0x558415085590 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 10 client.27778 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 10 client.27778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.166+0000 7f374bc66740 20 client.27778 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841510b3a0 0x55841510d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f374bc66740 4 client.27778 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558414ff2e60 con 0x55841510b3a0 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f3742800640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841510b3a0 0x55841510d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f3742800640 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841510b3a0 0x55841510d7c0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a145e65d6328c6c server_cookie=cfecea07d6412415 in_seq=0 out_seq=0 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.167+0000 7f3727e00640 10 client.27778 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f3727e00640 1 -- 172.21.15.59:0/3611160670 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558414ff2e60 con 0x55841510b3a0 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f3727e00640 10 client.27778 handle_command_reply: tid=0 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 client.27778 shutdown 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 msgr2=0x5584150938d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 0x5584150938d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f372c009850 tx=0x7f372c004040 comp rx=0 tx=0).stop 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 shutdown_connections 2024-09-05T12:12:45.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55841508fe50 0x558415093e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558415085590 0x5584150938d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55841510b3a0 0x55841510d7c0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 --2- 172.21.15.59:0/3611160670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558415084c70 0x5584150985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 >> 172.21.15.59:0/3611160670 conn(0x558415081200 msgr2=0x55841508ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f3727e00640 0 client.27778 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 shutdown_connections 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 1 -- 172.21.15.59:0/3611160670 wait complete. 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.168+0000 7f374bc66740 20 client.27778 trim_cache size 0 max 16384 2024-09-05T12:12:45.176 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:12:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 msgr2=0x7f37280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 0x7f37280a50c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37340cf990 tx=0x7f37340d5530 comp rx=0 tx=0).stop 2024-09-05T12:12:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 msgr2=0x7f374419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f373400ea60 tx=0x7f373400ef20 comp rx=0 tx=0).stop 2024-09-05T12:12:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 shutdown_connections 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f37280a2c00 0x7f37280a50c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3744109120 0x7f374419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37441049a0 0x7f374419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 --2- 172.21.15.59:0/2311380995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3744103ff0 0x7f374419a5f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 >> 172.21.15.59:0/2311380995 conn(0x7f37440ffa90 msgr2=0x7f3744101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 shutdown_connections 2024-09-05T12:12:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:45.169+0000 7f3749000640 1 -- 172.21.15.59:0/2311380995 wait complete. 2024-09-05T12:12:46.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:46 smithi189 ceph-mon[34772]: pgmap v847: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:46.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:46 smithi089 ceph-mon[34499]: pgmap v847: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:46.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:46 smithi059 ceph-mon[30025]: pgmap v847: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:46.953 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:12:46.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:12:47.338 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:47.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 -- 172.21.15.59:0/3630642247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c1049a0 msgr2=0x7f798c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 --2- 172.21.15.59:0/3630642247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c1049a0 0x7f798c1089f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7978009900 tx=0x7f797802a410 comp rx=0 tx=0).stop 2024-09-05T12:12:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 -- 172.21.15.59:0/3630642247 shutdown_connections 2024-09-05T12:12:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 --2- 172.21.15.59:0/3630642247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c109120 0x7f798c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 --2- 172.21.15.59:0/3630642247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c1049a0 0x7f798c1089f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 --2- 172.21.15.59:0/3630642247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c103ff0 0x7f798c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 -- 172.21.15.59:0/3630642247 >> 172.21.15.59:0/3630642247 conn(0x7f798c0ffa90 msgr2=0x7f798c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 -- 172.21.15.59:0/3630642247 shutdown_connections 2024-09-05T12:12:47.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.914+0000 7f7993e00640 1 -- 172.21.15.59:0/3630642247 wait complete. 2024-09-05T12:12:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.915+0000 7f7993e00640 1 Processor -- start 2024-09-05T12:12:47.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.915+0000 7f7993e00640 1 -- start start 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.915+0000 7f7993e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.915+0000 7f7993e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c1049a0 0x7f798c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.915+0000 7f7993e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c109120 0x7f798c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7993e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f798c114140 con 0x7f798c109120 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7993e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f798c113fc0 con 0x7f798c1049a0 2024-09-05T12:12:47.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7993e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1142c0 con 0x7f798c103ff0 2024-09-05T12:12:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36476/0 (socket says 172.21.15.59:36476) 2024-09-05T12:12:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 -- 172.21.15.59:0/2937325596 learned_addr learned my addr 172.21.15.59:0/2937325596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:47.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c109120 0x7f798c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7990a00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c1049a0 0x7f798c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 -- 172.21.15.59:0/2937325596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c1049a0 msgr2=0x7f798c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c1049a0 0x7f798c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 -- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c109120 msgr2=0x7f798c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c109120 0x7f798c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.916+0000 7f7991400640 1 -- 172.21.15.59:0/2937325596 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c19f650 con 0x7f798c103ff0 2024-09-05T12:12:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f7991400640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f798000ebc0 tx=0x7f798000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f798a000640 1 -- 172.21.15.59:0/2937325596 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980010630 con 0x7f798c103ff0 2024-09-05T12:12:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c19f940 con 0x7f798c103ff0 2024-09-05T12:12:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f798a000640 1 -- 172.21.15.59:0/2937325596 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980010cb0 con 0x7f798c103ff0 2024-09-05T12:12:47.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f798a000640 1 -- 172.21.15.59:0/2937325596 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980018bd0 con 0x7f798c103ff0 2024-09-05T12:12:47.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.917+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1a7220 con 0x7f798c103ff0 2024-09-05T12:12:47.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.921+0000 7f7996a68740 1 Processor -- start 2024-09-05T12:12:47.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.921+0000 7f7996a68740 1 -- start start 2024-09-05T12:12:47.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.921+0000 7f7996a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.921+0000 7f7996a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92f03e0 0x5641f92fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7996a68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f92fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7996a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641f9256290 con 0x5641f92efac0 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7996a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5641f91fdb40 con 0x5641f92f03e0 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7996a68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641f9252250 con 0x5641f92faca0 2024-09-05T12:12:47.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f798a000640 1 -- 172.21.15.59:0/2937325596 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f79800107d0 con 0x7f798c103ff0 2024-09-05T12:12:47.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44834/0 (socket says 172.21.15.59:44834) 2024-09-05T12:12:47.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7991e00640 1 -- 172.21.15.59:0/121718609 learned_addr learned my addr 172.21.15.59:0/121718609 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:47.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f92fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 -- 172.21.15.59:0/121718609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92f03e0 msgr2=0x5641f92fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92f03e0 0x5641f92fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 -- 172.21.15.59:0/121718609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 msgr2=0x5641f92efea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7990a00640 1 -- 172.21.15.59:0/121718609 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f8fc76a0 con 0x5641f92faca0 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.922+0000 7f7991e00640 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:47.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.923+0000 7f7990a00640 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f92fd090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7978005940 tx=0x7f7978005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47524669b80c604 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.923+0000 7f797e600640 1 -- 172.21.15.59:0/121718609 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978040070 con 0x5641f92faca0 2024-09-05T12:12:47.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.923+0000 7f797e600640 1 -- 172.21.15.59:0/121718609 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7978031870 con 0x5641f92faca0 2024-09-05T12:12:47.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.923+0000 7f797e600640 1 -- 172.21.15.59:0/121718609 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978035040 con 0x5641f92faca0 2024-09-05T12:12:47.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.923+0000 7f798a000640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 0x7f79740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f798a000640 1 -- 172.21.15.59:0/2937325596 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f79800cfa50 con 0x7f798c103ff0 2024-09-05T12:12:47.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7991e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 0x7f79740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 -- 172.21.15.59:0/121718609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 msgr2=0x5641f92fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f92fd090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7978005940 tx=0x7f7978005be0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 -- 172.21.15.59:0/121718609 shutdown_connections 2024-09-05T12:12:47.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f92fd090 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92f03e0 0x5641f92fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 --2- 172.21.15.59:0/121718609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92efac0 0x5641f92efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 -- 172.21.15.59:0/121718609 >> 172.21.15.59:0/121718609 conn(0x5641f92ec050 msgr2=0x5641f92ec430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 -- 172.21.15.59:0/121718609 shutdown_connections 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7996a68740 1 -- 172.21.15.59:0/121718609 wait complete. 2024-09-05T12:12:47.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.924+0000 7f7991e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 0x7f79740a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7984009f50 tx=0x7f7984005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 -- start start 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 Processor -- start 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f92f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5641f92fd8b0 con 0x5641f92faca0 2024-09-05T12:12:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5641f9210fa0 con 0x5641f92efac0 2024-09-05T12:12:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641f92fda30 con 0x5641f92f03e0 2024-09-05T12:12:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.925+0000 7f7996a68740 10 client.0 _ll_register_callbacks cb 0x5641f8f64ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f92f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36504/0 (socket says 172.21.15.59:36504) 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 -- 172.21.15.59:0/2704467105 learned_addr learned my addr 172.21.15.59:0/2704467105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7991400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 -- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 msgr2=0x5641f92fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 -- 172.21.15.59:0/2704467105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 msgr2=0x5641f92f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f92f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 -- 172.21.15.59:0/2704467105 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f92f8760 con 0x5641f92f03e0 2024-09-05T12:12:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7991400640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7990a00640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f797802a930 tx=0x7f7978005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d121049458ec1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.926+0000 7f7972a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978040030 con 0x5641f92f03e0 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7978031870 con 0x5641f92f03e0 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7978031670 con 0x5641f92f03e0 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7978035070 con 0x5641f92f03e0 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7996a68740 10 client.27790 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:47.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.927+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5641f92fd8b0 con 0x5641f92f03e0 2024-09-05T12:12:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7978031870 con 0x5641f92f03e0 2024-09-05T12:12:47.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f79780035f0 con 0x5641f92f03e0 2024-09-05T12:12:47.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 10 client.27790 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:47.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 10 client.27790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:47.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 20 client.27790 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:47.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f930a9f0 0x5641f930cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 4 client.27790 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:47.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f9263800 con 0x5641f930a9f0 2024-09-05T12:12:47.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.928+0000 7f7991400640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f930a9f0 0x5641f930cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.929+0000 7f7991400640 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f930a9f0 0x5641f930cde0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e8dfd8addf3053 server_cookie=20e1c4ba08eeb0cb in_seq=0 out_seq=0 2024-09-05T12:12:47.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.929+0000 7f7972a00640 10 client.27790 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:47.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7972a00640 1 -- 172.21.15.59:0/2704467105 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641f9263800 con 0x5641f930a9f0 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7972a00640 10 client.27790 handle_command_reply: tid=0 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 client.27790 shutdown 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 msgr2=0x5641f92f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f797802a930 tx=0x7f7978005250 comp rx=0 tx=0).stop 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 shutdown_connections 2024-09-05T12:12:47.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f930a9f0 0x5641f930cde0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f92f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f7940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 --2- 172.21.15.59:0/2704467105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7972a00640 0 client.27790 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 >> 172.21.15.59:0/2704467105 conn(0x5641f92ec050 msgr2=0x5641f92f1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 shutdown_connections 2024-09-05T12:12:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.930+0000 7f7996a68740 1 -- 172.21.15.59:0/2704467105 wait complete. 2024-09-05T12:12:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.931+0000 7f7996a68740 20 client.27790 trim_cache size 0 max 16384 2024-09-05T12:12:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 Processor -- start 2024-09-05T12:12:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 -- start start 2024-09-05T12:12:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f9312500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1aefe0 con 0x5641f92faca0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b1860 con 0x5641f92efac0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.949+0000 7f7996a68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1af650 con 0x5641f92f03e0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f9312500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f9312500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44840/0 (socket says 172.21.15.59:44840) 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991e00640 1 -- 172.21.15.59:0/321496781 learned_addr learned my addr 172.21.15.59:0/321496781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7990a00640 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 -- 172.21.15.59:0/321496781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 msgr2=0x5641f92f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 -- 172.21.15.59:0/321496781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 msgr2=0x5641f9312500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f9312500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 -- 172.21.15.59:0/321496781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f9306b10 con 0x5641f92efac0 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.950+0000 7f7991400640 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92f6880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f798000eb90 tx=0x7f7980000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1767c0df5bb078c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f797dc00640 1 -- 172.21.15.59:0/321496781 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798009e050 con 0x5641f92efac0 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f797dc00640 1 -- 172.21.15.59:0/321496781 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980010ac0 con 0x5641f92efac0 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f797dc00640 1 -- 172.21.15.59:0/321496781 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980010e60 con 0x5641f92efac0 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 -- 172.21.15.59:0/321496781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 msgr2=0x5641f92f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92f6880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f798000eb90 tx=0x7f7980000f40 comp rx=0 tx=0).stop 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 -- 172.21.15.59:0/321496781 shutdown_connections 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92faca0 0x5641f9312500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92f03e0 0x5641f92f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 --2- 172.21.15.59:0/321496781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92f6880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.951+0000 7f7996a68740 1 -- 172.21.15.59:0/321496781 >> 172.21.15.59:0/321496781 conn(0x5641f92ec050 msgr2=0x5641f92f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.952+0000 7f7996a68740 1 -- 172.21.15.59:0/321496781 shutdown_connections 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.952+0000 7f7996a68740 1 -- 172.21.15.59:0/321496781 wait complete. 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.952+0000 7f7996a68740 1 -- start start 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.952+0000 7f7996a68740 1 Processor -- start 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f93744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1afd20 con 0x5641f92f03e0 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b1190 con 0x5641f92efac0 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b03f0 con 0x5641f92faca0 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7996a68740 10 client.0 _ll_register_callbacks cb 0x7f7984009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7991400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44852/0 (socket says 172.21.15.59:44852) 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7991e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f93744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 -- 172.21.15.59:0/555208507 learned_addr learned my addr 172.21.15.59:0/555208507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 -- 172.21.15.59:0/555208507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 msgr2=0x5641f93744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f93744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 -- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 msgr2=0x5641f92fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.953+0000 7f7990a00640 1 -- 172.21.15.59:0/555208507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f92ffb80 con 0x5641f92f03e0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7991400640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7990a00640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7978003b80 tx=0x7f797802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd939d387622bf31 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7972a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978040040 con 0x5641f92f03e0 2024-09-05T12:12:47.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f79780055f0 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7980010ac0 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7978009520 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7996a68740 10 client.17868 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.954+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5641f9375b60 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980010ac0 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7978004850 con 0x5641f92f03e0 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 10 client.17868 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 20 client.17868 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:47.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f93760e0 0x5641f9378500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 4 client.17868 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641f925dd10 con 0x5641f93760e0 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.955+0000 7f7991400640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f93760e0 0x5641f9378500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.956+0000 7f7991400640 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f93760e0 0x5641f9378500 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dad620ded4c3433 server_cookie=2a12e93cd6d20f71 in_seq=0 out_seq=0 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.956+0000 7f7972a00640 10 client.17868 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7972a00640 1 -- 172.21.15.59:0/555208507 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641f925dd10 con 0x5641f93760e0 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7972a00640 10 client.17868 handle_command_reply: tid=0 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 client.17868 shutdown 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 msgr2=0x5641f92ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7978003b80 tx=0x7f797802a410 comp rx=0 tx=0).stop 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 shutdown_connections 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641f92faca0 0x5641f93744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5641f93760e0 0x5641f9378500 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5641f92f03e0 0x5641f92ff2a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 --2- 172.21.15.59:0/555208507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641f92efac0 0x5641f92fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 >> 172.21.15.59:0/555208507 conn(0x5641f92ec050 msgr2=0x5641f9306320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7972a00640 0 client.17868 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.957+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 shutdown_connections 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7996a68740 1 -- 172.21.15.59:0/555208507 wait complete. 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7996a68740 20 client.17868 trim_cache size 0 max 16384 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:12:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "4e3fdb78-cab9-4910-95ed-a26ade8180d5", 2024-09-05T12:12:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:12:47.970 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:47.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 msgr2=0x7f79740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 0x7f79740a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7984009f50 tx=0x7f7984005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:47.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 msgr2=0x7f798c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f798000ebc0 tx=0x7f798000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 shutdown_connections 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f79740a2c00 0x7f79740a50c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f798c109120 0x7f798c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f798c1049a0 0x7f798c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 --2- 172.21.15.59:0/2937325596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f798c103ff0 0x7f798c19a6a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.958+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 >> 172.21.15.59:0/2937325596 conn(0x7f798c0ffa90 msgr2=0x7f798c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.959+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 shutdown_connections 2024-09-05T12:12:47.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:47.959+0000 7f7993e00640 1 -- 172.21.15.59:0/2937325596 wait complete. 2024-09-05T12:12:48.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:48 smithi189 ceph-mon[34772]: pgmap v848: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:48.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:48 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:12:48.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:48 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:48 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:12:48.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:48 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:48 smithi089 ceph-mon[34499]: pgmap v848: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:48.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:48 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:12:48.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:48 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:48 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:12:48.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:48 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:48 smithi059 ceph-mon[30025]: pgmap v848: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 4 op/s 2024-09-05T12:12:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:48 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:12:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:48 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:48 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:12:48.708 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:48 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:12:48.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:12:49.204 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 -- 172.21.15.59:0/1047454071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc8102370 msgr2=0x7fbcc810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 --2- 172.21.15.59:0/1047454071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc8102370 0x7fbcc810e8a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009c10 tx=0x7fbcc002a6e0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 -- 172.21.15.59:0/1047454071 shutdown_connections 2024-09-05T12:12:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 --2- 172.21.15.59:0/1047454071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc8102370 0x7fbcc810e8a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 --2- 172.21.15.59:0/1047454071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 --2- 172.21.15.59:0/1047454071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc81079d0 0x7fbcc8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 -- 172.21.15.59:0/1047454071 >> 172.21.15.59:0/1047454071 conn(0x7fbcc80fd710 msgr2=0x7fbcc80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.771+0000 7fbccde00640 1 -- 172.21.15.59:0/1047454071 shutdown_connections 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.772+0000 7fbccde00640 1 -- 172.21.15.59:0/1047454071 wait complete. 2024-09-05T12:12:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.772+0000 7fbccde00640 1 Processor -- start 2024-09-05T12:12:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 -- start start 2024-09-05T12:12:49.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 0x7fbcc819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81079d0 0x7fbcc819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8111d80 con 0x7fbcc81079d0 2024-09-05T12:12:49.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8111c00 con 0x7fbcc8102370 2024-09-05T12:12:49.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccde00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8111f00 con 0x7fbcc81019d0 2024-09-05T12:12:49.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 0x7fbcc819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:36528/0 (socket says 172.21.15.59:36528) 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81079d0 0x7fbcc819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.773+0000 7fbccca00640 1 -- 172.21.15.59:0/2676625746 learned_addr learned my addr 172.21.15.59:0/2676625746 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 -- 172.21.15.59:0/2676625746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 msgr2=0x7fbcc819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 0x7fbcc819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 -- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81079d0 msgr2=0x7fbcc819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81079d0 0x7fbcc819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 -- 172.21.15.59:0/2676625746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819f3f0 con 0x7fbcc81019d0 2024-09-05T12:12:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 0x7fbcc819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbccca00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e9f0 tx=0x7fbcb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbcc6000640 1 -- 172.21.15.59:0/2676625746 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb800ccc0 con 0x7fbcc81019d0 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.774+0000 7fbcc6000640 1 -- 172.21.15.59:0/2676625746 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb8010910 con 0x7fbcc81019d0 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.775+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819f6e0 con 0x7fbcc81019d0 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.775+0000 7fbcc6000640 1 -- 172.21.15.59:0/2676625746 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80188a0 con 0x7fbcc81019d0 2024-09-05T12:12:49.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.775+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a6fc0 con 0x7fbcc81019d0 2024-09-05T12:12:49.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 Processor -- start 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 -- start start 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcc6000640 1 -- 172.21.15.59:0/2676625746 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbcb8010ac0 con 0x7fbcc81019d0 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de64519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de6454320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562de63ad3e0 con 0x562de6446d50 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562de6354d80 con 0x562de6447670 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbcd2038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562de63a93a0 con 0x562de6451f30 2024-09-05T12:12:49.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.778+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44876/0 (socket says 172.21.15.59:44876) 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 -- 172.21.15.59:0/3106583131 learned_addr learned my addr 172.21.15.59:0/3106583131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccca00640 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de64519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de6454320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 -- 172.21.15.59:0/3106583131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 msgr2=0x562de6454320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de6454320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 -- 172.21.15.59:0/3106583131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 msgr2=0x562de64519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de64519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.779+0000 7fbccd400640 1 -- 172.21.15.59:0/3106583131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de6373ae0 con 0x562de6446d50 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbccd400640 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009be0 tx=0x7fbcc0005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d00ac6d7a959ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbcbdc00640 1 -- 172.21.15.59:0/3106583131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0042030 con 0x562de6446d50 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbcbdc00640 1 -- 172.21.15.59:0/3106583131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0037040 con 0x562de6446d50 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbcbdc00640 1 -- 172.21.15.59:0/3106583131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0036070 con 0x562de6446d50 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbcc6000640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 0x7fbcac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbcc6000640 1 -- 172.21.15.59:0/2676625746 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbcb80cf850 con 0x7fbcc81019d0 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.780+0000 7fbccca00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 0x7fbcac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- 172.21.15.59:0/3106583131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 msgr2=0x562de6447130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009be0 tx=0x7fbcc0005c60 comp rx=0 tx=0).stop 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- 172.21.15.59:0/3106583131 shutdown_connections 2024-09-05T12:12:49.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de6454320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de64519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3106583131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6447130 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- 172.21.15.59:0/3106583131 >> 172.21.15.59:0/3106583131 conn(0x562de64432e0 msgr2=0x562de64436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- 172.21.15.59:0/3106583131 shutdown_connections 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- 172.21.15.59:0/3106583131 wait complete. 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbccca00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 0x7fbcac0a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80a4380 tx=0x7fbcb80d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.781+0000 7fbcd2038740 1 -- start start 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 Processor -- start 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de644ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de644f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562de6454b90 con 0x562de6447670 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562de63680f0 con 0x562de6446d50 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562de6454d10 con 0x562de6451f30 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcd2038740 10 client.0 _ll_register_callbacks cb 0x562de63599e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.782+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44890/0 (socket says 172.21.15.59:44890) 2024-09-05T12:12:49.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3296870508 learned_addr learned my addr 172.21.15.59:0/3296870508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbccd400640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de644f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbccd400640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de644ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3296870508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 msgr2=0x562de644f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de644f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 msgr2=0x562de644ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de644ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3296870508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de611e3c0 con 0x562de6447670 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbccd400640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de644ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0011d40 tx=0x7fbcb000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b60e39adeae77d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.783+0000 7fbcb6000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001f060 con 0x562de6447670 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc0037040 con 0x562de6447670 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbcb0004050 con 0x562de6447670 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbcb0026050 con 0x562de6447670 2024-09-05T12:12:49.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcd2038740 10 client.17880 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562de6460940 con 0x562de6447670 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0037040 con 0x562de6447670 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbcb0019370 con 0x562de6447670 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcd2038740 10 client.17880 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.784+0000 7fbcd2038740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbcd2038740 20 client.17880 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de6461da0 0x562de64641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbcd2038740 4 client.17880 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:49.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de63ba950 con 0x562de6461da0 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbccd400640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de6461da0 0x562de64641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbccd400640 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de6461da0 0x562de64641b0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82de5a951ca2bfda server_cookie=66ba79ba9e335107 in_seq=0 out_seq=0 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.785+0000 7fbcb6000640 10 client.17880 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.786+0000 7fbcb6000640 1 -- 172.21.15.59:0/3296870508 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562de63ba950 con 0x562de6461da0 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.786+0000 7fbcb6000640 10 client.17880 handle_command_reply: tid=0 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.786+0000 7fbcd2038740 1 client.17880 shutdown 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 msgr2=0x562de644f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0011d40 tx=0x7fbcb000f790 comp rx=0 tx=0).stop 2024-09-05T12:12:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 shutdown_connections 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de644f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de6461da0 0x562de64641b0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6447670 0x562de644f3a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3296870508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de644ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 >> 172.21.15.59:0/3296870508 conn(0x562de64432e0 msgr2=0x562de6449010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcb6000640 0 client.17880 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 shutdown_connections 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 1 -- 172.21.15.59:0/3296870508 wait complete. 2024-09-05T12:12:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.787+0000 7fbcd2038740 20 client.17880 trim_cache size 0 max 16384 2024-09-05T12:12:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 Processor -- start 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 -- start start 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de64497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6447670 0x562de6469ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81aeec0 con 0x562de6451f30 2024-09-05T12:12:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b1740 con 0x562de6446d50 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.805+0000 7fbcd2038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81af530 con 0x562de6447670 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44892/0 (socket says 172.21.15.59:44892) 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6447670 0x562de6469ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 -- 172.21.15.59:0/1701613073 learned_addr learned my addr 172.21.15.59:0/1701613073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de64497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 -- 172.21.15.59:0/1701613073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6447670 msgr2=0x562de6469ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6447670 0x562de6469ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 -- 172.21.15.59:0/1701613073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 msgr2=0x562de64497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de64497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 -- 172.21.15.59:0/1701613073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de6444390 con 0x562de6451f30 2024-09-05T12:12:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.806+0000 7fbccd400640 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbcc002ac00 tx=0x7fbcc0040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257b3cb9e5b323dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcbd200640 1 -- 172.21.15.59:0/1701613073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0036040 con 0x562de6451f30 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcbd200640 1 -- 172.21.15.59:0/1701613073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc00305a0 con 0x562de6451f30 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcbd200640 1 -- 172.21.15.59:0/1701613073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0042040 con 0x562de6451f30 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 -- 172.21.15.59:0/1701613073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 msgr2=0x562de646a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbcc002ac00 tx=0x7fbcc0040040 comp rx=0 tx=0).stop 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 -- 172.21.15.59:0/1701613073 shutdown_connections 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6451f30 0x562de646a000 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6447670 0x562de6469ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 --2- 172.21.15.59:0/1701613073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6446d50 0x562de64497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.807+0000 7fbcd2038740 1 -- 172.21.15.59:0/1701613073 >> 172.21.15.59:0/1701613073 conn(0x562de64432e0 msgr2=0x562de644d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.808+0000 7fbcd2038740 1 -- 172.21.15.59:0/1701613073 shutdown_connections 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.808+0000 7fbcd2038740 1 -- 172.21.15.59:0/1701613073 wait complete. 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.808+0000 7fbcd2038740 1 -- start start 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.808+0000 7fbcd2038740 1 Processor -- start 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de6456270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de64567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81afc00 con 0x562de6446d50 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b1070 con 0x562de6447670 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81b02d0 con 0x562de6451f30 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcd2038740 10 client.0 _ll_register_callbacks cb 0x7fbcb0011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbccd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44906/0 (socket says 172.21.15.59:44906) 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbccd400640 1 -- 172.21.15.59:0/3215682788 learned_addr learned my addr 172.21.15.59:0/3215682788 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:49.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de6456270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbccd400640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de64567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3215682788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 msgr2=0x562de64567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de64567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3215682788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 msgr2=0x562de6455d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbcbfe00640 1 -- 172.21.15.59:0/3215682788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de64cbac0 con 0x562de6447670 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.809+0000 7fbccd400640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcbfe00640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de6456270 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0019790 tx=0x7fbcb00195a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ee59273c860379 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcb6000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0019b90 con 0x562de6447670 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc00305a0 con 0x562de6447670 2024-09-05T12:12:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbcb0019d30 con 0x562de6447670 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbcb0038050 con 0x562de6447670 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcd2038740 10 client.27884 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.810+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562de64ccd90 con 0x562de6447670 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc00305a0 con 0x562de6447670 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbcb0003820 con 0x562de6447670 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 10 client.27884 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 10 client.27884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 20 client.27884 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de64cd530 0x562de64cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 4 client.27884 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562de63b4e60 con 0x562de64cd530 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.811+0000 7fbccd400640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de64cd530 0x562de64cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbccd400640 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de64cd530 0x562de64cf950 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949239cf45dc486c server_cookie=519671d391bd7bec in_seq=0 out_seq=0 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcb6000640 10 client.27884 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcb6000640 1 -- 172.21.15.59:0/3215682788 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562de63b4e60 con 0x562de64cd530 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcb6000640 10 client.27884 handle_command_reply: tid=0 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 client.27884 shutdown 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 msgr2=0x562de6456270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de6456270 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0019790 tx=0x7fbcb00195a0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 shutdown_connections 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562de6451f30 0x562de64567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562de6447670 0x562de6456270 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x562de64cd530 0x562de64cf950 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 --2- 172.21.15.59:0/3215682788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562de6446d50 0x562de6455d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcb6000640 0 client.27884 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.812+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 >> 172.21.15.59:0/3215682788 conn(0x562de64432e0 msgr2=0x562de6441b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 shutdown_connections 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbcd2038740 1 -- 172.21.15.59:0/3215682788 wait complete. 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbcd2038740 20 client.27884 trim_cache size 0 max 16384 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-09-05T12:12:49.826 INFO:teuthology.orchestra.run.smithi059.stdout: "4e3fdb78-cab9-4910-95ed-a26ade8180d5": { 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "4e3fdb78-cab9-4910-95ed-a26ade8180d5", 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,force" 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:49.827 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 msgr2=0x7fbcac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 0x7fbcac0a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80a4380 tx=0x7fbcb80d5530 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 msgr2=0x7fbcc819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.813+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e9f0 tx=0x7fbcb800edf0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 shutdown_connections 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fbcac0a2b80 0x7fbcac0a5040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcc81079d0 0x7fbcc819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc8102370 0x7fbcc819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 --2- 172.21.15.59:0/2676625746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcc81019d0 0x7fbcc819a440 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 >> 172.21.15.59:0/2676625746 conn(0x7fbcc80fd710 msgr2=0x7fbcc810d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 shutdown_connections 2024-09-05T12:12:49.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:49.814+0000 7fbccde00640 1 -- 172.21.15.59:0/2676625746 wait complete. 2024-09-05T12:12:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:50 smithi059 ceph-mon[30025]: pgmap v849: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-05T12:12:50.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:50 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:12:50.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:50 smithi189 ceph-mon[34772]: pgmap v849: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-05T12:12:50.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:50 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:12:50.675 INFO:tasks.cephfs.filesystem:scrub status for tag:4e3fdb78-cab9-4910-95ed-a26ade8180d5 - {'path': '/', 'tag': '4e3fdb78-cab9-4910-95ed-a26ade8180d5', 'options': 'recursive,force'} 2024-09-05T12:12:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:50 smithi089 ceph-mon[34499]: pgmap v849: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-05T12:12:50.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:50 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:12:51.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:52.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:52 smithi189 ceph-mon[34772]: pgmap v850: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-05T12:12:52.593 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:52 smithi059 ceph-mon[30025]: pgmap v850: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-05T12:12:52.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:52 smithi089 ceph-mon[34499]: pgmap v850: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-05T12:12:52.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 -- 172.21.15.59:0/944309180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 msgr2=0x7f4da01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:52.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 --2- 172.21.15.59:0/944309180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da01089f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009900 tx=0x7f4d8802a410 comp rx=0 tx=0).stop 2024-09-05T12:12:52.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 -- 172.21.15.59:0/944309180 shutdown_connections 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 --2- 172.21.15.59:0/944309180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0109120 0x7f4da0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 --2- 172.21.15.59:0/944309180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da01089f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 --2- 172.21.15.59:0/944309180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0103ff0 0x7f4da01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.905+0000 7f4da4c00640 1 -- 172.21.15.59:0/944309180 >> 172.21.15.59:0/944309180 conn(0x7f4da00ffa90 msgr2=0x7f4da0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.906+0000 7f4da4c00640 1 -- 172.21.15.59:0/944309180 shutdown_connections 2024-09-05T12:12:52.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.906+0000 7f4da4c00640 1 -- 172.21.15.59:0/944309180 wait complete. 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.906+0000 7f4da4c00640 1 Processor -- start 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 -- start start 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0103ff0 0x7f4da019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0109120 0x7f4da019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0114140 con 0x7f4da01049a0 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0113fc0 con 0x7f4da0103ff0 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4da4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01142c0 con 0x7f4da0109120 2024-09-05T12:12:52.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4d95400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4d95400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35940/0 (socket says 172.21.15.59:35940) 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4d95400640 1 -- 172.21.15.59:0/1967933347 learned_addr learned my addr 172.21.15.59:0/1967933347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4d9e800640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0109120 0x7f4da019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.907+0000 7f4d9de00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0103ff0 0x7f4da019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 -- 172.21.15.59:0/1967933347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0109120 msgr2=0x7f4da019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0109120 0x7f4da019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 -- 172.21.15.59:0/1967933347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0103ff0 msgr2=0x7f4da019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0103ff0 0x7f4da019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 -- 172.21.15.59:0/1967933347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da019f5d0 con 0x7f4da01049a0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d95400640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8803f030 con 0x7f4da01049a0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d88004440 con 0x7f4da01049a0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88035070 con 0x7f4da01049a0 2024-09-05T12:12:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.908+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da019f860 con 0x7f4da01049a0 2024-09-05T12:12:52.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.909+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da019fcf0 con 0x7f4da01049a0 2024-09-05T12:12:52.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.910+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da01043d0 con 0x7f4da01049a0 2024-09-05T12:12:52.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.911+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d88042030 con 0x7f4da01049a0 2024-09-05T12:12:52.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.913+0000 7f4d97400640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:52.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.913+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4d880ef770 con 0x7f4da01049a0 2024-09-05T12:12:52.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.913+0000 7f4d9de00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:52.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.914+0000 7f4d9de00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 0x7f4d840a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009bd0 tx=0x7f4d90005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:52.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:52.917+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d880bbc10 con 0x7f4da01049a0 2024-09-05T12:12:53.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.140+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da002d7c0 con 0x7f4da01049a0 2024-09-05T12:12:53.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.141+0000 7f4d97400640 1 -- 172.21.15.59:0/1967933347 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f4d880bff10 con 0x7f4da01049a0 2024-09-05T12:12:53.142 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:53.143 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:53.143 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:53.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.143+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 msgr2=0x7f4d840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:53.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.143+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 0x7f4d840a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009bd0 tx=0x7f4d90005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:53.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.143+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 msgr2=0x7f4da019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:53.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.143+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006010 tx=0x7f4d88004040 comp rx=0 tx=0).stop 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 shutdown_connections 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4d840a2c00 0x7f4d840a50c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4da0109120 0x7f4da019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4da01049a0 0x7f4da019ab60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 --2- 172.21.15.59:0/1967933347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4da0103ff0 0x7f4da019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 >> 172.21.15.59:0/1967933347 conn(0x7f4da00ffa90 msgr2=0x7f4da0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 shutdown_connections 2024-09-05T12:12:53.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:53.144+0000 7f4da4c00640 1 -- 172.21.15.59:0/1967933347 wait complete. 2024-09-05T12:12:53.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:53 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1967933347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:53.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:53 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1967933347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:53.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:53 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1967933347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:53.887 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 3587fa16 --release --await' 2024-09-05T12:12:53.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 3587fa16 --release --await 2024-09-05T12:12:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:54.584 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:54 smithi059 ceph-mon[30025]: pgmap v851: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:54.584 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:12:54.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:54 smithi189 ceph-mon[34772]: pgmap v851: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:54.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:12:54.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:54 smithi089 ceph-mon[34499]: pgmap v851: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-05T12:12:54.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 -- 172.21.15.59:0/2945455375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c10e280 msgr2=0x7f452c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 --2- 172.21.15.59:0/2945455375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c10e280 0x7f452c110670 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4524009be0 tx=0x7f452402a750 comp rx=0 tx=0).stop 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 -- 172.21.15.59:0/2945455375 shutdown_connections 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 --2- 172.21.15.59:0/2945455375 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c10e280 0x7f452c110670 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 --2- 172.21.15.59:0/2945455375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c074e30 0x7f452c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 --2- 172.21.15.59:0/2945455375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 -- 172.21.15.59:0/2945455375 >> 172.21.15.59:0/2945455375 conn(0x7f452c0fd740 msgr2=0x7f452c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 -- 172.21.15.59:0/2945455375 shutdown_connections 2024-09-05T12:12:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.866+0000 7f4531400640 1 -- 172.21.15.59:0/2945455375 wait complete. 2024-09-05T12:12:54.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.867+0000 7f4531400640 1 Processor -- start 2024-09-05T12:12:54.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.867+0000 7f4531400640 1 -- start start 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 0x7f452c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 0x7f452c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c113fb0 con 0x7f452c10e280 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f452c113e30 con 0x7f452c074e30 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4531400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f452c114130 con 0x7f452c074510 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f452be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f452be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43182/0 (socket says 172.21.15.59:43182) 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f452be00640 1 -- 172.21.15.59:0/2944525322 learned_addr learned my addr 172.21.15.59:0/2944525322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 0x7f452c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4530a00640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 0x7f452c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 -- 172.21.15.59:0/2944525322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 msgr2=0x7f452c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 -- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 msgr2=0x7f452c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 0x7f452c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4523400640 1 -- 172.21.15.59:0/2944525322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19f3e0 con 0x7f452c074e30 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.868+0000 7f4530a00640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 0x7f452c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4523400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 0x7f452c19a970 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f451400e790 tx=0x7f451400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4529600640 1 -- 172.21.15.59:0/2944525322 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400eea0 con 0x7f452c074e30 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4529600640 1 -- 172.21.15.59:0/2944525322 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451400ccc0 con 0x7f452c074e30 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4529600640 1 -- 172.21.15.59:0/2944525322 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514018780 con 0x7f452c074e30 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c19f6d0 con 0x7f452c074e30 2024-09-05T12:12:54.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.869+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a6fb0 con 0x7f452c074e30 2024-09-05T12:12:54.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.872+0000 7f4535624740 1 Processor -- start 2024-09-05T12:12:54.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.872+0000 7f4529600640 1 -- 172.21.15.59:0/2944525322 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4514010430 con 0x7f452c074e30 2024-09-05T12:12:54.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.872+0000 7f4535624740 1 -- start start 2024-09-05T12:12:54.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56198c22e650 con 0x56198c2d3700 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56198c1d5f10 con 0x56198c2c8520 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4535624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56198c22a610 con 0x56198c2c8e40 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38722/0 (socket says 172.21.15.59:38722) 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f452be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 -- 172.21.15.59:0/3686921563 learned_addr learned my addr 172.21.15.59:0/3686921563 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4523400640 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 -- 172.21.15.59:0/3686921563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 msgr2=0x56198c2d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 -- 172.21.15.59:0/3686921563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 msgr2=0x56198c2d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.873+0000 7f4530a00640 1 -- 172.21.15.59:0/3686921563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56198bf3d580 con 0x56198c2c8520 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4530a00640 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f452c0754f0 tx=0x7f4524030e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb99add5be644404 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4529600640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4521200640 1 -- 172.21.15.59:0/3686921563 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524041030 con 0x56198c2c8520 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4521200640 1 -- 172.21.15.59:0/3686921563 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4524003970 con 0x56198c2c8520 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4521200640 1 -- 172.21.15.59:0/3686921563 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524036040 con 0x56198c2c8520 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f4529600640 1 -- 172.21.15.59:0/2944525322 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45140cf5b0 con 0x7f452c074e30 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.874+0000 7f452be00640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 -- 172.21.15.59:0/3686921563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 msgr2=0x56198c2c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f452c0754f0 tx=0x7f4524030e50 comp rx=0 tx=0).stop 2024-09-05T12:12:54.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 -- 172.21.15.59:0/3686921563 shutdown_connections 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 --2- 172.21.15.59:0/3686921563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2c8900 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 -- 172.21.15.59:0/3686921563 >> 172.21.15.59:0/3686921563 conn(0x56198c2c4ab0 msgr2=0x56198c2c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 -- 172.21.15.59:0/3686921563 shutdown_connections 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f452be00640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 0x7f45100a50c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f451c009620 tx=0x7f451c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.875+0000 7f4535624740 1 -- 172.21.15.59:0/3686921563 wait complete. 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 -- start start 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 Processor -- start 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56198c2d6270 con 0x56198c2d3700 2024-09-05T12:12:54.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56198c1e9350 con 0x56198c2c8520 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56198c2d63f0 con 0x56198c2c8e40 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f4535624740 10 client.0 _ll_register_callbacks cb 0x56198c20c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.876+0000 7f452be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f452be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38728/0 (socket says 172.21.15.59:38728) 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f452be00640 1 -- 172.21.15.59:0/492188740 learned_addr learned my addr 172.21.15.59:0/492188740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4530a00640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 -- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 msgr2=0x56198c2d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 -- 172.21.15.59:0/492188740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 msgr2=0x56198c2d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 -- 172.21.15.59:0/492188740 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56198c007960 con 0x56198c2c8e40 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4523400640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d0ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56198c2c5e30 tx=0x7f451409b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fddb8280b54057d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.877+0000 7f4519600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451409fd50 con 0x56198c2c8e40 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4524003970 con 0x56198c2c8e40 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4514010720 con 0x56198c2c8e40 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45140a4610 con 0x56198c2c8e40 2024-09-05T12:12:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4535624740 10 client.27832 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56198c2e1f70 con 0x56198c2c8e40 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4524003970 con 0x56198c2c8e40 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f451409af30 con 0x56198c2c8e40 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4535624740 10 client.27832 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.878+0000 7f4535624740 10 client.27832 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.879+0000 7f4535624740 20 client.27832 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.879+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c2e3380 0x56198c2e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.879+0000 7f4535624740 4 client.27832 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.879+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56198c23bbc0 con 0x56198c2e3380 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.879+0000 7f452be00640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c2e3380 0x56198c2e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.880+0000 7f452be00640 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c2e3380 0x56198c2e5770 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26172b2d5dcb2c60 server_cookie=ea46589637ae1fcd in_seq=0 out_seq=0 2024-09-05T12:12:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.880+0000 7f4519600640 10 client.27832 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.880+0000 7f4519600640 1 -- 172.21.15.59:0/492188740 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56198c23bbc0 con 0x56198c2e3380 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.880+0000 7f4519600640 10 client.27832 handle_command_reply: tid=0 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.880+0000 7f4535624740 1 client.27832 shutdown 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 msgr2=0x56198c2d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d0ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56198c2c5e30 tx=0x7f451409b910 comp rx=0 tx=0).stop 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 shutdown_connections 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c2e3380 0x56198c2e5770 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2d3700 0x56198c2d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2d0ba0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 --2- 172.21.15.59:0/492188740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4519600640 0 client.27832 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 >> 172.21.15.59:0/492188740 conn(0x56198c2c4ab0 msgr2=0x56198c2c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 shutdown_connections 2024-09-05T12:12:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 1 -- 172.21.15.59:0/492188740 wait complete. 2024-09-05T12:12:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.881+0000 7f4535624740 20 client.27832 trim_cache size 0 max 16384 2024-09-05T12:12:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.899+0000 7f4535624740 1 Processor -- start 2024-09-05T12:12:54.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.899+0000 7f4535624740 1 -- start start 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 0x56198c2eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 0x56198c2cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1aeea0 con 0x56198c2c8520 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1b1720 con 0x56198c2d3700 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4535624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1af510 con 0x56198c2c8e40 2024-09-05T12:12:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4530a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 0x56198c2cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43200/0 (socket says 172.21.15.59:43200) 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4530a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 0x56198c2cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38738/0 (socket says 172.21.15.59:38738) 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 -- 172.21.15.59:0/2055693313 learned_addr learned my addr 172.21.15.59:0/2055693313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f452be00640 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 0x56198c2eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 -- 172.21.15.59:0/2055693313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 msgr2=0x56198c2cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 0x56198c2cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 -- 172.21.15.59:0/2055693313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 msgr2=0x56198c2eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 0x56198c2eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f4523400640 1 -- 172.21.15.59:0/2055693313 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56198c2e5ae0 con 0x56198c2c8e40 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.900+0000 7f452be00640 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 0x56198c2eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.901+0000 7f4523400640 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45140058c0 tx=0x7f451409b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7676b66c7dac029f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.901+0000 7f451be00640 1 -- 172.21.15.59:0/2055693313 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451409b3b0 con 0x56198c2c8e40 2024-09-05T12:12:54.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.901+0000 7f451be00640 1 -- 172.21.15.59:0/2055693313 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45140cf070 con 0x56198c2c8e40 2024-09-05T12:12:54.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.901+0000 7f451be00640 1 -- 172.21.15.59:0/2055693313 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451409a910 con 0x56198c2c8e40 2024-09-05T12:12:54.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- 172.21.15.59:0/2055693313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 msgr2=0x56198c2eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45140058c0 tx=0x7f451409b820 comp rx=0 tx=0).stop 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- 172.21.15.59:0/2055693313 shutdown_connections 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2d3700 0x56198c2cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2c8e40 0x56198c2eb580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 --2- 172.21.15.59:0/2055693313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8520 0x56198c2eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- 172.21.15.59:0/2055693313 >> 172.21.15.59:0/2055693313 conn(0x56198c2c4ab0 msgr2=0x56198c2ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- 172.21.15.59:0/2055693313 shutdown_connections 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- 172.21.15.59:0/2055693313 wait complete. 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.902+0000 7f4535624740 1 -- start start 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 Processor -- start 2024-09-05T12:12:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8e40 0x56198c2d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 0x56198c347050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1afbe0 con 0x56198c2c8e40 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1b1050 con 0x56198c2c8520 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1b02b0 con 0x56198c2d3700 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4535624740 10 client.0 _ll_register_callbacks cb 0x7f4514010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f452be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f452be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38746/0 (socket says 172.21.15.59:38746) 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f452be00640 1 -- 172.21.15.59:0/1272018236 learned_addr learned my addr 172.21.15.59:0/1272018236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4530a00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 0x56198c347050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.903+0000 7f4523400640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8e40 0x56198c2d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 -- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 msgr2=0x56198c2d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 -- 172.21.15.59:0/1272018236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8e40 msgr2=0x56198c2d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8e40 0x56198c2d81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 -- 172.21.15.59:0/1272018236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56198c3476c0 con 0x56198c2d3700 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f452be00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4530a00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 0x56198c347050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f452402ac70 tx=0x7f4524009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6539f7466c4c3df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4519600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524036070 con 0x56198c2d3700 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4524030620 con 0x56198c2d3700 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.904+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45140cf070 con 0x56198c2d3700 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4524058050 con 0x56198c2d3700 2024-09-05T12:12:54.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 10 client.27838 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56198c34ea80 con 0x56198c2d3700 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45140cf070 con 0x56198c2d3700 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4524004650 con 0x56198c2d3700 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 10 client.27838 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 10 client.27838 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 20 client.27838 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c34ed60 0x56198c351150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 4 client.27838 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3587fa16", "release": true, "await": true}] 2024-09-05T12:12:54.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.905+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3587fa16", "release": true, "await": true}) -- 0x56198c2360d0 con 0x56198c34ed60 2024-09-05T12:12:54.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.906+0000 7f452be00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c34ed60 0x56198c351150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:54.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.906+0000 7f452be00640 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c34ed60 0x56198c351150 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9832f0ce1029ff3 server_cookie=b157052e4ad10e0a in_seq=0 out_seq=0 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.906+0000 7f4519600640 10 client.27838 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4519600640 1 -- 172.21.15.59:0/1272018236 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56198c2360d0 con 0x56198c34ed60 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4519600640 10 client.27838 handle_command_reply: tid=0 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 client.27838 shutdown 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 msgr2=0x56198c347050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 0x56198c347050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f452402ac70 tx=0x7f4524009960 comp rx=0 tx=0).stop 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 shutdown_connections 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56198c2d3700 0x56198c347050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56198c34ed60 0x56198c351150 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56198c2c8e40 0x56198c2d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 --2- 172.21.15.59:0/1272018236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56198c2c8520 0x56198c2d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 >> 172.21.15.59:0/1272018236 conn(0x56198c2c4ab0 msgr2=0x56198c2c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4519600640 0 client.27838 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 shutdown_connections 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 1 -- 172.21.15.59:0/1272018236 wait complete. 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.908+0000 7f4535624740 20 client.27838 trim_cache size 0 max 16384 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 52, 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:12:54.913 INFO:teuthology.orchestra.run.smithi059.stdout: "3587fa16": { 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 52, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 140.0, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:12:54.914 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 msgr2=0x7f45100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 0x7f45100a50c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f451c009620 tx=0x7f451c005f90 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 msgr2=0x7f452c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 0x7f452c19a970 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f451400e790 tx=0x7f451400ec50 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 shutdown_connections 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f452c10e280 0x7f452c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f452c074e30 0x7f452c19a970 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 --2- 172.21.15.59:0/2944525322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452c074510 0x7f452c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 >> 172.21.15.59:0/2944525322 conn(0x7f452c0fd740 msgr2=0x7f452c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 shutdown_connections 2024-09-05T12:12:54.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:54.909+0000 7f4531400640 1 -- 172.21.15.59:0/2944525322 wait complete. 2024-09-05T12:12:55.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:12:55.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:12:55.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:12:55.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:12:55.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:12:55.591 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:12:55.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:12:55.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:12:55.684 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:12:55.727 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3587fa16', total seconds elapsed: 23.8 2024-09-05T12:12:55.727 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.0 2024-09-05T12:12:55.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:12:56.112 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:56.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.738+0000 7f57aaa00640 1 -- 172.21.15.59:0/3382300645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 msgr2=0x7f57a4110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.738+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3382300645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a4110820 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ca0 tx=0x7f579c02a950 comp rx=0 tx=0).stop 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 -- 172.21.15.59:0/3382300645 shutdown_connections 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3382300645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a4110820 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3382300645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 0x7f57a410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3382300645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 -- 172.21.15.59:0/3382300645 >> 172.21.15.59:0/3382300645 conn(0x7f57a40fd8b0 msgr2=0x7f57a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 -- 172.21.15.59:0/3382300645 shutdown_connections 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.739+0000 7f57aaa00640 1 -- 172.21.15.59:0/3382300645 wait complete. 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 Processor -- start 2024-09-05T12:12:56.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 -- start start 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 0x7f57a419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41141b0 con 0x7f57a410e430 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4114030 con 0x7f57a4074ee0 2024-09-05T12:12:56.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.740+0000 7f57aaa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4114330 con 0x7f57a40745c0 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43234/0 (socket says 172.21.15.59:43234) 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 -- 172.21.15.59:0/3443001293 learned_addr learned my addr 172.21.15.59:0/3443001293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a8a00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f579be00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 0x7f57a419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 -- 172.21.15.59:0/3443001293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 msgr2=0x7f57a419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 0x7f57a419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 -- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 msgr2=0x7f57a419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 -- 172.21.15.59:0/3443001293 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a419f5e0 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.741+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f579400d950 tx=0x7f579400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.742+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794017070 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.742+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a419f8d0 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.742+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5794009c30 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.742+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400f6b0 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.742+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a41a71b0 con 0x7f57a40745c0 2024-09-05T12:12:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.743+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a40749a0 con 0x7f57a40745c0 2024-09-05T12:12:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.745+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57940109d0 con 0x7f57a40745c0 2024-09-05T12:12:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.746+0000 7f57a1600640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.747+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f57940cefe0 con 0x7f57a40745c0 2024-09-05T12:12:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.747+0000 7f579be00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.747+0000 7f579be00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57a410f750 tx=0x7f578c00b390 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:56.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.749+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5794009de0 con 0x7f57a40745c0 2024-09-05T12:12:56.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.976+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57a41095e0 con 0x7f57a40745c0 2024-09-05T12:12:56.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:56 smithi059 ceph-mon[30025]: pgmap v852: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-05T12:12:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.977+0000 7f57a1600640 1 -- 172.21.15.59:0/3443001293 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f579409b4c0 con 0x7f57a40745c0 2024-09-05T12:12:56.979 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:12:56.979 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:12:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:12:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 msgr2=0x7f57880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57a410f750 tx=0x7f578c00b390 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 msgr2=0x7f57a419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f579400d950 tx=0x7f579400de10 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 shutdown_connections 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f57880a2b80 0x7f57880a5040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a410e430 0x7f57a419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57a4074ee0 0x7f57a419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 --2- 172.21.15.59:0/3443001293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57a40745c0 0x7f57a419a630 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 >> 172.21.15.59:0/3443001293 conn(0x7f57a40fd8b0 msgr2=0x7f57a40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 shutdown_connections 2024-09-05T12:12:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:56.980+0000 7f57aaa00640 1 -- 172.21.15.59:0/3443001293 wait complete. 2024-09-05T12:12:57.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:56 smithi189 ceph-mon[34772]: pgmap v852: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-05T12:12:57.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:56 smithi089 ceph-mon[34499]: pgmap v852: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-05T12:12:57.728 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0' 2024-09-05T12:12:57.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0 2024-09-05T12:12:57.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:57 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3443001293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:58.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:57 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3443001293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:12:58.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:57 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3443001293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 -- 172.21.15.59:0/4229162871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c108f60 msgr2=0x7f871c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 --2- 172.21.15.59:0/4229162871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c108f60 0x7f871c110ae0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8714009ca0 tx=0x7f871402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 -- 172.21.15.59:0/4229162871 shutdown_connections 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 --2- 172.21.15.59:0/4229162871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c108f60 0x7f871c110ae0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 --2- 172.21.15.59:0/4229162871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c1047e0 0x7f871c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 --2- 172.21.15.59:0/4229162871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c103e30 0x7f871c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 -- 172.21.15.59:0/4229162871 >> 172.21.15.59:0/4229162871 conn(0x7f871c0ff8d0 msgr2=0x7f871c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 -- 172.21.15.59:0/4229162871 shutdown_connections 2024-09-05T12:12:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.711+0000 7f871be00640 1 -- 172.21.15.59:0/4229162871 wait complete. 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 Processor -- start 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 -- start start 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c1047e0 0x7f871c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c108f60 0x7f871c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f871c114010 con 0x7f871c1047e0 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f871c113e90 con 0x7f871c103e30 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.712+0000 7f871be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f871c114190 con 0x7f871c108f60 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:38766/0 (socket says 172.21.15.59:38766) 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 -- 172.21.15.59:0/3056184727 learned_addr learned my addr 172.21.15.59:0/3056184727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:58.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f8713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c1047e0 0x7f871c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871b400640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c108f60 0x7f871c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 -- 172.21.15.59:0/3056184727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c108f60 msgr2=0x7f871c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c108f60 0x7f871c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 -- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c1047e0 msgr2=0x7f871c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c1047e0 0x7f871c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 -- 172.21.15.59:0/3056184727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c19f440 con 0x7f871c103e30 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.713+0000 7f871aa00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9f0 tx=0x7f870c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.714+0000 7f8713400640 1 -- 172.21.15.59:0/3056184727 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c00cd80 con 0x7f871c103e30 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.714+0000 7f8713400640 1 -- 172.21.15.59:0/3056184727 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f870c005500 con 0x7f871c103e30 2024-09-05T12:12:58.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.714+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c19f730 con 0x7f871c103e30 2024-09-05T12:12:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.714+0000 7f8713400640 1 -- 172.21.15.59:0/3056184727 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c010690 con 0x7f871c103e30 2024-09-05T12:12:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.714+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1a7010 con 0x7f871c103e30 2024-09-05T12:12:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 Processor -- start 2024-09-05T12:12:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 -- start start 2024-09-05T12:12:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c82b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ede3bdb530 con 0x55ede3c80740 2024-09-05T12:12:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ede3b82860 con 0x55ede3c75e80 2024-09-05T12:12:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f8713400640 1 -- 172.21.15.59:0/3056184727 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f870c0046c0 con 0x7f871c103e30 2024-09-05T12:12:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.717+0000 7f872407a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ede3bd74f0 con 0x55ede3c75560 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f8713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c82b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:43272/0 (socket says 172.21.15.59:43272) 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 -- 172.21.15.59:0/2599235410 learned_addr learned my addr 172.21.15.59:0/2599235410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871aa00640 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 -- 172.21.15.59:0/2599235410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 msgr2=0x55ede3c80200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c80200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 -- 172.21.15.59:0/2599235410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 msgr2=0x55ede3c82b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c82b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 -- 172.21.15.59:0/2599235410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede394cb90 con 0x55ede3c75560 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f8713e00640 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c82b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.718+0000 7f871b400640 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8714009ca0 tx=0x7f87140093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09b28b7b2b31202 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f870b400640 1 -- 172.21.15.59:0/2599235410 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714031db0 con 0x55ede3c75560 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f870b400640 1 -- 172.21.15.59:0/2599235410 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8714036030 con 0x55ede3c75560 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f870b400640 1 -- 172.21.15.59:0/2599235410 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140042a0 con 0x55ede3c75560 2024-09-05T12:12:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f8713400640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f8713400640 1 -- 172.21.15.59:0/3056184727 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f870c0cf890 con 0x7f871c103e30 2024-09-05T12:12:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f871aa00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f872407a740 1 -- 172.21.15.59:0/2599235410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 msgr2=0x55ede3c75940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.719+0000 7f872407a740 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8714009ca0 tx=0x7f87140093e0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 -- 172.21.15.59:0/2599235410 shutdown_connections 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c82b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c80200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 --2- 172.21.15.59:0/2599235410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75560 0x55ede3c75940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 -- 172.21.15.59:0/2599235410 >> 172.21.15.59:0/2599235410 conn(0x55ede3c71af0 msgr2=0x55ede3c71ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 -- 172.21.15.59:0/2599235410 shutdown_connections 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 -- 172.21.15.59:0/2599235410 wait complete. 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f871aa00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 0x7f87000a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f870c0a43c0 tx=0x7f870c0d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.720+0000 7f872407a740 1 -- start start 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 Processor -- start 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 0x55ede3c7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ede3c83130 con 0x55ede3c80740 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ede3b95e60 con 0x55ede3c75560 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ede3c832b0 con 0x55ede3c75e80 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f872407a740 10 client.0 _ll_register_callbacks cb 0x55ede3b9acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f871b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f8713e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 0x55ede3c7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36058/0 (socket says 172.21.15.59:36058) 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.721+0000 7f871b400640 1 -- 172.21.15.59:0/4162243285 learned_addr learned my addr 172.21.15.59:0/4162243285 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 -- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 msgr2=0x55ede3c7d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 -- 172.21.15.59:0/4162243285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 msgr2=0x55ede3c7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 -- 172.21.15.59:0/4162243285 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede3ba1990 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f871b400640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f8713e00640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 0x55ede3c7db70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f871c1000f0 tx=0x7f8704012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f0efbeec57772d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f86ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.722+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004050 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f87040041f0 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8714036030 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87040043f0 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f872407a740 10 client.27856 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ede3c8eda0 con 0x55ede3c75e80 2024-09-05T12:12:58.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8714036030 con 0x55ede3c75e80 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.723+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f87040039c0 con 0x55ede3c75e80 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 10 client.27856 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 10 client.27856 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 20 client.27856 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3c90260 0x55ede3c92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 4 client.27856 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:12:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ede3b7e310 con 0x55ede3c90260 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.724+0000 7f871b400640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3c90260 0x55ede3c92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.725+0000 7f871b400640 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3c90260 0x55ede3c92680 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9d10c2c2fc6c83 server_cookie=375a9396d9b5aaba in_seq=0 out_seq=0 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.725+0000 7f86ffe00640 10 client.27856 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.725+0000 7f86ffe00640 1 -- 172.21.15.59:0/4162243285 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ede3b7e310 con 0x55ede3c90260 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.725+0000 7f86ffe00640 10 client.27856 handle_command_reply: tid=0 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.725+0000 7f872407a740 1 client.27856 shutdown 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 msgr2=0x55ede3c7db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 0x55ede3c7db70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f871c1000f0 tx=0x7f8704012a80 comp rx=0 tx=0).stop 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 shutdown_connections 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3c90260 0x55ede3c92680 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c80740 0x55ede3c7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c75e80 0x55ede3c7db70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 --2- 172.21.15.59:0/4162243285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f86ffe00640 0 client.27856 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 >> 172.21.15.59:0/4162243285 conn(0x55ede3c71af0 msgr2=0x55ede3c70c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 shutdown_connections 2024-09-05T12:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 1 -- 172.21.15.59:0/4162243285 wait complete. 2024-09-05T12:12:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.726+0000 7f872407a740 20 client.27856 trim_cache size 0 max 16384 2024-09-05T12:12:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.744+0000 7f872407a740 1 Processor -- start 2024-09-05T12:12:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.744+0000 7f872407a740 1 -- start start 2024-09-05T12:12:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c97f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1aee60 con 0x55ede3c75e80 2024-09-05T12:12:58.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b16e0 con 0x55ede3c75560 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f872407a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1af4d0 con 0x55ede3c80740 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f8713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f8713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36072/0 (socket says 172.21.15.59:36072) 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f8713e00640 1 -- 172.21.15.59:0/273377365 learned_addr learned my addr 172.21.15.59:0/273377365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c97f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 -- 172.21.15.59:0/273377365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 msgr2=0x55ede3c97f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c97f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 -- 172.21.15.59:0/273377365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 msgr2=0x55ede3c7e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f871b400640 1 -- 172.21.15.59:0/273377365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede3c6c160 con 0x55ede3c75560 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.745+0000 7f8713e00640 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.746+0000 7f871b400640 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7dd00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87140318d0 tx=0x7f8714005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f7a7257224c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.746+0000 7f870aa00640 1 -- 172.21.15.59:0/273377365 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871404a040 con 0x55ede3c75560 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.746+0000 7f870aa00640 1 -- 172.21.15.59:0/273377365 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8714036070 con 0x55ede3c75560 2024-09-05T12:12:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.746+0000 7f870aa00640 1 -- 172.21.15.59:0/273377365 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871402f610 con 0x55ede3c75560 2024-09-05T12:12:58.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- 172.21.15.59:0/273377365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 msgr2=0x55ede3c7dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7dd00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87140318d0 tx=0x7f8714005df0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- 172.21.15.59:0/273377365 shutdown_connections 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c97f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75e80 0x55ede3c7e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 --2- 172.21.15.59:0/273377365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75560 0x55ede3c7dd00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- 172.21.15.59:0/273377365 >> 172.21.15.59:0/273377365 conn(0x55ede3c71af0 msgr2=0x55ede3c7d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- 172.21.15.59:0/273377365 shutdown_connections 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- 172.21.15.59:0/273377365 wait complete. 2024-09-05T12:12:58.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.747+0000 7f872407a740 1 -- start start 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 Processor -- start 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1afba0 con 0x55ede3c75560 2024-09-05T12:12:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b1010 con 0x55ede3c75e80 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1b0270 con 0x55ede3c80740 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f872407a740 10 client.0 _ll_register_callbacks cb 0x7f8704009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36086/0 (socket says 172.21.15.59:36086) 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f871b400640 1 -- 172.21.15.59:0/3114803830 learned_addr learned my addr 172.21.15.59:0/3114803830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:12:58.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.748+0000 7f8713e00640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 -- 172.21.15.59:0/3114803830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 msgr2=0x55ede3c851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 -- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 msgr2=0x55ede3c84ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:12:58.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c84ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 -- 172.21.15.59:0/3114803830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede3c783d0 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f871b400640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f871402ade0 tx=0x7f8714009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bd3319c5d9d913 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f86ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714065060 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8714004200 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.749+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8714036070 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f871404a040 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 10 client.17934 fetch_fsmap learned FSMap version 21 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ede3cfba50 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8714036070 con 0x55ede3c75560 2024-09-05T12:12:58.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8714043070 con 0x55ede3c75560 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 10 client.17934 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 10 client.17934 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 20 client.17934 populate_metadata read hostname 'smithi059' 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3cfbda0 0x55ede3cfe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 4 client.17934 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}] 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.750+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}) -- 0x55ede3be8c60 con 0x55ede3cfbda0 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.751+0000 7f8713e00640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3cfbda0 0x55ede3cfe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.751+0000 7f8713e00640 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3cfbda0 0x55ede3cfe190 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb0e9c966c925a9 server_cookie=326c98c7a592f730 in_seq=0 out_seq=0 2024-09-05T12:12:58.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.751+0000 7f86ffe00640 10 client.17934 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:58.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.761+0000 7f86ffe00640 1 -- 172.21.15.59:0/3114803830 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ede3be8c60 con 0x55ede3cfbda0 2024-09-05T12:12:58.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.761+0000 7f86ffe00640 10 client.17934 handle_command_reply: tid=0 2024-09-05T12:12:58.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.761+0000 7f872407a740 1 client.17934 shutdown 2024-09-05T12:12:58.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 msgr2=0x55ede3c88960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f871402ade0 tx=0x7f8714009bd0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 shutdown_connections 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ede3c80740 0x55ede3c851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ede3c75e80 0x55ede3c84ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55ede3cfbda0 0x55ede3cfe190 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 --2- 172.21.15.59:0/3114803830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ede3c75560 0x55ede3c88960 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 >> 172.21.15.59:0/3114803830 conn(0x55ede3c71af0 msgr2=0x55ede3c7de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f86ffe00640 0 client.17934 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 shutdown_connections 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 1 -- 172.21.15.59:0/3114803830 wait complete. 2024-09-05T12:12:58.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.762+0000 7f872407a740 20 client.17934 trim_cache size 0 max 16384 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 54, 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "2497c9ed": { 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 54, 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:58.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 140.0, 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:58.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:58.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:12:58.767 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:12:58.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 msgr2=0x7f87000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 0x7f87000a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f870c0a43c0 tx=0x7f870c0d5530 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 msgr2=0x7f871c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9f0 tx=0x7f870c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 shutdown_connections 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f871c108f60 0x7f871c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c1047e0 0x7f871c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 --2- 172.21.15.59:0/3056184727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f871c103e30 0x7f871c19a490 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 >> 172.21.15.59:0/3056184727 conn(0x7f871c0ff8d0 msgr2=0x7f871c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 shutdown_connections 2024-09-05T12:12:58.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:12:58.763+0000 7f871be00640 1 -- 172.21.15.59:0/3056184727 wait complete. 2024-09-05T12:12:58.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:12:58 smithi059 ceph-mon[30025]: pgmap v853: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:12:59.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:12:58 smithi189 ceph-mon[34772]: pgmap v853: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:12:59.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:12:58 smithi089 ceph-mon[34499]: pgmap v853: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:12:59.546 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2497c9ed', quiesce took 3.8 seconds. Will release after: 16.2 2024-09-05T12:13:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:00 smithi189 ceph-mon[34772]: pgmap v854: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-05T12:13:01.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:00 smithi089 ceph-mon[34499]: pgmap v854: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-05T12:13:01.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:00 smithi059 ceph-mon[30025]: pgmap v854: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-05T12:13:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:02 smithi189 ceph-mon[34772]: pgmap v855: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:13:03.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:02 smithi089 ceph-mon[34499]: pgmap v855: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:13:03.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:02 smithi059 ceph-mon[30025]: pgmap v855: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-09-05T12:13:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:04 smithi189 ceph-mon[34772]: pgmap v856: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.9 KiB/s wr, 4 op/s 2024-09-05T12:13:05.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:04 smithi089 ceph-mon[34499]: pgmap v856: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.9 KiB/s wr, 4 op/s 2024-09-05T12:13:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:04 smithi059 ceph-mon[30025]: pgmap v856: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.9 KiB/s wr, 4 op/s 2024-09-05T12:13:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:06 smithi189 ceph-mon[34772]: pgmap v857: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:13:07.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:06 smithi089 ceph-mon[34499]: pgmap v857: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:13:07.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:06 smithi059 ceph-mon[30025]: pgmap v857: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-09-05T12:13:09.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:08 smithi189 ceph-mon[34772]: pgmap v858: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:13:09.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:08 smithi089 ceph-mon[34499]: pgmap v858: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:13:09.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:08 smithi059 ceph-mon[30025]: pgmap v858: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-05T12:13:11.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:10 smithi189 ceph-mon[34772]: pgmap v859: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 2 op/s 2024-09-05T12:13:11.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:10 smithi089 ceph-mon[34499]: pgmap v859: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 2 op/s 2024-09-05T12:13:11.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:10 smithi059 ceph-mon[30025]: pgmap v859: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 2 op/s 2024-09-05T12:13:13.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:12 smithi189 ceph-mon[34772]: pgmap v860: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:13.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:12 smithi089 ceph-mon[34499]: pgmap v860: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:13.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:12 smithi059 ceph-mon[30025]: pgmap v860: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:15.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:14 smithi089 ceph-mon[34499]: pgmap v861: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:15.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:14 smithi059 ceph-mon[30025]: pgmap v861: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:15.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:14 smithi189 ceph-mon[34772]: pgmap v861: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.1 KiB/s wr, 3 op/s 2024-09-05T12:13:15.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:16.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 -- 172.21.15.59:0/1658241341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5310101df0 msgr2=0x7f53101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 --2- 172.21.15.59:0/1658241341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5310101df0 0x7f53101021d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5300009900 tx=0x7f530002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 -- 172.21.15.59:0/1658241341 shutdown_connections 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 --2- 172.21.15.59:0/1658241341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 0x7f531010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 --2- 172.21.15.59:0/1658241341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53101027a0 0x7f53101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 --2- 172.21.15.59:0/1658241341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5310101df0 0x7f53101021d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 -- 172.21.15.59:0/1658241341 >> 172.21.15.59:0/1658241341 conn(0x7f53100fd8b0 msgr2=0x7f53100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 -- 172.21.15.59:0/1658241341 shutdown_connections 2024-09-05T12:13:16.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.723+0000 7f5316800640 1 -- 172.21.15.59:0/1658241341 wait complete. 2024-09-05T12:13:16.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 Processor -- start 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 -- start start 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 0x7f53100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 0x7f5310074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 0x7f5310074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5310111f80 con 0x7f53101027a0 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5310111e00 con 0x7f5310101df0 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.724+0000 7f5316800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5310112100 con 0x7f5310106f20 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f530f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 0x7f53100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 0x7f5310074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f530f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 0x7f53100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46630/0 (socket says 172.21.15.59:46630) 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f530f400640 1 -- 172.21.15.59:0/2646168701 learned_addr learned my addr 172.21.15.59:0/2646168701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f530fe00640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 0x7f5310074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 -- 172.21.15.59:0/2646168701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 msgr2=0x7f5310074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 0x7f5310074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 -- 172.21.15.59:0/2646168701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 msgr2=0x7f53100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 0x7f53100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.725+0000 7f5306a00640 1 -- 172.21.15.59:0/2646168701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53101aad90 con 0x7f53101027a0 2024-09-05T12:13:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f5306a00640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 0x7f5310074a30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f52f800e9f0 tx=0x7f52f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f800cd80 con 0x7f53101027a0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f80035f0 con 0x7f53101027a0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8010690 con 0x7f53101027a0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53101aafc0 con 0x7f53101027a0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.726+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101ab470 con 0x7f53101027a0 2024-09-05T12:13:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.727+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53101032d0 con 0x7f53101027a0 2024-09-05T12:13:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.730+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52f8003b30 con 0x7f53101027a0 2024-09-05T12:13:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.731+0000 7f530cc00640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 0x7f52f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.731+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f52f80cf690 con 0x7f53101027a0 2024-09-05T12:13:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.732+0000 7f530f400640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 0x7f52f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.732+0000 7f530f400640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 0x7f52f40a50c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53000096c0 tx=0x7f5300005a20 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.735+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52f809c180 con 0x7f53101027a0 2024-09-05T12:13:16.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:16 smithi059 ceph-mon[30025]: pgmap v862: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:13:16.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.963+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53101021d0 con 0x7f53101027a0 2024-09-05T12:13:16.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.963+0000 7f530cc00640 1 -- 172.21.15.59:0/2646168701 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f53101021d0 con 0x7f53101027a0 2024-09-05T12:13:16.965 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:16.966 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:16.966 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:16.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 msgr2=0x7f52f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:16.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 0x7f52f40a50c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f53000096c0 tx=0x7f5300005a20 comp rx=0 tx=0).stop 2024-09-05T12:13:16.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 msgr2=0x7f5310074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:16.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 0x7f5310074a30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f52f800e9f0 tx=0x7f52f800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 shutdown_connections 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52f40a2c00 0x7f52f40a50c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5310106f20 0x7f5310074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53101027a0 0x7f5310074a30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 --2- 172.21.15.59:0/2646168701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310101df0 0x7f53100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 >> 172.21.15.59:0/2646168701 conn(0x7f53100fd8b0 msgr2=0x7f53100feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 shutdown_connections 2024-09-05T12:13:16.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:16.966+0000 7f5316800640 1 -- 172.21.15.59:0/2646168701 wait complete. 2024-09-05T12:13:17.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:16 smithi089 ceph-mon[34499]: pgmap v862: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:13:17.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:16 smithi189 ceph-mon[34772]: pgmap v862: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-05T12:13:17.772 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --set-id 2497c9ed --release --await' 2024-09-05T12:13:17.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --set-id 2497c9ed --release --await 2024-09-05T12:13:18.150 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:18.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:17 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2646168701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:18.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:17 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2646168701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:18.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:17 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2646168701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 -- 172.21.15.59:0/3639463248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b01049a0 msgr2=0x7f85b01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 --2- 172.21.15.59:0/3639463248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b01049a0 0x7f85b01089f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a410 comp rx=0 tx=0).stop 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 -- 172.21.15.59:0/3639463248 shutdown_connections 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 --2- 172.21.15.59:0/3639463248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0109120 0x7f85b0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 --2- 172.21.15.59:0/3639463248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b01049a0 0x7f85b01089f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 --2- 172.21.15.59:0/3639463248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b0103ff0 0x7f85b01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.743+0000 7f85b6c00640 1 -- 172.21.15.59:0/3639463248 >> 172.21.15.59:0/3639463248 conn(0x7f85b00ffa90 msgr2=0x7f85b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.744+0000 7f85b6c00640 1 -- 172.21.15.59:0/3639463248 shutdown_connections 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.744+0000 7f85b6c00640 1 -- 172.21.15.59:0/3639463248 wait complete. 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.744+0000 7f85b6c00640 1 Processor -- start 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 -- start start 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b01049a0 0x7f85b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0114140 con 0x7f85b0109120 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0113fc0 con 0x7f85b0103ff0 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b6c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01142c0 con 0x7f85b01049a0 2024-09-05T12:13:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85afe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85afe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46660/0 (socket says 172.21.15.59:46660) 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85afe00640 1 -- 172.21.15.59:0/213891167 learned_addr learned my addr 172.21.15.59:0/213891167 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85b4c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49462/0 (socket says 172.21.15.59:49462) 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.745+0000 7f85af400640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b01049a0 0x7f85b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 -- 172.21.15.59:0/213891167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b01049a0 msgr2=0x7f85b019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b01049a0 0x7f85b019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 -- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 msgr2=0x7f85b019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 -- 172.21.15.59:0/213891167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b019f5d0 con 0x7f85b0103ff0 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85b4c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85afe00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85a000e9f0 tx=0x7f85a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85acc00640 1 -- 172.21.15.59:0/213891167 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000cd80 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b019f8c0 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85acc00640 1 -- 172.21.15.59:0/213891167 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0005500 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.746+0000 7f85acc00640 1 -- 172.21.15.59:0/213891167 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0010690 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.747+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a71a0 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 Processor -- start 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85acc00640 1 -- 172.21.15.59:0/213891167 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85a00046c0 con 0x7f85b0103ff0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 -- start start 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f0937648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0936bd650 con 0x55f093757310 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f093664f10 con 0x55f0937624f0 2024-09-05T12:13:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.750+0000 7f85b964d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0936b9610 con 0x55f093757c30 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57426/0 (socket says 172.21.15.59:57426) 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 -- 172.21.15.59:0/452015193 learned_addr learned my addr 172.21.15.59:0/452015193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85af400640 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f0937648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 -- 172.21.15.59:0/452015193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 msgr2=0x55f0937648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f0937648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 -- 172.21.15.59:0/452015193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 msgr2=0x55f0937576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.751+0000 7f85afe00640 1 -- 172.21.15.59:0/452015193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0933cc580 con 0x55f093757c30 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85acc00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 0x7f85940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85acc00640 1 -- 172.21.15.59:0/213891167 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f85a00cf830 con 0x7f85b0103ff0 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85afe00640 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85a00d5a90 tx=0x7f85a00d5b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfc7f0a47ff33315 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85b4c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 0x7f85940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85a5200640 1 -- 172.21.15.59:0/452015193 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0097ec0 con 0x55f093757c30 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85a5200640 1 -- 172.21.15.59:0/452015193 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a009c320 con 0x55f093757c30 2024-09-05T12:13:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.752+0000 7f85a5200640 1 -- 172.21.15.59:0/452015193 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a009b580 con 0x55f093757c30 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 -- 172.21.15.59:0/452015193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 msgr2=0x55f093761fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b4c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 0x7f85940a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009fd0 tx=0x7f85a8009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85a00d5a90 tx=0x7f85a00d5b80 comp rx=0 tx=0).stop 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 -- 172.21.15.59:0/452015193 shutdown_connections 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f0937648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f093761fb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 --2- 172.21.15.59:0/452015193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.753+0000 7f85b964d740 1 -- 172.21.15.59:0/452015193 >> 172.21.15.59:0/452015193 conn(0x55f0937538a0 msgr2=0x55f093753c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.754+0000 7f85b964d740 1 -- 172.21.15.59:0/452015193 shutdown_connections 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.754+0000 7f85b964d740 1 -- 172.21.15.59:0/452015193 wait complete. 2024-09-05T12:13:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.754+0000 7f85b964d740 1 -- start start 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 Processor -- start 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09375f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f09375fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0937650c0 con 0x55f093757310 2024-09-05T12:13:18.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f093678350 con 0x55f093757c30 2024-09-05T12:13:18.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f093765240 con 0x55f0937624f0 2024-09-05T12:13:18.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85b964d740 10 client.0 _ll_register_callbacks cb 0x55f09369b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:18.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85af400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85af400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46672/0 (socket says 172.21.15.59:46672) 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.755+0000 7f85af400640 1 -- 172.21.15.59:0/2169351898 learned_addr learned my addr 172.21.15.59:0/2169351898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85b4c00640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f09375fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85afe00640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09375f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 -- 172.21.15.59:0/2169351898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 msgr2=0x55f09375fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f09375fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 -- 172.21.15.59:0/2169351898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 msgr2=0x55f09375f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09375f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 -- 172.21.15.59:0/2169351898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f093496960 con 0x55f093757c30 2024-09-05T12:13:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a009c320 con 0x55f093757c30 2024-09-05T12:13:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85b4c00640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f09375fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f85af400640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f859c02a930 tx=0x7f859c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3847f5551acf6f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.756+0000 7f8599600640 10 client.27950 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f8599600640 1 -- 172.21.15.59:0/2169351898 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c035040 con 0x55f093757c30 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f8599600640 1 -- 172.21.15.59:0/2169351898 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f859c004240 con 0x55f093757c30 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f8599600640 1 -- 172.21.15.59:0/2169351898 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f859c004440 con 0x55f093757c30 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f85b964d740 10 client.27950 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55f093770cc0 con 0x55f093757c30 2024-09-05T12:13:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.757+0000 7f8599600640 1 -- 172.21.15.59:0/2169351898 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f859c0035f0 con 0x55f093757c30 2024-09-05T12:13:18.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 10 client.27950 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:18.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 10 client.27950 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:13:18.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 20 client.27950 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:18.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f093772070 0x55f0937744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 4 client.27950 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0936cabc0 con 0x55f093772070 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85afe00640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f093772070 0x55f0937744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f85afe00640 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f093772070 0x55f0937744e0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886e7433756648e0 server_cookie=422185e2c534e567 in_seq=0 out_seq=0 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.758+0000 7f8599600640 10 client.27950 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f8599600640 1 -- 172.21.15.59:0/2169351898 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0936cabc0 con 0x55f093772070 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f8599600640 10 client.27950 handle_command_reply: tid=0 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f85b964d740 1 client.27950 shutdown 2024-09-05T12:13:18.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 msgr2=0x55f09375f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f859c02a930 tx=0x7f859c031850 comp rx=0 tx=0).stop 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.759+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 shutdown_connections 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f09375fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f09375f9b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f093772070 0x55f0937744e0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 --2- 172.21.15.59:0/2169351898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09375f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f8599600640 0 client.27950 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 >> 172.21.15.59:0/2169351898 conn(0x55f0937538a0 msgr2=0x55f0937595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 shutdown_connections 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 1 -- 172.21.15.59:0/2169351898 wait complete. 2024-09-05T12:13:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.760+0000 7f85b964d740 20 client.27950 trim_cache size 0 max 16384 2024-09-05T12:13:18.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 Processor -- start 2024-09-05T12:13:18.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 -- start start 2024-09-05T12:13:18.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f093779db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01af050 con 0x55f093757310 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b18d0 con 0x55f0937624f0 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.778+0000 7f85b964d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01af6c0 con 0x55f093757c30 2024-09-05T12:13:18.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f093779db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57444/0 (socket says 172.21.15.59:57444) 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 -- 172.21.15.59:0/3336029103 learned_addr learned my addr 172.21.15.59:0/3336029103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f093779db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46688/0 (socket says 172.21.15.59:46688) 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 -- 172.21.15.59:0/3336029103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 msgr2=0x55f093779db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f093779db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 -- 172.21.15.59:0/3336029103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 msgr2=0x55f0937528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 -- 172.21.15.59:0/3336029103 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f093758890 con 0x55f093757c30 2024-09-05T12:13:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.779+0000 7f85af400640 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd9cc1dc06c2f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.780+0000 7f859be00640 1 -- 172.21.15.59:0/3336029103 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c056070 con 0x55f093757c30 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.780+0000 7f859be00640 1 -- 172.21.15.59:0/3336029103 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a009c320 con 0x55f093757c30 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.780+0000 7f859be00640 1 -- 172.21.15.59:0/3336029103 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c035070 con 0x55f093757c30 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- 172.21.15.59:0/3336029103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 msgr2=0x55f09375dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c03f000 comp rx=0 tx=0).stop 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- 172.21.15.59:0/3336029103 shutdown_connections 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f0937624f0 0x55f093779db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f093757c30 0x55f09375dd40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 --2- 172.21.15.59:0/3336029103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f0937528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- 172.21.15.59:0/3336029103 >> 172.21.15.59:0/3336029103 conn(0x55f0937538a0 msgr2=0x55f0937595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- 172.21.15.59:0/3336029103 shutdown_connections 2024-09-05T12:13:18.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- 172.21.15.59:0/3336029103 wait complete. 2024-09-05T12:13:18.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.781+0000 7f85b964d740 1 -- start start 2024-09-05T12:13:18.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 Processor -- start 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09376ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f093766f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01afd90 con 0x55f093757310 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b1200 con 0x55f093757c30 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01b0460 con 0x55f0937624f0 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.782+0000 7f85b964d740 10 client.0 _ll_register_callbacks cb 0x7f859c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09376ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85af400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f093766f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:57456/0 (socket says 172.21.15.59:57456) 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 -- 172.21.15.59:0/4107414983 learned_addr learned my addr 172.21.15.59:0/4107414983 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 -- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 msgr2=0x55f093766f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f093766f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 -- 172.21.15.59:0/4107414983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 msgr2=0x55f09376ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09376ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 -- 172.21.15.59:0/4107414983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f093767cf0 con 0x55f0937624f0 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85af400640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f093766f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f85b4c00640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85a800f910 tx=0x7f85a8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbfdda513e2d4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.783+0000 7f8599600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8004b20 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f85a8004cc0 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a009c320 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f85a8012020 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f85b964d740 10 client.27886 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f0937dde90 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.784+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a009c320 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f85a8029070 con 0x55f0937624f0 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 10 client.27886 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 10 client.27886 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 20 client.27886 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f0937de600 0x55f0937e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 4 client.27886 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2497c9ed", "release": true, "await": true}] 2024-09-05T12:13:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2497c9ed", "release": true, "await": true}) -- 0x55f0936c50d0 con 0x55f0937de600 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.785+0000 7f85afe00640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f0937de600 0x55f0937e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.786+0000 7f85afe00640 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f0937de600 0x55f0937e09f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2320ed8a0112ef39 server_cookie=6caf8e53c095965d in_seq=0 out_seq=0 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.786+0000 7f8599600640 10 client.27886 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f8599600640 1 -- 172.21.15.59:0/4107414983 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f0936c50d0 con 0x55f0937de600 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f8599600640 10 client.27886 handle_command_reply: tid=0 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 client.27886 shutdown 2024-09-05T12:13:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 msgr2=0x55f0937674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85a800f910 tx=0x7f85a8004ab0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 shutdown_connections 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0937624f0 0x55f0937674c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f093757c30 0x55f093766f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f0937de600 0x55f0937e09f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 --2- 172.21.15.59:0/4107414983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f093757310 0x55f09376ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 >> 172.21.15.59:0/4107414983 conn(0x55f0937538a0 msgr2=0x55f093758db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f8599600640 0 client.27886 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 shutdown_connections 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.788+0000 7f85b964d740 1 -- 172.21.15.59:0/4107414983 wait complete. 2024-09-05T12:13:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.789+0000 7f85b964d740 20 client.27886 trim_cache size 0 max 16384 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 56, 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "2497c9ed": { 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 56, 2024-09-05T12:13:18.792 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 90.0, 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 140.0, 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-09-05T12:13:18.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:18.794 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.789+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 msgr2=0x7f85940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.789+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 0x7f85940a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009fd0 tx=0x7f85a8009500 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.789+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 msgr2=0x7f85b019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.789+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85a000e9f0 tx=0x7f85a000edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 shutdown_connections 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f85940a2c00 0x7f85940a50c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85b0109120 0x7f85b019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85b01049a0 0x7f85b019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 --2- 172.21.15.59:0/213891167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85b0103ff0 0x7f85b019a620 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 >> 172.21.15.59:0/213891167 conn(0x7f85b00ffa90 msgr2=0x7f85b0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 shutdown_connections 2024-09-05T12:13:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:18.790+0000 7f85b6c00640 1 -- 172.21.15.59:0/213891167 wait complete. 2024-09-05T12:13:18.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:18 smithi059 ceph-mon[30025]: pgmap v863: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:13:19.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:18 smithi089 ceph-mon[34499]: pgmap v863: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:13:19.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:18 smithi189 ceph-mon[34772]: pgmap v863: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:13:19.528 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2497c9ed', total seconds elapsed: 23.8 2024-09-05T12:13:20.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:13:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:21.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:20 smithi089 ceph-mon[34499]: pgmap v864: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:13:21.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:20 smithi059 ceph-mon[30025]: pgmap v864: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:13:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:20 smithi189 ceph-mon[34772]: pgmap v864: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-05T12:13:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 -- 172.21.15.59:0/1676540861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818112b90 msgr2=0x7f0818114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 --2- 172.21.15.59:0/1676540861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818112b90 0x7f0818114f80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002a950 comp rx=0 tx=0).stop 2024-09-05T12:13:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 -- 172.21.15.59:0/1676540861 shutdown_connections 2024-09-05T12:13:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 --2- 172.21.15.59:0/1676540861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818112b90 0x7f0818114f80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 --2- 172.21.15.59:0/1676540861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818074eb0 0x7f0818112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 --2- 172.21.15.59:0/1676540861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074590 0x7f0818074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.627+0000 7f081da00640 1 -- 172.21.15.59:0/1676540861 >> 172.21.15.59:0/1676540861 conn(0x7f08180fd8c0 msgr2=0x7f08180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.628+0000 7f081da00640 1 -- 172.21.15.59:0/1676540861 shutdown_connections 2024-09-05T12:13:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.628+0000 7f081da00640 1 -- 172.21.15.59:0/1676540861 wait complete. 2024-09-05T12:13:21.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.628+0000 7f081da00640 1 Processor -- start 2024-09-05T12:13:21.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 -- start start 2024-09-05T12:13:21.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 0x7f081819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818112b90 0x7f08181a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08181185c0 con 0x7f0818074590 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0818118440 con 0x7f0818074eb0 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f081da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0818118740 con 0x7f0818112b90 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f0816a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f0816a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58020/0 (socket says 172.21.15.59:58020) 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f0816a00640 1 -- 172.21.15.59:0/1965831202 learned_addr learned my addr 172.21.15.59:0/1965831202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.629+0000 7f0816000640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 0x7f081819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0817400640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818112b90 0x7f08181a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 -- 172.21.15.59:0/1965831202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818112b90 msgr2=0x7f08181a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818112b90 0x7f08181a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 -- 172.21.15.59:0/1965831202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 msgr2=0x7f081819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 0x7f081819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 -- 172.21.15.59:0/1965831202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a3b40 con 0x7f0818074590 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816000640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 0x7f081819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f0816a00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9f0 tx=0x7f080800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.630+0000 7f080f400640 1 -- 172.21.15.59:0/1965831202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800cd80 con 0x7f0818074590 2024-09-05T12:13:21.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.631+0000 7f080f400640 1 -- 172.21.15.59:0/1965831202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808010910 con 0x7f0818074590 2024-09-05T12:13:21.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.631+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a3e30 con 0x7f0818074590 2024-09-05T12:13:21.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.631+0000 7f080f400640 1 -- 172.21.15.59:0/1965831202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808018990 con 0x7f0818074590 2024-09-05T12:13:21.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.631+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181ab6c0 con 0x7f0818074590 2024-09-05T12:13:21.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 Processor -- start 2024-09-05T12:13:21.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f080f400640 1 -- 172.21.15.59:0/1965831202 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0808010ac0 con 0x7f0818074590 2024-09-05T12:13:21.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 -- start start 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63cbd70 0x55c2e63ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c2e63273e0 con 0x55c2e63c14b0 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c2e62ced80 con 0x55c2e63c0b90 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f082065c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2e63233a0 con 0x55c2e63cbd70 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.634+0000 7f0816a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0816000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63cbd70 0x55c2e63ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43788/0 (socket says 172.21.15.59:43788) 2024-09-05T12:13:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 -- 172.21.15.59:0/307546599 learned_addr learned my addr 172.21.15.59:0/307546599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0816a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58024/0 (socket says 172.21.15.59:58024) 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 -- 172.21.15.59:0/307546599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63cbd70 msgr2=0x55c2e63ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63cbd70 0x55c2e63ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 -- 172.21.15.59:0/307546599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 msgr2=0x55c2e63cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 -- 172.21.15.59:0/307546599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e62edae0 con 0x55c2e63c0b90 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0816a00640 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f080f400640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 0x7f07fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.635+0000 7f0817400640 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0818076600 tx=0x7f0810005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eddf9ef2b0fdcfbb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.636+0000 7f080f400640 1 -- 172.21.15.59:0/1965831202 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f08080cf720 con 0x7f0818074590 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.636+0000 7f0816a00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 0x7f07fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.636+0000 7f0803e00640 1 -- 172.21.15.59:0/307546599 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810048070 con 0x55c2e63c0b90 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.636+0000 7f0803e00640 1 -- 172.21.15.59:0/307546599 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810003e50 con 0x55c2e63c0b90 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.636+0000 7f0803e00640 1 -- 172.21.15.59:0/307546599 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810034040 con 0x55c2e63c0b90 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 -- 172.21.15.59:0/307546599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 msgr2=0x55c2e63c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f0816a00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 0x7f07fc0a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f080809b8c0 tx=0x7f08080d5530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0818076600 tx=0x7f0810005f80 comp rx=0 tx=0).stop 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 -- 172.21.15.59:0/307546599 shutdown_connections 2024-09-05T12:13:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63cbd70 0x55c2e63ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c14b0 0x55c2e63cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 --2- 172.21.15.59:0/307546599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c0b90 0x55c2e63c0f70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 -- 172.21.15.59:0/307546599 >> 172.21.15.59:0/307546599 conn(0x55c2e63bd120 msgr2=0x55c2e63bd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 -- 172.21.15.59:0/307546599 shutdown_connections 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.637+0000 7f082065c740 1 -- 172.21.15.59:0/307546599 wait complete. 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 -- start start 2024-09-05T12:13:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 Processor -- start 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c2e63ce980 con 0x55c2e63cbd70 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c2e62e20f0 con 0x55c2e63c14b0 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2e63ceb00 con 0x55c2e63c0b90 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.638+0000 7f082065c740 10 client.0 _ll_register_callbacks cb 0x55c2e62d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0817400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43802/0 (socket says 172.21.15.59:43802) 2024-09-05T12:13:21.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0817400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45010/0 (socket says 172.21.15.59:45010) 2024-09-05T12:13:21.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 -- 172.21.15.59:0/1060319528 learned_addr learned my addr 172.21.15.59:0/1060319528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:21.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0817400640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 -- 172.21.15.59:0/1060319528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 msgr2=0x55c2e63c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 -- 172.21.15.59:0/1060319528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 msgr2=0x55c2e63c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 -- 172.21.15.59:0/1060319528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e60983c0 con 0x55c2e63c14b0 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0816000640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0804012d50 tx=0x7f080400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2af45689739622 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.639+0000 7f0800c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:13:21.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080400fe60 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f08040120d0 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810003e50 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0804025050 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f082065c740 10 client.27968 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c2e63da790 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.640+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810003e50 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0804010550 con 0x55c2e63c14b0 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 10 client.27968 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:21.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 10 client.27968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 20 client.27968 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e63dbca0 0x55c2e63de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 4 client.27968 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e6334950 con 0x55c2e63dbca0 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.641+0000 7f0817400640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e63dbca0 0x55c2e63de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.642+0000 7f0817400640 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e63dbca0 0x55c2e63de0c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b45012f519897e server_cookie=260b0f67a81514bd in_seq=0 out_seq=0 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.642+0000 7f0800c00640 10 client.27968 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f0800c00640 1 -- 172.21.15.59:0/1060319528 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2e6334950 con 0x55c2e63dbca0 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f0800c00640 10 client.27968 handle_command_reply: tid=0 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 client.27968 shutdown 2024-09-05T12:13:21.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 msgr2=0x55c2e63c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0804012d50 tx=0x7f080400f790 comp rx=0 tx=0).stop 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 shutdown_connections 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e63dbca0 0x55c2e63de0c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63c9210 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 --2- 172.21.15.59:0/1060319528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 >> 172.21.15.59:0/1060319528 conn(0x55c2e63bd120 msgr2=0x55c2e63c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f0800c00640 0 client.27968 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 shutdown_connections 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 1 -- 172.21.15.59:0/1060319528 wait complete. 2024-09-05T12:13:21.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.643+0000 7f082065c740 20 client.27968 trim_cache size 0 max 16384 2024-09-05T12:13:21.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.661+0000 7f082065c740 1 Processor -- start 2024-09-05T12:13:21.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.661+0000 7f082065c740 1 -- start start 2024-09-05T12:13:21.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b3520 con 0x55c2e63cbd70 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b5da0 con 0x55c2e63c14b0 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f082065c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b3b90 con 0x55c2e63c0b90 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0817400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0816000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0817400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0816000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43806/0 (socket says 172.21.15.59:43806) 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0816000640 1 -- 172.21.15.59:0/3782269250 learned_addr learned my addr 172.21.15.59:0/3782269250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.662+0000 7f0817400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45016/0 (socket says 172.21.15.59:45016) 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 -- 172.21.15.59:0/3782269250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 msgr2=0x55c2e63e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 -- 172.21.15.59:0/3782269250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 msgr2=0x55c2e63c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 -- 172.21.15.59:0/3782269250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e63c9ca0 con 0x55c2e63c14b0 2024-09-05T12:13:21.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0817400640 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0816000640 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0804009710 tx=0x7f0804011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2924b0daab1b9c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0803400640 1 -- 172.21.15.59:0/3782269250 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080400fe60 con 0x55c2e63c14b0 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0803400640 1 -- 172.21.15.59:0/3782269250 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08040055b0 con 0x55c2e63c14b0 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.663+0000 7f0803400640 1 -- 172.21.15.59:0/3782269250 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804010430 con 0x55c2e63c14b0 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- 172.21.15.59:0/3782269250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 msgr2=0x55c2e63e3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0804009710 tx=0x7f0804011f70 comp rx=0 tx=0).stop 2024-09-05T12:13:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- 172.21.15.59:0/3782269250 shutdown_connections 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63cbd70 0x55c2e63c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63c14b0 0x55c2e63e3ed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 --2- 172.21.15.59:0/3782269250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c0b90 0x55c2e63e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- 172.21.15.59:0/3782269250 >> 172.21.15.59:0/3782269250 conn(0x55c2e63bd120 msgr2=0x55c2e63c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- 172.21.15.59:0/3782269250 shutdown_connections 2024-09-05T12:13:21.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- 172.21.15.59:0/3782269250 wait complete. 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 -- start start 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.664+0000 7f082065c740 1 Processor -- start 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 0x55c2e63c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63cbd70 0x55c2e63c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b4260 con 0x55c2e63c0b90 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b56d0 con 0x55c2e63cbd70 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08181b4930 con 0x55c2e63c14b0 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f082065c740 10 client.0 _ll_register_callbacks cb 0x7f0804012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:21.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f0817400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f0817400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58054/0 (socket says 172.21.15.59:58054) 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f0817400640 1 -- 172.21.15.59:0/541585479 learned_addr learned my addr 172.21.15.59:0/541585479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.665+0000 7f0816000640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 0x55c2e63c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63cbd70 0x55c2e63c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 -- 172.21.15.59:0/541585479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 msgr2=0x55c2e63c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 0x55c2e63c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 -- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63cbd70 msgr2=0x55c2e63c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63cbd70 0x55c2e63c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 -- 172.21.15.59:0/541585479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e63c6530 con 0x55c2e63c0b90 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0816000640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 0x55c2e63c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0817400640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f081002af40 tx=0x7f0810055040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9baf0f0b957724f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0800c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810048040 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08040055b0 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.666+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0810003b80 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0810003d80 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 10 client.17976 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c2e6446870 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08040055b0 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f081005b680 con 0x55c2e63c0b90 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 10 client.17976 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:21.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 20 client.17976 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e6446b50 0x55c2e6448f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 4 client.17976 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.667+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2e632ee60 con 0x55c2e6446b50 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.668+0000 7f0816000640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e6446b50 0x55c2e6448f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.668+0000 7f0816000640 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e6446b50 0x55c2e6448f40 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936c51328adeb2dc server_cookie=3e2ca850d30add7d in_seq=0 out_seq=0 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.668+0000 7f0800c00640 10 client.17976 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.668+0000 7f0800c00640 1 -- 172.21.15.59:0/541585479 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2e632ee60 con 0x55c2e6446b50 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.668+0000 7f0800c00640 10 client.17976 handle_command_reply: tid=0 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 client.17976 shutdown 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 msgr2=0x55c2e63d43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f081002af40 tx=0x7f0810055040 comp rx=0 tx=0).stop 2024-09-05T12:13:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 shutdown_connections 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2e63cbd70 0x55c2e63c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2e63c14b0 0x55c2e63c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55c2e6446b50 0x55c2e6448f40 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 --2- 172.21.15.59:0/541585479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2e63c0b90 0x55c2e63d43c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 >> 172.21.15.59:0/541585479 conn(0x55c2e63bd120 msgr2=0x55c2e63ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f0800c00640 0 client.17976 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 shutdown_connections 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 1 -- 172.21.15.59:0/541585479 wait complete. 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.669+0000 7f082065c740 20 client.17976 trim_cache size 0 max 16384 2024-09-05T12:13:21.675 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 msgr2=0x7f07fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 0x7f07fc0a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f080809b8c0 tx=0x7f08080d5530 comp rx=0 tx=0).stop 2024-09-05T12:13:21.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 msgr2=0x7f081819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9f0 tx=0x7f080800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 shutdown_connections 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f07fc0a2c00 0x7f07fc0a50c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0818112b90 0x7f08181a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0818074eb0 0x7f081819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 --2- 172.21.15.59:0/1965831202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0818074590 0x7f081819eb10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 >> 172.21.15.59:0/1965831202 conn(0x7f08180fd8c0 msgr2=0x7f08180ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 shutdown_connections 2024-09-05T12:13:21.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:21.670+0000 7f081da00640 1 -- 172.21.15.59:0/1965831202 wait complete. 2024-09-05T12:13:22.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:13:22.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:22.844 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:23.146 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:22 smithi059 ceph-mon[30025]: pgmap v865: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:13:23.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:22 smithi089 ceph-mon[34499]: pgmap v865: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:13:23.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:22 smithi189 ceph-mon[34772]: pgmap v865: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 -- 172.21.15.59:0/1487330038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 msgr2=0x7f2c00101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 --2- 172.21.15.59:0/1487330038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c00101e20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009930 tx=0x7f2be802a420 comp rx=0 tx=0).stop 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 -- 172.21.15.59:0/1487330038 shutdown_connections 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 --2- 172.21.15.59:0/1487330038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 0x7f2c0010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 --2- 172.21.15.59:0/1487330038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c00101e20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 --2- 172.21.15.59:0/1487330038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 0x7f2c00107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 -- 172.21.15.59:0/1487330038 >> 172.21.15.59:0/1487330038 conn(0x7f2c000fd740 msgr2=0x7f2c000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 -- 172.21.15.59:0/1487330038 shutdown_connections 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.417+0000 7f2c04a00640 1 -- 172.21.15.59:0/1487330038 wait complete. 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.418+0000 7f2c04a00640 1 Processor -- start 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 -- start start 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 0x7f2c0019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 0x7f2c0019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00111e10 con 0x7f2c001019c0 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00111c90 con 0x7f2c001079c0 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2c04a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00111f90 con 0x7f2c00102360 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.419+0000 7f2bff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58062/0 (socket says 172.21.15.59:58062) 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 -- 172.21.15.59:0/577229103 learned_addr learned my addr 172.21.15.59:0/577229103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:23.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bf6a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 0x7f2c0019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bffe00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 0x7f2c0019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 -- 172.21.15.59:0/577229103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 msgr2=0x7f2c0019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 0x7f2c0019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 -- 172.21.15.59:0/577229103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 msgr2=0x7f2c0019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 0x7f2c0019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 -- 172.21.15.59:0/577229103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0019f480 con 0x7f2c001019c0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.420+0000 7f2bff400640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000ebc0 tx=0x7f2bf000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.421+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0010630 con 0x7f2c001019c0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.421+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0010cb0 con 0x7f2c001019c0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.421+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0019f770 con 0x7f2c001019c0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.421+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0018bd0 con 0x7f2c001019c0 2024-09-05T12:13:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.421+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a7050 con 0x7f2c001019c0 2024-09-05T12:13:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.422+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c00101e20 con 0x7f2c001019c0 2024-09-05T12:13:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.424+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2bf00107d0 con 0x7f2c001019c0 2024-09-05T12:13:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.425+0000 7f2bfcc00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 0x7f2be40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.425+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2bf00cfa10 con 0x7f2c001019c0 2024-09-05T12:13:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.425+0000 7f2bf6a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 0x7f2be40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.426+0000 7f2bf6a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 0x7f2be40a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be8000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:23.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.429+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bf009beb0 con 0x7f2c001019c0 2024-09-05T12:13:23.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.653+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c001a73b0 con 0x7f2c001019c0 2024-09-05T12:13:23.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.654+0000 7f2bfcc00640 1 -- 172.21.15.59:0/577229103 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f2bf00a01b0 con 0x7f2c001019c0 2024-09-05T12:13:23.655 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:23.656 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:23.656 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 msgr2=0x7f2be40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 0x7f2be40a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be8000bc0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 msgr2=0x7f2c0019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000ebc0 tx=0x7f2bf000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 shutdown_connections 2024-09-05T12:13:23.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f2be40a2b80 0x7f2be40a5040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c001079c0 0x7f2c0019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c00102360 0x7f2c0019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 --2- 172.21.15.59:0/577229103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c001019c0 0x7f2c0019a4d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.656+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 >> 172.21.15.59:0/577229103 conn(0x7f2c000fd740 msgr2=0x7f2c0010cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.657+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 shutdown_connections 2024-09-05T12:13:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:23.657+0000 7f2c04a00640 1 -- 172.21.15.59:0/577229103 wait complete. 2024-09-05T12:13:23.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:23 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/577229103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:24.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:23 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/577229103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:24.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:23 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/577229103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:24.464 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:13:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:25.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:24 smithi089 ceph-mon[34499]: pgmap v866: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-09-05T12:13:25.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:24 smithi059 ceph-mon[30025]: pgmap v866: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-09-05T12:13:25.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:24 smithi189 ceph-mon[34772]: pgmap v866: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-09-05T12:13:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 -- 172.21.15.59:0/158743136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7681049a0 msgr2=0x7fa7681089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 --2- 172.21.15.59:0/158743136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7681049a0 0x7fa7681089f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa750009900 tx=0x7fa75002a410 comp rx=0 tx=0).stop 2024-09-05T12:13:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 -- 172.21.15.59:0/158743136 shutdown_connections 2024-09-05T12:13:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 --2- 172.21.15.59:0/158743136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa768109120 0x7fa768110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 --2- 172.21.15.59:0/158743136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7681049a0 0x7fa7681089f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 --2- 172.21.15.59:0/158743136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768103ff0 0x7fa7681043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 -- 172.21.15.59:0/158743136 >> 172.21.15.59:0/158743136 conn(0x7fa7680ffa90 msgr2=0x7fa768101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 -- 172.21.15.59:0/158743136 shutdown_connections 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.469+0000 7fa76fc00640 1 -- 172.21.15.59:0/158743136 wait complete. 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 Processor -- start 2024-09-05T12:13:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 -- start start 2024-09-05T12:13:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7681049a0 0x7fa76819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 0x7fa76819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa768114140 con 0x7fa768103ff0 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa768113fc0 con 0x7fa768109120 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.470+0000 7fa76fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681142c0 con 0x7fa7681049a0 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58080/0 (socket says 172.21.15.59:58080) 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76d200640 1 -- 172.21.15.59:0/1338058588 learned_addr learned my addr 172.21.15.59:0/1338058588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 0x7fa76819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa75fe00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7681049a0 0x7fa76819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 -- 172.21.15.59:0/1338058588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7681049a0 msgr2=0x7fa76819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7681049a0 0x7fa76819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 -- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 msgr2=0x7fa76819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76dc00640 1 -- 172.21.15.59:0/1338058588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76819f650 con 0x7fa768109120 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.471+0000 7fa76d200640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa76dc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 0x7fa76819ef70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa760009fd0 tx=0x7fa76000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa766000640 1 -- 172.21.15.59:0/1338058588 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76000ce20 con 0x7fa768109120 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76819f940 con 0x7fa768109120 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa766000640 1 -- 172.21.15.59:0/1338058588 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa760005020 con 0x7fa768109120 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa766000640 1 -- 172.21.15.59:0/1338058588 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760010720 con 0x7fa768109120 2024-09-05T12:13:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.472+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7681a7220 con 0x7fa768109120 2024-09-05T12:13:25.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.475+0000 7fa7726a5740 1 Processor -- start 2024-09-05T12:13:25.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.475+0000 7fa766000640 1 -- 172.21.15.59:0/1338058588 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa760005540 con 0x7fa768109120 2024-09-05T12:13:25.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.475+0000 7fa7726a5740 1 -- start start 2024-09-05T12:13:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4145c70 0x559aa4146050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4153240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559aa40ac3e0 con 0x559aa4150e50 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559aa4053d80 con 0x559aa4146590 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa7726a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559aa40a83a0 con 0x559aa4145c70 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa76d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa76d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43860/0 (socket says 172.21.15.59:43860) 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa76d200640 1 -- 172.21.15.59:0/4280632098 learned_addr learned my addr 172.21.15.59:0/4280632098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.476+0000 7fa75fe00640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4145c70 0x559aa4146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa75fe00640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 -- 172.21.15.59:0/4280632098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4145c70 msgr2=0x559aa4146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4145c70 0x559aa4146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 -- 172.21.15.59:0/4280632098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 msgr2=0x559aa4153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 -- 172.21.15.59:0/4280632098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa4072ae0 con 0x559aa4146590 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa75fe00640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa76d200640 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa758011650 tx=0x7fa758009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c449b0127619ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa766000640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 0x7fa74c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa766000640 1 -- 172.21.15.59:0/1338058588 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7600cfa40 con 0x7fa768109120 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa75fe00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 0x7fa74c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.477+0000 7fa75e000640 1 -- 172.21.15.59:0/4280632098 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75801e070 con 0x559aa4146590 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa75e000640 1 -- 172.21.15.59:0/4280632098 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa75800fba0 con 0x559aa4146590 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa75e000640 1 -- 172.21.15.59:0/4280632098 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758019d80 con 0x559aa4146590 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 -- 172.21.15.59:0/4280632098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 msgr2=0x559aa4150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa758011650 tx=0x7fa758009f90 comp rx=0 tx=0).stop 2024-09-05T12:13:25.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa75fe00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 0x7fa74c0a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa750009230 tx=0x7fa7500058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 -- 172.21.15.59:0/4280632098 shutdown_connections 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4146590 0x559aa4150910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 --2- 172.21.15.59:0/4280632098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4145c70 0x559aa4146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 -- 172.21.15.59:0/4280632098 >> 172.21.15.59:0/4280632098 conn(0x559aa4142200 msgr2=0x559aa41425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 -- 172.21.15.59:0/4280632098 shutdown_connections 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.478+0000 7fa7726a5740 1 -- 172.21.15.59:0/4280632098 wait complete. 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 -- start start 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 Processor -- start 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa414e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559aa4153a20 con 0x559aa4146590 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559aa40670f0 con 0x559aa4145c70 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.479+0000 7fa7726a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559aa4153ba0 con 0x559aa4150e50 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa7726a5740 10 client.0 _ll_register_callbacks cb 0x559aa404ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58094/0 (socket says 172.21.15.59:58094) 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa75fe00640 1 -- 172.21.15.59:0/2879507322 learned_addr learned my addr 172.21.15.59:0/2879507322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76d200640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 -- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 msgr2=0x559aa414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa414dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 -- 172.21.15.59:0/2879507322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 msgr2=0x559aa414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 -- 172.21.15.59:0/2879507322 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa3e1d3c0 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa76dc00640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa414e810 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa76009bb30 tx=0x7fa76009c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4822886205c6e2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.480+0000 7fa756a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a4610 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa75800fba0 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7600a47b0 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa76007f050 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa7726a5740 10 client.27922 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559aa41682e0 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa75800fba0 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa76009ae80 con 0x559aa4150e50 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.481+0000 7fa7726a5740 10 client.27922 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa7726a5740 10 client.27922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa7726a5740 20 client.27922 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:25.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41696f0 0x559aa416bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa7726a5740 4 client.27922 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:25.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aa40b9950 con 0x559aa41696f0 2024-09-05T12:13:25.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.482+0000 7fa76d200640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41696f0 0x559aa416bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.483+0000 7fa76d200640 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41696f0 0x559aa416bb60 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267655222bc5ac94 server_cookie=368a7b8ca141575b in_seq=0 out_seq=0 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.483+0000 7fa756a00640 10 client.27922 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa756a00640 1 -- 172.21.15.59:0/2879507322 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559aa40b9950 con 0x559aa41696f0 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa756a00640 10 client.27922 handle_command_reply: tid=0 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 client.27922 shutdown 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 msgr2=0x559aa414e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa414e810 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa76009bb30 tx=0x7fa76009c670 comp rx=0 tx=0).stop 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 shutdown_connections 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa414e810 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41696f0 0x559aa416bb60 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 --2- 172.21.15.59:0/2879507322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 >> 172.21.15.59:0/2879507322 conn(0x559aa4142200 msgr2=0x559aa4141380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa756a00640 0 client.27922 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 shutdown_connections 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 1 -- 172.21.15.59:0/2879507322 wait complete. 2024-09-05T12:13:25.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.484+0000 7fa7726a5740 20 client.27922 trim_cache size 0 max 16384 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 Processor -- start 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 -- start start 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa41486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4146590 0x559aa414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4171430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681aad60 con 0x559aa4150e50 2024-09-05T12:13:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681ad5c0 con 0x559aa4145c70 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.498+0000 7fa7726a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681ab3b0 con 0x559aa4146590 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa41486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4171430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa41486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43880/0 (socket says 172.21.15.59:43880) 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76d200640 1 -- 172.21.15.59:0/407192951 learned_addr learned my addr 172.21.15.59:0/407192951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa75fe00640 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4146590 0x559aa414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 -- 172.21.15.59:0/407192951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4146590 msgr2=0x559aa414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4146590 0x559aa414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 -- 172.21.15.59:0/407192951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 msgr2=0x559aa41486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa41486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.499+0000 7fa76dc00640 1 -- 172.21.15.59:0/407192951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa413d770 con 0x559aa4150e50 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa76dc00640 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4171430 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa7600987b0 tx=0x7fa76009a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54320da3f3a15c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa75d600640 1 -- 172.21.15.59:0/407192951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76009b790 con 0x559aa4150e50 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa75d600640 1 -- 172.21.15.59:0/407192951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7600051d0 con 0x559aa4150e50 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa75d600640 1 -- 172.21.15.59:0/407192951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600980d0 con 0x559aa4150e50 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 -- 172.21.15.59:0/407192951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 msgr2=0x559aa4171430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4171430 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa7600987b0 tx=0x7fa76009a340 comp rx=0 tx=0).stop 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 -- 172.21.15.59:0/407192951 shutdown_connections 2024-09-05T12:13:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4150e50 0x559aa4171430 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4146590 0x559aa414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.500+0000 7fa7726a5740 1 --2- 172.21.15.59:0/407192951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa41486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.501+0000 7fa7726a5740 1 -- 172.21.15.59:0/407192951 >> 172.21.15.59:0/407192951 conn(0x559aa4142200 msgr2=0x559aa414f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.501+0000 7fa7726a5740 1 -- 172.21.15.59:0/407192951 shutdown_connections 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.501+0000 7fa7726a5740 1 -- 172.21.15.59:0/407192951 wait complete. 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.501+0000 7fa7726a5740 1 -- start start 2024-09-05T12:13:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.501+0000 7fa7726a5740 1 Processor -- start 2024-09-05T12:13:25.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681aba80 con 0x559aa4146590 2024-09-05T12:13:25.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681acef0 con 0x559aa4145c70 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7681ac150 con 0x559aa4150e50 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa7726a5740 10 client.0 _ll_register_callbacks cb 0x7fa760005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa76dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58122/0 (socket says 172.21.15.59:58122) 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 -- 172.21.15.59:0/863867055 learned_addr learned my addr 172.21.15.59:0/863867055 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa76d200640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 -- 172.21.15.59:0/863867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 msgr2=0x559aa415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 -- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 msgr2=0x559aa4161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.502+0000 7fa75fe00640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa4161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa75fe00640 1 -- 172.21.15.59:0/863867055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa415ddf0 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa76dc00640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa76d200640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa4161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa75fe00640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa75002a930 tx=0x7fa7500314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcdd52fb19b065a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa756a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500037f0 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa750003990 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7600051d0 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa750003b90 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa7726a5740 10 client.17997 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.503+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559aa41d4390 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7600051d0 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa750044040 con 0x559aa4146590 2024-09-05T12:13:25.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 10 client.17997 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 20 client.17997 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41d4b10 0x559aa41d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 4 client.17997 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.504+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aa40b3e60 con 0x559aa41d4b10 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa76d200640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41d4b10 0x559aa41d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa76d200640 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41d4b10 0x559aa41d6f30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f3493260c9c25b server_cookie=91c26d3aa01ae377 in_seq=0 out_seq=0 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa756a00640 10 client.17997 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa756a00640 1 -- 172.21.15.59:0/863867055 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aa40b3e60 con 0x559aa41d4b10 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa756a00640 10 client.17997 handle_command_reply: tid=0 2024-09-05T12:13:25.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa7726a5740 1 client.17997 shutdown 2024-09-05T12:13:25.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 msgr2=0x559aa415d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.505+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa75002a930 tx=0x7fa7500314e0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 shutdown_connections 2024-09-05T12:13:25.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aa4150e50 0x559aa415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x559aa41d4b10 0x559aa41d6f30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aa4146590 0x559aa415d080 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 --2- 172.21.15.59:0/863867055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559aa4145c70 0x559aa4161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 >> 172.21.15.59:0/863867055 conn(0x559aa4142200 msgr2=0x559aa414f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa756a00640 0 client.17997 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 shutdown_connections 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 1 -- 172.21.15.59:0/863867055 wait complete. 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.506+0000 7fa7726a5740 20 client.17997 trim_cache size 0 max 16384 2024-09-05T12:13:25.511 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 msgr2=0x7fa74c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 0x7fa74c0a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa750009230 tx=0x7fa7500058d0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 msgr2=0x7fa76819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 0x7fa76819ef70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa760009fd0 tx=0x7fa76000ef10 comp rx=0 tx=0).stop 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 shutdown_connections 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa74c0a2b80 0x7fa74c0a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa768109120 0x7fa76819ef70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7681049a0 0x7fa76819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 --2- 172.21.15.59:0/1338058588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa768103ff0 0x7fa76819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:25.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 >> 172.21.15.59:0/1338058588 conn(0x7fa7680ffa90 msgr2=0x7fa768100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:25.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 shutdown_connections 2024-09-05T12:13:25.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:25.507+0000 7fa76fc00640 1 -- 172.21.15.59:0/1338058588 wait complete. 2024-09-05T12:13:27.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:26 smithi089 ceph-mon[34499]: pgmap v867: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:13:27.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:26 smithi059 ceph-mon[30025]: pgmap v867: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:13:27.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:13:27.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:13:27.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:26 smithi189 ceph-mon[34772]: pgmap v867: 129 pgs: 129 active+clean; 7.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-09-05T12:13:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 -- 172.21.15.59:0/464461660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 msgr2=0x7f4568110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 --2- 172.21.15.59:0/464461660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f4568110650 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a950 comp rx=0 tx=0).stop 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 -- 172.21.15.59:0/464461660 shutdown_connections 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 --2- 172.21.15.59:0/464461660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f4568110650 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 --2- 172.21.15.59:0/464461660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568074ee0 0x7f456810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 --2- 172.21.15.59:0/464461660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45680745c0 0x7f45680749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 -- 172.21.15.59:0/464461660 >> 172.21.15.59:0/464461660 conn(0x7f45680fd6e0 msgr2=0x7f45680ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 -- 172.21.15.59:0/464461660 shutdown_connections 2024-09-05T12:13:28.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.220+0000 7f456e600640 1 -- 172.21.15.59:0/464461660 wait complete. 2024-09-05T12:13:28.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 Processor -- start 2024-09-05T12:13:28.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 -- start start 2024-09-05T12:13:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 0x7f456819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 0x7f456819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f456819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4568113f70 con 0x7f456810e260 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4568113df0 con 0x7f4568074ee0 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.221+0000 7f456e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45681140f0 con 0x7f45680745c0 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f456819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f456d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 0x7f456819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 0x7f456819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f456d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 0x7f456819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f456819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58144/0 (socket says 172.21.15.59:58144) 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f456d200640 1 -- 172.21.15.59:0/3641030087 learned_addr learned my addr 172.21.15.59:0/3641030087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 -- 172.21.15.59:0/3641030087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 msgr2=0x7f456819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 0x7f456819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 -- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 msgr2=0x7f456819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f456819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 -- 172.21.15.59:0/3641030087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456819f3e0 con 0x7f4568074ee0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.222+0000 7f455fe00640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 0x7f456819a970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f455000e9f0 tx=0x7f455000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.223+0000 7f4566000640 1 -- 172.21.15.59:0/3641030087 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455000ccc0 con 0x7f4568074ee0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.223+0000 7f4566000640 1 -- 172.21.15.59:0/3641030087 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550010910 con 0x7f4568074ee0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.223+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456819f6d0 con 0x7f4568074ee0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.223+0000 7f4566000640 1 -- 172.21.15.59:0/3641030087 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550018960 con 0x7f4568074ee0 2024-09-05T12:13:28.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.223+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a6fb0 con 0x7f4568074ee0 2024-09-05T12:13:28.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 Processor -- start 2024-09-05T12:13:28.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f4566000640 1 -- 172.21.15.59:0/3641030087 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4550010ac0 con 0x7f4568074ee0 2024-09-05T12:13:28.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 -- start start 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 0x5638b6924950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6927280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638b6880290 con 0x5638b6924e90 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5638b6827b40 con 0x5638b6919cb0 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.226+0000 7f45727ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638b687c250 con 0x5638b691a5d0 2024-09-05T12:13:28.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43910/0 (socket says 172.21.15.59:43910) 2024-09-05T12:13:28.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 -- 172.21.15.59:0/468113977 learned_addr learned my addr 172.21.15.59:0/468113977 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:28.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456d200640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 0x5638b6924950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f455fe00640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6927280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 -- 172.21.15.59:0/468113977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 msgr2=0x5638b6924950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 0x5638b6924950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 -- 172.21.15.59:0/468113977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 msgr2=0x5638b6927280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6927280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 -- 172.21.15.59:0/468113977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b65f16a0 con 0x5638b6919cb0 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456d200640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 0x5638b6924950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f455fe00640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6927280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:28.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.227+0000 7f456dc00640 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f45600093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5dc7850e366583 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f455e000640 1 -- 172.21.15.59:0/468113977 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560048070 con 0x5638b6919cb0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f455e000640 1 -- 172.21.15.59:0/468113977 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456002f940 con 0x5638b6919cb0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f455e000640 1 -- 172.21.15.59:0/468113977 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560034040 con 0x5638b6919cb0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f4566000640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 0x7f454c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f4566000640 1 -- 172.21.15.59:0/3641030087 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45500cf760 con 0x7f4568074ee0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f45727ce740 1 -- 172.21.15.59:0/468113977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 msgr2=0x5638b691a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f45727ce740 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f45600093e0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.228+0000 7f456d200640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 0x7f454c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 -- 172.21.15.59:0/468113977 shutdown_connections 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6927280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b691a5d0 0x5638b6924950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 --2- 172.21.15.59:0/468113977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b6919cb0 0x5638b691a090 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 -- 172.21.15.59:0/468113977 >> 172.21.15.59:0/468113977 conn(0x5638b6916240 msgr2=0x5638b6916620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 -- 172.21.15.59:0/468113977 shutdown_connections 2024-09-05T12:13:28.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 -- 172.21.15.59:0/468113977 wait complete. 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f456d200640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 0x7f454c0a50c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4558009cf0 tx=0x7f4558005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.229+0000 7f45727ce740 1 -- start start 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 Processor -- start 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6924780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6921a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638b6927aa0 con 0x5638b6924e90 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5638b683afa0 con 0x5638b691a5d0 2024-09-05T12:13:28.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638b6927c20 con 0x5638b6919cb0 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f45727ce740 10 client.0 _ll_register_callbacks cb 0x5638b658eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.230+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58174/0 (socket says 172.21.15.59:58174) 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 -- 172.21.15.59:0/1844753210 learned_addr learned my addr 172.21.15.59:0/1844753210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456d200640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6924780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f455fe00640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6921a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 -- 172.21.15.59:0/1844753210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 msgr2=0x5638b6924780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6924780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 -- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 msgr2=0x5638b6921a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6921a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 -- 172.21.15.59:0/1844753210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b6922750 con 0x5638b6924e90 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f455fe00640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6921a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:28.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f456dc00640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4560005e30 tx=0x7f4560003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713be3e460ef89f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.231+0000 7f4556a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560048030 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456002f940 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4560040040 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4560034050 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f45727ce740 10 client.18009 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638b69339c0 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456002f940 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.232+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f456003a550 con 0x5638b6924e90 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 10 client.18009 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 20 client.18009 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:28.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b6934d10 0x5638b6937100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 4 client.18009 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:28.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b688d800 con 0x5638b6934d10 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.233+0000 7f456d200640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b6934d10 0x5638b6937100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.234+0000 7f456d200640 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b6934d10 0x5638b6937100 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca725615075ea8 server_cookie=34a93c4852c53fc6 in_seq=0 out_seq=0 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.234+0000 7f4556a00640 10 client.18009 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.234+0000 7f4556a00640 1 -- 172.21.15.59:0/1844753210 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638b688d800 con 0x5638b6934d10 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.234+0000 7f4556a00640 10 client.18009 handle_command_reply: tid=0 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 client.18009 shutdown 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 msgr2=0x5638b6921f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4560005e30 tx=0x7f4560003970 comp rx=0 tx=0).stop 2024-09-05T12:13:28.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 shutdown_connections 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b6934d10 0x5638b6937100 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6921f70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6921a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 --2- 172.21.15.59:0/1844753210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6924780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f4556a00640 0 client.18009 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 >> 172.21.15.59:0/1844753210 conn(0x5638b6916240 msgr2=0x5638b691bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 shutdown_connections 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 1 -- 172.21.15.59:0/1844753210 wait complete. 2024-09-05T12:13:28.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.235+0000 7f45727ce740 20 client.18009 trim_cache size 0 max 16384 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.253+0000 7f45727ce740 1 Processor -- start 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.253+0000 7f45727ce740 1 -- start start 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.253+0000 7f45727ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b69239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.253+0000 7f45727ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b693a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f45727ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b693a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f45727ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45681aee60 con 0x5638b6924e90 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f45727ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b16e0 con 0x5638b691a5d0 2024-09-05T12:13:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f45727ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45681af4d0 con 0x5638b6919cb0 2024-09-05T12:13:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b693a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b69239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b693a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58182/0 (socket says 172.21.15.59:58182) 2024-09-05T12:13:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456dc00640 1 -- 172.21.15.59:0/693926147 learned_addr learned my addr 172.21.15.59:0/693926147 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f455fe00640 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b693a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 -- 172.21.15.59:0/693926147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 msgr2=0x5638b693a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b693a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 -- 172.21.15.59:0/693926147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 msgr2=0x5638b693a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b693a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.254+0000 7f456d200640 1 -- 172.21.15.59:0/693926147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b6916b80 con 0x5638b6919cb0 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f456d200640 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b69239a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5638b6922ec0 tx=0x7f455801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929302a218da0ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f455d600640 1 -- 172.21.15.59:0/693926147 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455801dda0 con 0x5638b6919cb0 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f455d600640 1 -- 172.21.15.59:0/693926147 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455802b910 con 0x5638b6919cb0 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f455d600640 1 -- 172.21.15.59:0/693926147 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558029b30 con 0x5638b6919cb0 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f45727ce740 1 -- 172.21.15.59:0/693926147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 msgr2=0x5638b69239a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f45727ce740 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b69239a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5638b6922ec0 tx=0x7f455801b000 comp rx=0 tx=0).stop 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f45727ce740 1 -- 172.21.15.59:0/693926147 shutdown_connections 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.255+0000 7f45727ce740 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b693a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b693a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 --2- 172.21.15.59:0/693926147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b69239a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 -- 172.21.15.59:0/693926147 >> 172.21.15.59:0/693926147 conn(0x5638b6916240 msgr2=0x5638b6923290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 -- 172.21.15.59:0/693926147 shutdown_connections 2024-09-05T12:13:28.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 -- 172.21.15.59:0/693926147 wait complete. 2024-09-05T12:13:28.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 -- start start 2024-09-05T12:13:28.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.256+0000 7f45727ce740 1 Processor -- start 2024-09-05T12:13:28.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6998050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6998590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45681afba0 con 0x5638b6924e90 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b1010 con 0x5638b691a5d0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b0270 con 0x5638b6919cb0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f45727ce740 10 client.0 _ll_register_callbacks cb 0x7f4560046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6998050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6998050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:45150/0 (socket says 172.21.15.59:45150) 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58186/0 (socket says 172.21.15.59:58186) 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456d200640 1 -- 172.21.15.59:0/655022202 learned_addr learned my addr 172.21.15.59:0/655022202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f455fe00640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6998590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 -- 172.21.15.59:0/655022202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 msgr2=0x5638b6998050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6998050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 -- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 msgr2=0x5638b6998590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6998590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.257+0000 7f456dc00640 1 -- 172.21.15.59:0/655022202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b693a530 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f456dc00640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f456002ae70 tx=0x7f4560005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32763af1165a279 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f4556a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560048040 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455802b910 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45600039e0 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f456003a430 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f45727ce740 10 client.18015 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.258+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638b699f880 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455802b910 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4560043410 con 0x5638b6924e90 2024-09-05T12:13:28.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 10 client.18015 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 10 client.18015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 20 client.18015 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b699fd00 0x5638b69a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 4 client.18015 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638b6887d10 con 0x5638b699fd00 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.259+0000 7f456d200640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b699fd00 0x5638b69a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.260+0000 7f456d200640 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b699fd00 0x5638b69a2120 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d14024a64f58afc server_cookie=e84a7a8ec2421919 in_seq=0 out_seq=0 2024-09-05T12:13:28.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.260+0000 7f4556a00640 10 client.18015 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f4556a00640 1 -- 172.21.15.59:0/655022202 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638b6887d10 con 0x5638b699fd00 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f4556a00640 10 client.18015 handle_command_reply: tid=0 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 client.18015 shutdown 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 msgr2=0x5638b6939e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f456002ae70 tx=0x7f4560005ef0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 shutdown_connections 2024-09-05T12:13:28.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5638b699fd00 0x5638b69a2120 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638b6924e90 0x5638b6939e40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b691a5d0 0x5638b6998590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 --2- 172.21.15.59:0/655022202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638b6919cb0 0x5638b6998050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f4556a00640 0 client.18015 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 >> 172.21.15.59:0/655022202 conn(0x5638b6916240 msgr2=0x5638b6915270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.261+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 shutdown_connections 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f45727ce740 1 -- 172.21.15.59:0/655022202 wait complete. 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f45727ce740 20 client.18015 trim_cache size 0 max 16384 2024-09-05T12:13:28.266 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "0df9a08e-b018-42ec-a78e-a73591d4f2cb", 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 msgr2=0x7f454c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 0x7f454c0a50c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4558009cf0 tx=0x7f4558005f90 comp rx=0 tx=0).stop 2024-09-05T12:13:28.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 msgr2=0x7f456819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 0x7f456819a970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f455000e9f0 tx=0x7f455000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 shutdown_connections 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f454c0a2c00 0x7f454c0a50c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f456810e260 0x7f456819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4568074ee0 0x7f456819a970 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.262+0000 7f456e600640 1 --2- 172.21.15.59:0/3641030087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45680745c0 0x7f456819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.263+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 >> 172.21.15.59:0/3641030087 conn(0x7f45680fd6e0 msgr2=0x7f45680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.263+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 shutdown_connections 2024-09-05T12:13:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:28.263+0000 7f456e600640 1 -- 172.21.15.59:0/3641030087 wait complete. 2024-09-05T12:13:28.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:28 smithi059 ceph-mon[30025]: pgmap v868: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:29.040 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:13:29.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:28 smithi089 ceph-mon[34499]: pgmap v868: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:29.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:28 smithi189 ceph-mon[34772]: pgmap v868: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:29.420 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:29.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 -- 172.21.15.59:0/889929148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6288103e30 msgr2=0x7f6288104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:29.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 --2- 172.21.15.59:0/889929148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6288103e30 0x7f6288104210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6278009860 tx=0x7f627802a340 comp rx=0 tx=0).stop 2024-09-05T12:13:29.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 -- 172.21.15.59:0/889929148 shutdown_connections 2024-09-05T12:13:29.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 --2- 172.21.15.59:0/889929148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f6288110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 --2- 172.21.15.59:0/889929148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62881047e0 0x7f6288108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 --2- 172.21.15.59:0/889929148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6288103e30 0x7f6288104210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 -- 172.21.15.59:0/889929148 >> 172.21.15.59:0/889929148 conn(0x7f62880ff910 msgr2=0x7f6288101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:29.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 -- 172.21.15.59:0/889929148 shutdown_connections 2024-09-05T12:13:29.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.987+0000 7f628d800640 1 -- 172.21.15.59:0/889929148 wait complete. 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.988+0000 7f628d800640 1 Processor -- start 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.988+0000 7f628d800640 1 -- start start 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 0x7f628819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62881047e0 0x7f628819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f628819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6288113fc0 con 0x7f6288108f60 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6288113e40 con 0x7f62881047e0 2024-09-05T12:13:29.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6288114140 con 0x7f6288103e30 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f628819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f628819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58204/0 (socket says 172.21.15.59:58204) 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f6287e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 0x7f628819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f628ce00640 1 -- 172.21.15.59:0/726870481 learned_addr learned my addr 172.21.15.59:0/726870481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.989+0000 7f627f400640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62881047e0 0x7f628819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 -- 172.21.15.59:0/726870481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62881047e0 msgr2=0x7f628819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62881047e0 0x7f628819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 -- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 msgr2=0x7f628819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f628819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 -- 172.21.15.59:0/726870481 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628819f4d0 con 0x7f6288103e30 2024-09-05T12:13:29.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6287e00640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 0x7f628819a520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f627802a860 tx=0x7f6278033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:29.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6285600640 1 -- 172.21.15.59:0/726870481 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278037040 con 0x7f6288103e30 2024-09-05T12:13:29.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6285600640 1 -- 172.21.15.59:0/726870481 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6278033d50 con 0x7f6288103e30 2024-09-05T12:13:29.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628819f760 con 0x7f6288103e30 2024-09-05T12:13:29.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.990+0000 7f6285600640 1 -- 172.21.15.59:0/726870481 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278003960 con 0x7f6288103e30 2024-09-05T12:13:29.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.991+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a70a0 con 0x7f6288103e30 2024-09-05T12:13:29.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.993+0000 7f6291888740 1 Processor -- start 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 -- start start 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6285600640 1 -- 172.21.15.59:0/726870481 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6278042020 con 0x7f6288103e30 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d7ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7fff30 0x56472d802320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56472d75b3e0 con 0x56472d7f5670 2024-09-05T12:13:29.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56472d702d80 con 0x56472d7f4d50 2024-09-05T12:13:29.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6291888740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56472d7573a0 con 0x56472d7fff30 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f628ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f6287e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d7ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f628ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:43980/0 (socket says 172.21.15.59:43980) 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f628ce00640 1 -- 172.21.15.59:0/629439691 learned_addr learned my addr 172.21.15.59:0/629439691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.994+0000 7f627f400640 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7fff30 0x56472d802320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 -- 172.21.15.59:0/629439691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7fff30 msgr2=0x56472d802320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7fff30 0x56472d802320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 -- 172.21.15.59:0/629439691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 msgr2=0x56472d7ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d7ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 -- 172.21.15.59:0/629439691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472d721ae0 con 0x56472d7f4d50 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f6287e00640 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d7ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f628ce00640 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6280009fd0 tx=0x7f62800095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed44f717606438cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.995+0000 7f627d600640 1 -- 172.21.15.59:0/629439691 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628000d910 con 0x56472d7f4d50 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f627d600640 1 -- 172.21.15.59:0/629439691 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f628001c070 con 0x56472d7f4d50 2024-09-05T12:13:29.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f627d600640 1 -- 172.21.15.59:0/629439691 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280016380 con 0x56472d7f4d50 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6285600640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 0x7f626c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6285600640 1 -- 172.21.15.59:0/726870481 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f62780eea00 con 0x7f6288103e30 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6287e00640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 0x7f626c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 -- 172.21.15.59:0/629439691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 msgr2=0x56472d7f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6280009fd0 tx=0x7f62800095c0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 -- 172.21.15.59:0/629439691 shutdown_connections 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7fff30 0x56472d802320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d7ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 --2- 172.21.15.59:0/629439691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7f5130 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.996+0000 7f6291888740 1 -- 172.21.15.59:0/629439691 >> 172.21.15.59:0/629439691 conn(0x56472d7f12e0 msgr2=0x56472d7f16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.997+0000 7f6291888740 1 -- 172.21.15.59:0/629439691 shutdown_connections 2024-09-05T12:13:30.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.997+0000 7f6291888740 1 -- 172.21.15.59:0/629439691 wait complete. 2024-09-05T12:13:30.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.997+0000 7f6287e00640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 0x7f626c0a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780f4530 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:30.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.997+0000 7f6291888740 1 -- start start 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.997+0000 7f6291888740 1 Processor -- start 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d7fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56472d802ba0 con 0x56472d7fff30 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56472d7160f0 con 0x56472d7f4d50 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56472d802d20 con 0x56472d7f5670 2024-09-05T12:13:30.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f6291888740 10 client.0 _ll_register_callbacks cb 0x56472d7079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58212/0 (socket says 172.21.15.59:58212) 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f628ce00640 1 -- 172.21.15.59:0/2070912844 learned_addr learned my addr 172.21.15.59:0/2070912844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f627f400640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d7fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.998+0000 7f628ce00640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 -- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 msgr2=0x56472d7fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 -- 172.21.15.59:0/2070912844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 msgr2=0x56472d7fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 -- 172.21.15.59:0/2070912844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472d4cc3c0 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f628ce00640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f627f400640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d7fd3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62700126d0 tx=0x7f6270012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9102b35e4cc7e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f6276000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000f920 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f628001c070 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:29.999+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f627000fac0 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f627000fcc0 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 10 client.27958 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56472d80e970 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f628001c070 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f627001c420 con 0x56472d7f5670 2024-09-05T12:13:30.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 10 client.27958 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 10 client.27958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 20 client.27958 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d80fe30 0x56472d812220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 4 client.27958 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.000+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472d768950 con 0x56472d80fe30 2024-09-05T12:13:30.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.001+0000 7f628ce00640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d80fe30 0x56472d812220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.001+0000 7f628ce00640 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d80fe30 0x56472d812220 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f4edc68c7cbde9 server_cookie=80ce87896b109844 in_seq=0 out_seq=0 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.001+0000 7f6276000640 10 client.27958 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6276000640 1 -- 172.21.15.59:0/2070912844 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56472d768950 con 0x56472d80fe30 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6276000640 10 client.27958 handle_command_reply: tid=0 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 client.27958 shutdown 2024-09-05T12:13:30.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 msgr2=0x56472d7fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d7fd3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62700126d0 tx=0x7f6270012b90 comp rx=0 tx=0).stop 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 shutdown_connections 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d80fe30 0x56472d812220 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7fff30 0x56472d7fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d7fd3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 --2- 172.21.15.59:0/2070912844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7f4d50 0x56472d7fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 >> 172.21.15.59:0/2070912844 conn(0x56472d7f12e0 msgr2=0x56472d7f7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:30.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.002+0000 7f6276000640 0 client.27958 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:30.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.003+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 shutdown_connections 2024-09-05T12:13:30.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.003+0000 7f6291888740 1 -- 172.21.15.59:0/2070912844 wait complete. 2024-09-05T12:13:30.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.003+0000 7f6291888740 20 client.27958 trim_cache size 0 max 16384 2024-09-05T12:13:30.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 Processor -- start 2024-09-05T12:13:30.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 -- start start 2024-09-05T12:13:30.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f4d50 0x56472d817a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d7f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62881aef30 con 0x56472d7f5670 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62881b17b0 con 0x56472d7fff30 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f6291888740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62881af5a0 con 0x56472d7f4d50 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f627f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f627f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58214/0 (socket says 172.21.15.59:58214) 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f627f400640 1 -- 172.21.15.59:0/846909045 learned_addr learned my addr 172.21.15.59:0/846909045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.021+0000 7f628ce00640 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f4d50 0x56472d817a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f628ce00640 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d7f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 -- 172.21.15.59:0/846909045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f4d50 msgr2=0x56472d817a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f4d50 0x56472d817a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 -- 172.21.15.59:0/846909045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 msgr2=0x56472d7f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d7f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 -- 172.21.15.59:0/846909045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472d802660 con 0x56472d7f5670 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627f400640 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f627001c910 tx=0x7f62700057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cda51f74f0e19c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627cc00640 1 -- 172.21.15.59:0/846909045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270000e60 con 0x56472d7f5670 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.022+0000 7f627cc00640 1 -- 172.21.15.59:0/846909045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627000f920 con 0x56472d7f5670 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f627cc00640 1 -- 172.21.15.59:0/846909045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700055e0 con 0x56472d7f5670 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 -- 172.21.15.59:0/846909045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 msgr2=0x56472d817fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f627001c910 tx=0x7f62700057d0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 -- 172.21.15.59:0/846909045 shutdown_connections 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d7f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f5670 0x56472d817fc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 --2- 172.21.15.59:0/846909045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f4d50 0x56472d817a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 -- 172.21.15.59:0/846909045 >> 172.21.15.59:0/846909045 conn(0x56472d7f12e0 msgr2=0x56472d7fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 -- 172.21.15.59:0/846909045 shutdown_connections 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.023+0000 7f6291888740 1 -- 172.21.15.59:0/846909045 wait complete. 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 -- start start 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 Processor -- start 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d803a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d878a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62881afc70 con 0x56472d7f4d50 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62881b10e0 con 0x56472d7fff30 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62881b0340 con 0x56472d7f5670 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.024+0000 7f6291888740 10 client.0 _ll_register_callbacks cb 0x7f6270009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:30.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58220/0 (socket says 172.21.15.59:58220) 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 -- 172.21.15.59:0/2830500315 learned_addr learned my addr 172.21.15.59:0/2830500315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f627f400640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d803a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d878a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 -- 172.21.15.59:0/2830500315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 msgr2=0x56472d803a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d803a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 -- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 msgr2=0x56472d878a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d878a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 -- 172.21.15.59:0/2830500315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472d879000 con 0x56472d7f4d50 2024-09-05T12:13:30.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f628ce00640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6280017a30 tx=0x7f6270009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c66365d235f0551 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.025+0000 7f6276000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628002e040 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6280004c00 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627000f920 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6280004e00 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6291888740 10 client.18033 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56472d87a310 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627000f920 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.026+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f628000b040 con 0x56472d7f4d50 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 10 client.18033 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 20 client.18033 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:30.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d87a760 0x56472d87cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:30.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 4 client.18033 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:13:30.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56472d762e60 con 0x56472d87a760 2024-09-05T12:13:30.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f627f400640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d87a760 0x56472d87cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:30.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f627f400640 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d87a760 0x56472d87cb50 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b83ab5054ba14a5 server_cookie=f44013aee49c0a43 in_seq=0 out_seq=0 2024-09-05T12:13:30.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.027+0000 7f6276000640 10 client.18033 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:30.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6276000640 1 -- 172.21.15.59:0/2830500315 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56472d762e60 con 0x56472d87a760 2024-09-05T12:13:30.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6276000640 10 client.18033 handle_command_reply: tid=0 2024-09-05T12:13:30.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 client.18033 shutdown 2024-09-05T12:13:30.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 msgr2=0x56472d803550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6280017a30 tx=0x7f6270009e30 comp rx=0 tx=0).stop 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 shutdown_connections 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56472d7fff30 0x56472d878a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56472d7f5670 0x56472d803a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56472d87a760 0x56472d87cb50 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 --2- 172.21.15.59:0/2830500315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56472d7f4d50 0x56472d803550 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 >> 172.21.15.59:0/2830500315 conn(0x56472d7f12e0 msgr2=0x56472d7efb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6276000640 0 client.18033 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 shutdown_connections 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 1 -- 172.21.15.59:0/2830500315 wait complete. 2024-09-05T12:13:30.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.028+0000 7f6291888740 20 client.18033 trim_cache size 0 max 16384 2024-09-05T12:13:30.037 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:30.037 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:13:30.037 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:13:30.037 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:30.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 msgr2=0x7f626c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 0x7f626c0a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780f4530 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 msgr2=0x7f628819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 0x7f628819a520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f627802a860 tx=0x7f6278033b60 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 shutdown_connections 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f626c0a2b80 0x7f626c0a5040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288108f60 0x7f628819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62881047e0 0x7f628819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 --2- 172.21.15.59:0/726870481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6288103e30 0x7f628819a520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 >> 172.21.15.59:0/726870481 conn(0x7f62880ff910 msgr2=0x7f6288100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 shutdown_connections 2024-09-05T12:13:30.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:30.029+0000 7f628d800640 1 -- 172.21.15.59:0/726870481 wait complete. 2024-09-05T12:13:30.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:29 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:13:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:29 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:29 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:13:30.185 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:29 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:29 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:13:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:29 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:29 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:13:30.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:29 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:29 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:13:30.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:29 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:29 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:13:30.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:29 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:30.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:13:30.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:31.167 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:30 smithi089 ceph-mon[34499]: pgmap v869: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:31.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:30 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:13:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:30 smithi059 ceph-mon[30025]: pgmap v869: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:31.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:30 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:13:31.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:30 smithi189 ceph-mon[34772]: pgmap v869: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:31.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:30 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 -- 172.21.15.59:0/130841542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18077040 msgr2=0x7fed181124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 --2- 172.21.15.59:0/130841542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18077040 0x7fed181124f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed04009900 tx=0x7fed0402a410 comp rx=0 tx=0).stop 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 -- 172.21.15.59:0/130841542 shutdown_connections 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 --2- 172.21.15.59:0/130841542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18112a30 0x7fed18114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 --2- 172.21.15.59:0/130841542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18077040 0x7fed181124f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 --2- 172.21.15.59:0/130841542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed18076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 -- 172.21.15.59:0/130841542 >> 172.21.15.59:0/130841542 conn(0x7fed180ffa90 msgr2=0x7fed18101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 -- 172.21.15.59:0/130841542 shutdown_connections 2024-09-05T12:13:31.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.742+0000 7fed1e400640 1 -- 172.21.15.59:0/130841542 wait complete. 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 Processor -- start 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 -- start start 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 0x7fed181a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18112a30 0x7fed1819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed181185f0 con 0x7fed18112a30 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fed18118470 con 0x7fed18076720 2024-09-05T12:13:31.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed1e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed18118770 con 0x7fed18077040 2024-09-05T12:13:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed17400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.744+0000 7fed17400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44048/0 (socket says 172.21.15.59:44048) 2024-09-05T12:13:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 -- 172.21.15.59:0/1452849541 learned_addr learned my addr 172.21.15.59:0/1452849541 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17e00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18112a30 0x7fed1819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:31.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed16a00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 0x7fed181a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 -- 172.21.15.59:0/1452849541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 msgr2=0x7fed181a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 0x7fed181a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 -- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18112a30 msgr2=0x7fed1819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:31.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18112a30 0x7fed1819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 -- 172.21.15.59:0/1452849541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1819eff0 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed16a00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 0x7fed181a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.745+0000 7fed17400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed0800e9f0 tx=0x7fed0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.746+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0800cd80 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.746+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08010910 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.746+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1819f2e0 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.746+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08018990 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.746+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181ab5d0 con 0x7fed18076720 2024-09-05T12:13:31.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.747+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed18076b00 con 0x7fed18076720 2024-09-05T12:13:31.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.749+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed08010ac0 con 0x7fed18076720 2024-09-05T12:13:31.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.750+0000 7fed0fe00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 0x7fecfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.750+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fed080cfc00 con 0x7fed18076720 2024-09-05T12:13:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.750+0000 7fed16a00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 0x7fecfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.751+0000 7fed16a00640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 0x7fecfc0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed04031510 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:31.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.753+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed0809c0a0 con 0x7fed18076720 2024-09-05T12:13:31.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.974+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed181ab7d0 con 0x7fed18076720 2024-09-05T12:13:31.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.976+0000 7fed0fe00640 1 -- 172.21.15.59:0/1452849541 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fed080a03a0 con 0x7fed18076720 2024-09-05T12:13:31.978 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:31.978 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:31.978 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:31.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 msgr2=0x7fecfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 0x7fecfc0a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed04031510 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 msgr2=0x7fed181a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed0800e9f0 tx=0x7fed0800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 shutdown_connections 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fecfc0a2b80 0x7fecfc0a5040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed18112a30 0x7fed1819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed18077040 0x7fed181a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 --2- 172.21.15.59:0/1452849541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed18076720 0x7fed181a4700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 >> 172.21.15.59:0/1452849541 conn(0x7fed180ffa90 msgr2=0x7fed18101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:31.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 shutdown_connections 2024-09-05T12:13:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:31.978+0000 7fed1e400640 1 -- 172.21.15.59:0/1452849541 wait complete. 2024-09-05T12:13:32.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:13:33.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:32 smithi189 ceph-mon[34772]: pgmap v870: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:33.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:32 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1452849541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:33.163 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:32 smithi089 ceph-mon[34499]: pgmap v870: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:33.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:32 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1452849541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:33.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:32 smithi059 ceph-mon[30025]: pgmap v870: 129 pgs: 129 active+clean; 7.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.0 KiB/s wr, 5 op/s 2024-09-05T12:13:33.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:32 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1452849541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 -- 172.21.15.59:0/1676645795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c1047e0 msgr2=0x7f4b2c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 --2- 172.21.15.59:0/1676645795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c108830 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009900 tx=0x7f4b1402a410 comp rx=0 tx=0).stop 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 -- 172.21.15.59:0/1676645795 shutdown_connections 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 --2- 172.21.15.59:0/1676645795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 0x7f4b2c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 --2- 172.21.15.59:0/1676645795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c108830 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 --2- 172.21.15.59:0/1676645795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c103e30 0x7f4b2c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 -- 172.21.15.59:0/1676645795 >> 172.21.15.59:0/1676645795 conn(0x7f4b2c0ff8d0 msgr2=0x7f4b2c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 -- 172.21.15.59:0/1676645795 shutdown_connections 2024-09-05T12:13:33.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.719+0000 7f4b32400640 1 -- 172.21.15.59:0/1676645795 wait complete. 2024-09-05T12:13:33.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.720+0000 7f4b32400640 1 Processor -- start 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.720+0000 7f4b32400640 1 -- start start 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 0x7f4b2c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 0x7f4b2c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c114010 con 0x7f4b2c103e30 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c113e90 con 0x7f4b2c108f60 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b32400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c114190 con 0x7f4b2c1047e0 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b31000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 0x7f4b2c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41862/0 (socket says 172.21.15.59:41862) 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b23e00640 1 -- 172.21.15.59:0/1270257078 learned_addr learned my addr 172.21.15.59:0/1270257078 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:33.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b31a00640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 0x7f4b2c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.721+0000 7f4b23e00640 1 -- 172.21.15.59:0/1270257078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 msgr2=0x7f4b2c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b23e00640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 0x7f4b2c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b23e00640 1 -- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 msgr2=0x7f4b2c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b23e00640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 0x7f4b2c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b23e00640 1 -- 172.21.15.59:0/1270257078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c19f420 con 0x7f4b2c1047e0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b31000640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 0x7f4b2c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b23e00640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b14004770 tx=0x7f4b14003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b2a000640 1 -- 172.21.15.59:0/1270257078 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1403f070 con 0x7f4b2c1047e0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c19f6b0 con 0x7f4b2c1047e0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b2a000640 1 -- 172.21.15.59:0/1270257078 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b14031ca0 con 0x7f4b2c1047e0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.722+0000 7f4b2a000640 1 -- 172.21.15.59:0/1270257078 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14035030 con 0x7f4b2c1047e0 2024-09-05T12:13:33.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.723+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a6ff0 con 0x7f4b2c1047e0 2024-09-05T12:13:33.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 Processor -- start 2024-09-05T12:13:33.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b2a000640 1 -- 172.21.15.59:0/1270257078 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b14042070 con 0x7f4b2c1047e0 2024-09-05T12:13:33.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 -- start start 2024-09-05T12:13:33.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d11220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d1e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7c773e0 con 0x55f1a7d1c020 2024-09-05T12:13:33.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7c1ed80 con 0x55f1a7d10e40 2024-09-05T12:13:33.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.726+0000 7f4b365cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7c733a0 con 0x55f1a7d11760 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d1e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d11220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d1e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41946/0 (socket says 172.21.15.59:41946) 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b23e00640 1 -- 172.21.15.59:0/3370752219 learned_addr learned my addr 172.21.15.59:0/3370752219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31000640 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 -- 172.21.15.59:0/3370752219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 msgr2=0x55f1a7d1bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 -- 172.21.15.59:0/3370752219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 msgr2=0x55f1a7d1e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d1e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 -- 172.21.15.59:0/3370752219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a7c3dae0 con 0x55f1a7d10e40 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.727+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d11220 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009f00 tx=0x7f4b24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b134e94e4cb15f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b2a000640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 0x7f4b100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b22000640 1 -- 172.21.15.59:0/3370752219 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004d20 con 0x55f1a7d10e40 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b22000640 1 -- 172.21.15.59:0/3370752219 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24004050 con 0x55f1a7d10e40 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b22000640 1 -- 172.21.15.59:0/3370752219 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24005220 con 0x55f1a7d10e40 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b2a000640 1 -- 172.21.15.59:0/1270257078 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4b140ef9c0 con 0x7f4b2c1047e0 2024-09-05T12:13:33.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b31000640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 0x7f4b100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b365cb740 1 -- 172.21.15.59:0/3370752219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 msgr2=0x55f1a7d11220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.728+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d11220 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009f00 tx=0x7f4b24011810 comp rx=0 tx=0).stop 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 -- 172.21.15.59:0/3370752219 shutdown_connections 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d1e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3370752219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d11220 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 -- 172.21.15.59:0/3370752219 >> 172.21.15.59:0/3370752219 conn(0x55f1a7d0d3d0 msgr2=0x55f1a7d0d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b31000640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 0x7f4b100a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c012640 tx=0x7f4b1c005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 -- 172.21.15.59:0/3370752219 shutdown_connections 2024-09-05T12:13:33.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 -- 172.21.15.59:0/3370752219 wait complete. 2024-09-05T12:13:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 -- start start 2024-09-05T12:13:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.729+0000 7f4b365cb740 1 Processor -- start 2024-09-05T12:13:33.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 0x55f1a7d19490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 0x55f1a7d199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7d1ebf0 con 0x55f1a7d11760 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7c320f0 con 0x55f1a7d10e40 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1a7d1ed70 con 0x55f1a7d1c020 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b365cb740 10 client.0 _ll_register_callbacks cb 0x55f1a7c19bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:33.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44066/0 (socket says 172.21.15.59:44066) 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b31000640 1 -- 172.21.15.59:0/2502805561 learned_addr learned my addr 172.21.15.59:0/2502805561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 0x55f1a7d19490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.730+0000 7f4b31a00640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 0x55f1a7d199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 -- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 msgr2=0x55f1a7d18f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 -- 172.21.15.59:0/2502805561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 msgr2=0x55f1a7d19490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 0x55f1a7d19490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 -- 172.21.15.59:0/2502805561 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a79e83c0 con 0x55f1a7d1c020 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31000640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b23e00640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 0x55f1a7d19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b31a00640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 0x55f1a7d199d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000f80 tx=0x7f4b240044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c299db34fc334f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24029020 con 0x55f1a7d1c020 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b24004050 con 0x55f1a7d1c020 2024-09-05T12:13:33.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.731+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4b24025470 con 0x55f1a7d1c020 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b24034050 con 0x55f1a7d1c020 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 10 client.27982 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1a7d2ac80 con 0x55f1a7d1c020 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24004050 con 0x55f1a7d1c020 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b240195c0 con 0x55f1a7d1c020 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 10 client.27982 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 10 client.27982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 20 client.27982 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d2c0e0 0x55f1a7d2e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 4 client.27982 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:33.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.732+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1a7c84950 con 0x55f1a7d2c0e0 2024-09-05T12:13:33.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.733+0000 7f4b31000640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d2c0e0 0x55f1a7d2e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.733+0000 7f4b31000640 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d2c0e0 0x55f1a7d2e4d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2acddee538924c66 server_cookie=56240636d5994fdb in_seq=0 out_seq=0 2024-09-05T12:13:33.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.733+0000 7f4b1aa00640 10 client.27982 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:33.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.734+0000 7f4b1aa00640 1 -- 172.21.15.59:0/2502805561 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1a7c84950 con 0x55f1a7d2c0e0 2024-09-05T12:13:33.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.734+0000 7f4b1aa00640 10 client.27982 handle_command_reply: tid=0 2024-09-05T12:13:33.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.734+0000 7f4b365cb740 1 client.27982 shutdown 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 msgr2=0x55f1a7d199d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 0x55f1a7d199d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000f80 tx=0x7f4b240044b0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 shutdown_connections 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d1c020 0x55f1a7d199d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d2c0e0 0x55f1a7d2e4d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d11760 0x55f1a7d19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 --2- 172.21.15.59:0/2502805561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d18f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 >> 172.21.15.59:0/2502805561 conn(0x55f1a7d0d3d0 msgr2=0x55f1a7d130b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b1aa00640 0 client.27982 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 shutdown_connections 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 1 -- 172.21.15.59:0/2502805561 wait complete. 2024-09-05T12:13:33.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.735+0000 7f4b365cb740 20 client.27982 trim_cache size 0 max 16384 2024-09-05T12:13:33.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 Processor -- start 2024-09-05T12:13:33.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 -- start start 2024-09-05T12:13:33.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1aef10 con 0x55f1a7d1c020 2024-09-05T12:13:33.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b1790 con 0x55f1a7d10e40 2024-09-05T12:13:33.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b365cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1af580 con 0x55f1a7d11760 2024-09-05T12:13:33.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b31a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44074/0 (socket says 172.21.15.59:44074) 2024-09-05T12:13:33.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b31000640 1 -- 172.21.15.59:0/3816709718 learned_addr learned my addr 172.21.15.59:0/3816709718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.749+0000 7f4b23e00640 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 -- 172.21.15.59:0/3816709718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 msgr2=0x55f1a7d341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 -- 172.21.15.59:0/3816709718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 msgr2=0x55f1a7d161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 -- 172.21.15.59:0/3816709718 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a7d174a0 con 0x55f1a7d10e40 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b31000640 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f1a7d2ec00 tx=0x7f4b1c022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e6279b82cdcabb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b21600640 1 -- 172.21.15.59:0/3816709718 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00fb10 con 0x55f1a7d10e40 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b21600640 1 -- 172.21.15.59:0/3816709718 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c005500 con 0x55f1a7d10e40 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.750+0000 7f4b21600640 1 -- 172.21.15.59:0/3816709718 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c004d80 con 0x55f1a7d10e40 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- 172.21.15.59:0/3816709718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 msgr2=0x55f1a7d33c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f1a7d2ec00 tx=0x7f4b1c022040 comp rx=0 tx=0).stop 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- 172.21.15.59:0/3816709718 shutdown_connections 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d1c020 0x55f1a7d161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3816709718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d10e40 0x55f1a7d33c90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- 172.21.15.59:0/3816709718 >> 172.21.15.59:0/3816709718 conn(0x55f1a7d0d3d0 msgr2=0x55f1a7d130b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- 172.21.15.59:0/3816709718 shutdown_connections 2024-09-05T12:13:33.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- 172.21.15.59:0/3816709718 wait complete. 2024-09-05T12:13:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.751+0000 7f4b365cb740 1 -- start start 2024-09-05T12:13:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 Processor -- start 2024-09-05T12:13:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d10e40 0x55f1a7d1f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1afc50 con 0x55f1a7d10e40 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b10c0 con 0x55f1a7d1c020 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b0320 con 0x55f1a7d11760 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.752+0000 7f4b365cb740 10 client.0 _ll_register_callbacks cb 0x7f4b140f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d10e40 0x55f1a7d1f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44076/0 (socket says 172.21.15.59:44076) 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 -- 172.21.15.59:0/3994801874 learned_addr learned my addr 172.21.15.59:0/3994801874 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 -- 172.21.15.59:0/3994801874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 msgr2=0x55f1a7d1fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 -- 172.21.15.59:0/3994801874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d10e40 msgr2=0x55f1a7d1f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d10e40 0x55f1a7d1f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 -- 172.21.15.59:0/3994801874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a7d8f260 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b31a00640 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4b24023420 tx=0x7f4b24009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e6f6c93d0d7b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.753+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24029040 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f4b24019430 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c005500 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b24019630 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 10 client.28058 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1a7d96610 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c005500 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b240257a0 con 0x55f1a7d1c020 2024-09-05T12:13:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 10 client.28058 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 20 client.28058 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d96d50 0x55f1a7d99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 4 client.28058 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.754+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1a7c7ee60 con 0x55f1a7d96d50 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b31000640 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d96d50 0x55f1a7d99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b31000640 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d96d50 0x55f1a7d99170 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d69ea8d13abb30 server_cookie=4c8d32cce1d0eb61 in_seq=0 out_seq=0 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b1aa00640 10 client.28058 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b1aa00640 1 -- 172.21.15.59:0/3994801874 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1a7c7ee60 con 0x55f1a7d96d50 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b1aa00640 10 client.28058 handle_command_reply: tid=0 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.755+0000 7f4b365cb740 1 client.28058 shutdown 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 msgr2=0x55f1a7d8ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4b24023420 tx=0x7f4b24009e00 comp rx=0 tx=0).stop 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 shutdown_connections 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1a7d1c020 0x55f1a7d8ec20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1a7d11760 0x55f1a7d1fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1a7d96d50 0x55f1a7d99170 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 --2- 172.21.15.59:0/3994801874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1a7d10e40 0x55f1a7d1f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 >> 172.21.15.59:0/3994801874 conn(0x55f1a7d0d3d0 msgr2=0x55f1a7d17820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b1aa00640 0 client.28058 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 shutdown_connections 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 1 -- 172.21.15.59:0/3994801874 wait complete. 2024-09-05T12:13:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.756+0000 7f4b365cb740 20 client.28058 trim_cache size 0 max 16384 2024-09-05T12:13:33.764 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:13:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 msgr2=0x7f4b100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 0x7f4b100a50c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c012640 tx=0x7f4b1c005f90 comp rx=0 tx=0).stop 2024-09-05T12:13:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 msgr2=0x7f4b2c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b14004770 tx=0x7f4b14003c10 comp rx=0 tx=0).stop 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 shutdown_connections 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f4b100a2c00 0x7f4b100a50c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b2c108f60 0x7f4b2c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c1047e0 0x7f4b2c19a9b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 --2- 172.21.15.59:0/1270257078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b2c103e30 0x7f4b2c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 >> 172.21.15.59:0/1270257078 conn(0x7f4b2c0ff8d0 msgr2=0x7f4b2c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 shutdown_connections 2024-09-05T12:13:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:33.757+0000 7f4b32400640 1 -- 172.21.15.59:0/1270257078 wait complete. 2024-09-05T12:13:33.912 INFO:tasks.workunit.client.0.smithi059.stdout:561/600: open fd = 3 2024-09-05T12:13:33.912 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.912 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:562/600: open fd = 3 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:563/600: open fd = 3 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:564/600: open fd = 3 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.913 INFO:tasks.workunit.client.0.smithi059.stdout:565/600: open fd = 3 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:566/600: open fd = 3 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:567/600: open fd = 3 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:568/600: open fd = 3 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.914 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:569/600: open fd = 3 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:570/600: open fd = 3 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:571/600: open fd = 3 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:572/600: open fd = 3 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.915 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:573/600: open fd = 3 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:574/600: open fd = 3 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:575/600: open fd = 3 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:576/600: open fd = 3 2024-09-05T12:13:33.916 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:577/600: open fd = 3 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:578/600: open fd = 3 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:579/600: open fd = 3 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.917 INFO:tasks.workunit.client.0.smithi059.stdout:580/600: open fd = 3 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:581/600: open fd = 3 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:582/600: open fd = 3 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:583/600: open fd = 3 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.918 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:584/600: open fd = 3 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:585/600: open fd = 3 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:586/600: open fd = 3 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.919 INFO:tasks.workunit.client.0.smithi059.stdout:587/600: open fd = 3 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:588/600: open fd = 3 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:589/600: open fd = 3 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:590/600: open fd = 3 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.920 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:591/600: open fd = 3 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:592/600: open fd = 3 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:593/600: open fd = 3 2024-09-05T12:13:33.921 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:594/600: open fd = 3 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:595/600: open fd = 3 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:596/600: open fd = 3 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.922 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:597/600: open fd = 3 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:598/600: open fd = 3 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:599/600: open fd = 3 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:600/600: open fd = 3 2024-09-05T12:13:33.923 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.924 INFO:tasks.workunit.client.0.smithi059.stdout:write ret = 32 2024-09-05T12:13:33.924 INFO:tasks.workunit.client.0.smithi059.stdout:pread ret = 64 2024-09-05T12:13:33.924 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo OK 2024-09-05T12:13:33.925 INFO:tasks.workunit.client.0.smithi059.stdout:OK 2024-09-05T12:13:33.925 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-05T12:13:33.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-05T12:13:33.998 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-05T12:13:33.998 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-05T12:13:35.098 DEBUG:teuthology.parallel:result is None 2024-09-05T12:13:35.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T12:13:35.147 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:34 smithi059 ceph-mon[30025]: pgmap v871: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:13:35.159 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-05T12:13:35.159 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-05T12:13:35.170 INFO:tasks.quiescer:joining Quiescers 2024-09-05T12:13:35.170 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-05T12:13:35.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:35.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:34 smithi089 ceph-mon[34499]: pgmap v871: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:13:35.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:34 smithi189 ceph-mon[34772]: pgmap v871: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-09-05T12:13:35.523 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-05T12:13:35.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-05T12:13:35.578 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:36.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 -- 172.21.15.59:0/4009961705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c103ff0 msgr2=0x7fd04c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 --2- 172.21.15.59:0/4009961705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c103ff0 0x7fd04c1043d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd040009860 tx=0x7fd04002a340 comp rx=0 tx=0).stop 2024-09-05T12:13:36.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 -- 172.21.15.59:0/4009961705 shutdown_connections 2024-09-05T12:13:36.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 --2- 172.21.15.59:0/4009961705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c109120 0x7fd04c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 --2- 172.21.15.59:0/4009961705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c1049a0 0x7fd04c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 --2- 172.21.15.59:0/4009961705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c103ff0 0x7fd04c1043d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.128+0000 7fd053400640 1 -- 172.21.15.59:0/4009961705 >> 172.21.15.59:0/4009961705 conn(0x7fd04c0ffa90 msgr2=0x7fd04c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.129+0000 7fd053400640 1 -- 172.21.15.59:0/4009961705 shutdown_connections 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.129+0000 7fd053400640 1 -- 172.21.15.59:0/4009961705 wait complete. 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 Processor -- start 2024-09-05T12:13:36.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 -- start start 2024-09-05T12:13:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c103ff0 0x7fd04c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 0x7fd04c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c114120 con 0x7fd04c103ff0 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c113fa0 con 0x7fd04c109120 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.130+0000 7fd053400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1142a0 con 0x7fd04c1049a0 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd051400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 0x7fd04c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:41924/0 (socket says 172.21.15.59:41924) 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 -- 172.21.15.59:0/2306072095 learned_addr learned my addr 172.21.15.59:0/2306072095 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd051400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 0x7fd04c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44100/0 (socket says 172.21.15.59:44100) 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd050a00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c103ff0 0x7fd04c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 -- 172.21.15.59:0/2306072095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 msgr2=0x7fd04c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 0x7fd04c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 -- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c103ff0 msgr2=0x7fd04c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c103ff0 0x7fd04c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.131+0000 7fd04be00640 1 -- 172.21.15.59:0/2306072095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c19f5b0 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd04be00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd03800e9f0 tx=0x7fd03800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800cd80 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c19f8a0 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038010910 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0380188f0 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.132+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a7180 con 0x7fd04c1049a0 2024-09-05T12:13:36.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.134+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c1043d0 con 0x7fd04c1049a0 2024-09-05T12:13:36.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.135+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd038010ac0 con 0x7fd04c1049a0 2024-09-05T12:13:36.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.137+0000 7fd049600640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.137+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd0380cf7f0 con 0x7fd04c1049a0 2024-09-05T12:13:36.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.137+0000 7fd050a00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.138+0000 7fd050a00640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 0x7fd0340a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd04002a930 tx=0x7fd0400058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.140+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd03809bc90 con 0x7fd04c1049a0 2024-09-05T12:13:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:36.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.376+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c10c360 con 0x7fd04c1049a0 2024-09-05T12:13:36.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.377+0000 7fd049600640 1 -- 172.21.15.59:0/2306072095 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7fd03809ff90 con 0x7fd04c1049a0 2024-09-05T12:13:36.379 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:36.379 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:36.379 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:36.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 msgr2=0x7fd0340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 0x7fd0340a50c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd04002a930 tx=0x7fd0400058d0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 msgr2=0x7fd04c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd03800e9f0 tx=0x7fd03800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 shutdown_connections 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd0340a2c00 0x7fd0340a50c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd04c109120 0x7fd04c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c1049a0 0x7fd04c19ab40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 --2- 172.21.15.59:0/2306072095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd04c103ff0 0x7fd04c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 >> 172.21.15.59:0/2306072095 conn(0x7fd04c0ffa90 msgr2=0x7fd04c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 shutdown_connections 2024-09-05T12:13:36.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.380+0000 7fd053400640 1 -- 172.21.15.59:0/2306072095 wait complete. 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 -- 172.21.15.59:0/17604699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 msgr2=0x7fdba0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 --2- 172.21.15.59:0/17604699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba0101e20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009930 tx=0x7fdb8802a420 comp rx=0 tx=0).stop 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 -- 172.21.15.59:0/17604699 shutdown_connections 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 --2- 172.21.15.59:0/17604699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 0x7fdba010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 --2- 172.21.15.59:0/17604699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba0101e20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 --2- 172.21.15.59:0/17604699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 0x7fdba0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 -- 172.21.15.59:0/17604699 >> 172.21.15.59:0/17604699 conn(0x7fdba00fd740 msgr2=0x7fdba00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 -- 172.21.15.59:0/17604699 shutdown_connections 2024-09-05T12:13:36.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.921+0000 7fdb9f200640 1 -- 172.21.15.59:0/17604699 wait complete. 2024-09-05T12:13:36.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 Processor -- start 2024-09-05T12:13:36.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 -- start start 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 0x7fdba019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 0x7fdba019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111e10 con 0x7fdba01019c0 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111c90 con 0x7fdba0102360 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.923+0000 7fdb9f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111f90 con 0x7fdba01079c0 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42036/0 (socket says 172.21.15.59:42036) 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 -- 172.21.15.59:0/3086304210 learned_addr learned my addr 172.21.15.59:0/3086304210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb95400640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 0x7fdba019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9e800640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 0x7fdba019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 -- 172.21.15.59:0/3086304210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 msgr2=0x7fdba019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 0x7fdba019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 -- 172.21.15.59:0/3086304210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 msgr2=0x7fdba019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 0x7fdba019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 -- 172.21.15.59:0/3086304210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba019f4a0 con 0x7fdba01019c0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.924+0000 7fdb9de00640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000eb90 tx=0x7fdb9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.925+0000 7fdb97400640 1 -- 172.21.15.59:0/3086304210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004d20 con 0x7fdba01019c0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.925+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba019f790 con 0x7fdba01019c0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.925+0000 7fdb97400640 1 -- 172.21.15.59:0/3086304210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9000ee90 con 0x7fdba01019c0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.925+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba01a7070 con 0x7fdba01019c0 2024-09-05T12:13:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.925+0000 7fdb97400640 1 -- 172.21.15.59:0/3086304210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900107d0 con 0x7fdba01019c0 2024-09-05T12:13:36.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdba7122740 1 Processor -- start 2024-09-05T12:13:36.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdb97400640 1 -- 172.21.15.59:0/3086304210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb90005000 con 0x7fdba01019c0 2024-09-05T12:13:36.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdba7122740 1 -- start start 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdba7122740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdba7122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa0845d0 0x5587aa08e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.928+0000 7fdba7122740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa091280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdba7122740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587a9fea290 con 0x5587aa08ee90 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdba7122740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5587a9f91b40 con 0x5587aa083cb0 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdba7122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5587a9fe6250 con 0x5587aa0845d0 2024-09-05T12:13:36.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb95400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa091280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44120/0 (socket says 172.21.15.59:44120) 2024-09-05T12:13:36.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 -- 172.21.15.59:0/145368117 learned_addr learned my addr 172.21.15.59:0/145368117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9de00640 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa0845d0 0x5587aa08e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 -- 172.21.15.59:0/145368117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa0845d0 msgr2=0x5587aa08e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa0845d0 0x5587aa08e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 -- 172.21.15.59:0/145368117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 msgr2=0x5587aa091280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa091280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.929+0000 7fdb9e800640 1 -- 172.21.15.59:0/145368117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a9d5b6a0 con 0x5587aa083cb0 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb97400640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 0x7fdb840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb9e800640 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b700 tx=0x7fdb9800bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ca7e742141544a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb97400640 1 -- 172.21.15.59:0/3086304210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb900cfba0 con 0x7fdba01019c0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb8f000640 1 -- 172.21.15.59:0/145368117 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003dd0 con 0x5587aa083cb0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb9de00640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 0x7fdb840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb8f000640 1 -- 172.21.15.59:0/145368117 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb98015070 con 0x5587aa083cb0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.930+0000 7fdb8f000640 1 -- 172.21.15.59:0/145368117 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980106a0 con 0x5587aa083cb0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- 172.21.15.59:0/145368117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 msgr2=0x5587aa084090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdb9de00640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 0x7fdb840a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb900cfe40 tx=0x7fdb900d35e0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b700 tx=0x7fdb9800bbc0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- 172.21.15.59:0/145368117 shutdown_connections 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa091280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa0845d0 0x5587aa08e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 --2- 172.21.15.59:0/145368117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa083cb0 0x5587aa084090 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- 172.21.15.59:0/145368117 >> 172.21.15.59:0/145368117 conn(0x5587aa080240 msgr2=0x5587aa080620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- 172.21.15.59:0/145368117 shutdown_connections 2024-09-05T12:13:36.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- 172.21.15.59:0/145368117 wait complete. 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.931+0000 7fdba7122740 1 -- start start 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 Processor -- start 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa08e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa08ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa08bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587aa091b00 con 0x5587aa083cb0 2024-09-05T12:13:36.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5587a9fa4fa0 con 0x5587aa0845d0 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5587aa091c80 con 0x5587aa08ee90 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdba7122740 10 client.0 _ll_register_callbacks cb 0x5587a9cf8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa08e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa08e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42046/0 (socket says 172.21.15.59:42046) 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdb9e800640 1 -- 172.21.15.59:0/224720593 learned_addr learned my addr 172.21.15.59:0/224720593 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdb95400640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa08ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.932+0000 7fdb9e800640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa08bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 -- 172.21.15.59:0/224720593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 msgr2=0x5587aa08bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa08bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 -- 172.21.15.59:0/224720593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 msgr2=0x5587aa08e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa08e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 -- 172.21.15.59:0/224720593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587aa08c820 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb95400640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa08ba90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a940 tx=0x7fdb88004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce222f6f143d5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb83400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88035040 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.933+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb98015070 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb88005450 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb88005650 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 10 client.28076 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5587aa09da90 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb98015070 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdb880039c0 con 0x5587aa0845d0 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 10 client.28076 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 10 client.28076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 20 client.28076 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:36.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa09eea0 0x5587aa0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 4 client.28076 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.934+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a9ff7800 con 0x5587aa09eea0 2024-09-05T12:13:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.935+0000 7fdb9e800640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa09eea0 0x5587aa0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.935+0000 7fdb9e800640 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa09eea0 0x5587aa0a12d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f538961ad74f93e server_cookie=2da3e4cf8f770498 in_seq=0 out_seq=0 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.935+0000 7fdb83400640 10 client.28076 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdb83400640 1 -- 172.21.15.59:0/224720593 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a9ff7800 con 0x5587aa09eea0 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdb83400640 10 client.28076 handle_command_reply: tid=0 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 client.28076 shutdown 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 msgr2=0x5587aa08ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa08ba90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802a940 tx=0x7fdb88004440 comp rx=0 tx=0).stop 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 shutdown_connections 2024-09-05T12:13:36.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa08bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa08ba90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa09eea0 0x5587aa0a12d0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 --2- 172.21.15.59:0/224720593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa08e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdb83400640 0 client.28076 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.936+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 >> 172.21.15.59:0/224720593 conn(0x5587aa080240 msgr2=0x5587aa085f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.937+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 shutdown_connections 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.937+0000 7fdba7122740 1 -- 172.21.15.59:0/224720593 wait complete. 2024-09-05T12:13:36.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.937+0000 7fdba7122740 20 client.28076 trim_cache size 0 max 16384 2024-09-05T12:13:36.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 Processor -- start 2024-09-05T12:13:36.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 -- start start 2024-09-05T12:13:36.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa0a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa07ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01aefd0 con 0x5587aa083cb0 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01b1850 con 0x5587aa0845d0 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.955+0000 7fdba7122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01af640 con 0x5587aa08ee90 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42056/0 (socket says 172.21.15.59:42056) 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa0a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb9e800640 1 -- 172.21.15.59:0/3267548216 learned_addr learned my addr 172.21.15.59:0/3267548216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb9e800640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa07ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 -- 172.21.15.59:0/3267548216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 msgr2=0x5587aa07ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa07ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 -- 172.21.15.59:0/3267548216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 msgr2=0x5587aa0a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 -- 172.21.15.59:0/3267548216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587aa08a000 con 0x5587aa0845d0 2024-09-05T12:13:36.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb9e800640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:36.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.956+0000 7fdb95400640 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa0a70e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdb88003f80 tx=0x7fdb8802ab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11ef22b28e15803 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdb8e600640 1 -- 172.21.15.59:0/3267548216 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88009520 con 0x5587aa0845d0 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdb8e600640 1 -- 172.21.15.59:0/3267548216 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb88041070 con 0x5587aa0845d0 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdb8e600640 1 -- 172.21.15.59:0/3267548216 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88003850 con 0x5587aa0845d0 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 -- 172.21.15.59:0/3267548216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 msgr2=0x5587aa0a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa0a70e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdb88003f80 tx=0x7fdb8802ab60 comp rx=0 tx=0).stop 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 -- 172.21.15.59:0/3267548216 shutdown_connections 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa08ee90 0x5587aa07ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa0a70e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 --2- 172.21.15.59:0/3267548216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa083cb0 0x5587aa0a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.957+0000 7fdba7122740 1 -- 172.21.15.59:0/3267548216 >> 172.21.15.59:0/3267548216 conn(0x5587aa080240 msgr2=0x5587aa08aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.958+0000 7fdba7122740 1 -- 172.21.15.59:0/3267548216 shutdown_connections 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.958+0000 7fdba7122740 1 -- 172.21.15.59:0/3267548216 wait complete. 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.958+0000 7fdba7122740 1 -- start start 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.958+0000 7fdba7122740 1 Processor -- start 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa083cb0 0x5587aa0977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa092c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01afd10 con 0x5587aa08ee90 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01b1180 con 0x5587aa0845d0 2024-09-05T12:13:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01b03e0 con 0x5587aa083cb0 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdba7122740 10 client.0 _ll_register_callbacks cb 0x5587aa08c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa083cb0 0x5587aa0977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb95400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa092c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb95400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa092c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44146/0 (socket says 172.21.15.59:44146) 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42072/0 (socket says 172.21.15.59:42072) 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.959+0000 7fdb95400640 1 -- 172.21.15.59:0/4193560059 learned_addr learned my addr 172.21.15.59:0/4193560059 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 -- 172.21.15.59:0/4193560059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa083cb0 msgr2=0x5587aa0977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa083cb0 0x5587aa0977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 -- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 msgr2=0x5587aa092c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa092c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 -- 172.21.15.59:0/4193560059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587aa108b70 con 0x5587aa08ee90 2024-09-05T12:13:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb9e800640 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdb98011e00 tx=0x7fdb9800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4e79dab70b5d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb83400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98023040 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb88041070 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fdb98011640 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.960+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb98040050 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 10 client.18081 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5587aa109d90 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb88041070 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdb9802a820 con 0x5587aa08ee90 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 10 client.18081 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 20 client.18081 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa10a2d0 0x5587aa10c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 4 client.18081 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.961+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587a9ff1d10 con 0x5587aa10a2d0 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.962+0000 7fdb95400640 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa10a2d0 0x5587aa10c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.962+0000 7fdb95400640 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa10a2d0 0x5587aa10c6c0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d331fdb9a4d524 server_cookie=f130f444452972d7 in_seq=0 out_seq=0 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.962+0000 7fdb83400640 10 client.18081 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.963+0000 7fdb83400640 1 -- 172.21.15.59:0/4193560059 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587a9ff1d10 con 0x5587aa10a2d0 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.963+0000 7fdb83400640 10 client.18081 handle_command_reply: tid=0 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.963+0000 7fdba7122740 1 client.18081 shutdown 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.963+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 msgr2=0x5587aa0933a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.963+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdb98011e00 tx=0x7fdb9800b2e0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 shutdown_connections 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x5587aa10a2d0 0x5587aa10c6c0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587aa08ee90 0x5587aa0933a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5587aa0845d0 0x5587aa092c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 --2- 172.21.15.59:0/4193560059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587aa083cb0 0x5587aa0977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdb83400640 0 client.18081 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 >> 172.21.15.59:0/4193560059 conn(0x5587aa080240 msgr2=0x5587aa07ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 shutdown_connections 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 1 -- 172.21.15.59:0/4193560059 wait complete. 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.964+0000 7fdba7122740 20 client.18081 trim_cache size 0 max 16384 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "5cef2407-6606-4957-8ddd-cc60a87c368e", 2024-09-05T12:13:36.968 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 msgr2=0x7fdb840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 0x7fdb840a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb900cfe40 tx=0x7fdb900d35e0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 msgr2=0x7fdba019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000eb90 tx=0x7fdb9000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 shutdown_connections 2024-09-05T12:13:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fdb840a2b80 0x7fdb840a5040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba01079c0 0x7fdba019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdba0102360 0x7fdba019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 --2- 172.21.15.59:0/3086304210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba01019c0 0x7fdba019a4f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 >> 172.21.15.59:0/3086304210 conn(0x7fdba00fd740 msgr2=0x7fdba010d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 shutdown_connections 2024-09-05T12:13:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:36.965+0000 7fdb9f200640 1 -- 172.21.15.59:0/3086304210 wait complete. 2024-09-05T12:13:37.067 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:36 smithi059 ceph-mon[30025]: pgmap v872: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:13:37.067 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:36 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2306072095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:36 smithi089 ceph-mon[34499]: pgmap v872: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:13:37.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:36 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2306072095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:36 smithi189 ceph-mon[34772]: pgmap v872: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-05T12:13:37.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:36 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2306072095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:37.808 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24545 quiesce db --cancel --all' 2024-09-05T12:13:37.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24545 quiesce db --cancel --all 2024-09-05T12:13:38.081 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:37 smithi059 ceph-mon[30025]: scrub queued for path: / 2024-09-05T12:13:38.081 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:37 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.081 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:37 smithi059 ceph-mon[30025]: scrub summary: active paths [/] 2024-09-05T12:13:38.081 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:37 smithi059 ceph-mon[30025]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-05T12:13:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:37 smithi089 ceph-mon[34499]: scrub queued for path: / 2024-09-05T12:13:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:37 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:37 smithi089 ceph-mon[34499]: scrub summary: active paths [/] 2024-09-05T12:13:38.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:37 smithi089 ceph-mon[34499]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.207 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:38.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:37 smithi189 ceph-mon[34772]: scrub queued for path: / 2024-09-05T12:13:38.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:37 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:37 smithi189 ceph-mon[34772]: scrub summary: active paths [/] 2024-09-05T12:13:38.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:37 smithi189 ceph-mon[34772]: scrub summary: idle+waiting paths [/] 2024-09-05T12:13:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 -- 172.21.15.59:0/3078867074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074f40 msgr2=0x7ff8e010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 --2- 172.21.15.59:0/3078867074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074f40 0x7ff8e010de80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009900 tx=0x7ff8c802a480 comp rx=0 tx=0).stop 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 -- 172.21.15.59:0/3078867074 shutdown_connections 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 --2- 172.21.15.59:0/3078867074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 0x7ff8e0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 --2- 172.21.15.59:0/3078867074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074f40 0x7ff8e010de80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 --2- 172.21.15.59:0/3078867074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074620 0x7ff8e0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 -- 172.21.15.59:0/3078867074 >> 172.21.15.59:0/3078867074 conn(0x7ff8e00fd8c0 msgr2=0x7ff8e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 -- 172.21.15.59:0/3078867074 shutdown_connections 2024-09-05T12:13:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.803+0000 7ff8e6000640 1 -- 172.21.15.59:0/3078867074 wait complete. 2024-09-05T12:13:38.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.804+0000 7ff8e6000640 1 Processor -- start 2024-09-05T12:13:38.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.804+0000 7ff8e6000640 1 -- start start 2024-09-05T12:13:38.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074f40 0x7ff8e019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 0x7ff8e019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114190 con 0x7ff8e0074620 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114010 con 0x7ff8e0074f40 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8e6000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114310 con 0x7ff8e010e3c0 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42088/0 (socket says 172.21.15.59:42088) 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 -- 172.21.15.59:0/4105596345 learned_addr learned my addr 172.21.15.59:0/4105596345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 0x7ff8e019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074f40 0x7ff8e019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 -- 172.21.15.59:0/4105596345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 msgr2=0x7ff8e019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 0x7ff8e019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 -- 172.21.15.59:0/4105596345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074f40 msgr2=0x7ff8e019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.805+0000 7ff8df400640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074f40 0x7ff8e019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8df400640 1 -- 172.21.15.59:0/4105596345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e019f640 con 0x7ff8e0074620 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8df400640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000e9f0 tx=0x7ff8d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8dcc00640 1 -- 172.21.15.59:0/4105596345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000ccc0 con 0x7ff8e0074620 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e019f930 con 0x7ff8e0074620 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8dcc00640 1 -- 172.21.15.59:0/4105596345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0010910 con 0x7ff8e0074620 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.806+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a7210 con 0x7ff8e0074620 2024-09-05T12:13:38.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.807+0000 7ff8dcc00640 1 -- 172.21.15.59:0/4105596345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0018840 con 0x7ff8e0074620 2024-09-05T12:13:38.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.810+0000 7ff8dcc00640 1 -- 172.21.15.59:0/4105596345 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff8d0010ac0 con 0x7ff8e0074620 2024-09-05T12:13:38.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.810+0000 7ff8e8ada740 1 Processor -- start 2024-09-05T12:13:38.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.810+0000 7ff8e8ada740 1 -- start start 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f72fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f73a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f73cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56190f696340 con 0x56190f72f6d0 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56190f63dbf0 con 0x56190f72fff0 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56190f692300 con 0x56190f73a8b0 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f72fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8df400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f73a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8df400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f73a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:44170/0 (socket says 172.21.15.59:44170) 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8df400640 1 -- 172.21.15.59:0/1553910118 learned_addr learned my addr 172.21.15.59:0/1553910118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f73cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.811+0000 7ff8dcc00640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8dcc00640 1 -- 172.21.15.59:0/4105596345 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff8d00cf8c0 con 0x7ff8e0074620 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 -- 172.21.15.59:0/1553910118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 msgr2=0x56190f73a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f73a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 -- 172.21.15.59:0/1553910118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 msgr2=0x56190f72fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f72fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 -- 172.21.15.59:0/1553910118 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190f407620 con 0x56190f73a8b0 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.812+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f73cca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8c802aa70 tx=0x7ff8c8005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9add713818de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8cfe00640 1 -- 172.21.15.59:0/1553910118 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8035040 con 0x56190f73a8b0 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8cfe00640 1 -- 172.21.15.59:0/1553910118 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c8004400 con 0x56190f73a8b0 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8cfe00640 1 -- 172.21.15.59:0/1553910118 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80035f0 con 0x56190f73a8b0 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1553910118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 msgr2=0x56190f73cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f73cca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8c802aa70 tx=0x7ff8c8005de0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 0x7ff8c40a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80094d0 tx=0x7ff8d8004ab0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1553910118 shutdown_connections 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f73cca0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f73a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1553910118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f72fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.813+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1553910118 >> 172.21.15.59:0/1553910118 conn(0x56190f72bc60 msgr2=0x56190f72c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.814+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1553910118 shutdown_connections 2024-09-05T12:13:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.814+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1553910118 wait complete. 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.814+0000 7ff8e8ada740 1 -- start start 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.814+0000 7ff8e8ada740 1 Processor -- start 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72f6d0 0x56190f7377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f737cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56190f73d420 con 0x56190f73a8b0 2024-09-05T12:13:38.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56190f651060 con 0x56190f72fff0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56190f73d5a0 con 0x56190f72f6d0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8e8ada740 10 client.0 _ll_register_callbacks cb 0x56190f63faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42100/0 (socket says 172.21.15.59:42100) 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8dfe00640 1 -- 172.21.15.59:0/3951112589 learned_addr learned my addr 172.21.15.59:0/3951112589 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8df400640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72f6d0 0x56190f7377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.815+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f737cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 -- 172.21.15.59:0/3951112589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72f6d0 msgr2=0x56190f7377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72f6d0 0x56190f7377b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 -- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 msgr2=0x56190f737cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f737cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 -- 172.21.15.59:0/3951112589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190f746940 con 0x56190f73a8b0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f737cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800fb80 tx=0x7ff8d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357bd7d7554064f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.816+0000 7ff8ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8022450 con 0x56190f73a8b0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff8d80225f0 con 0x56190f73a8b0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8c8004400 con 0x56190f73a8b0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff8d80227f0 con 0x56190f73a8b0 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8e8ada740 10 client.18093 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:38.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56190f749360 con 0x56190f73a8b0 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c8004400 con 0x56190f73a8b0 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.817+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff8d801f040 con 0x56190f73a8b0 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 10 client.18093 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 10 client.18093 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 20 client.18093 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f74a820 0x56190f74cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 4 client.18093 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190f6a38b0 con 0x56190f74a820 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8df400640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f74a820 0x56190f74cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.818+0000 7ff8df400640 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f74a820 0x56190f74cc50 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46ef962d1b433d4 server_cookie=adc59b9d59c35377 in_seq=0 out_seq=0 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.819+0000 7ff8ccc00640 10 client.18093 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.819+0000 7ff8ccc00640 1 -- 172.21.15.59:0/3951112589 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56190f6a38b0 con 0x56190f74a820 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.819+0000 7ff8ccc00640 10 client.18093 handle_command_reply: tid=0 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.819+0000 7ff8e8ada740 1 client.18093 shutdown 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 msgr2=0x56190f738230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800fb80 tx=0x7ff8d8004a40 comp rx=0 tx=0).stop 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 shutdown_connections 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f74a820 0x56190f74cc50 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f73a8b0 0x56190f738230 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72fff0 0x56190f737cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/3951112589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72f6d0 0x56190f7377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 >> 172.21.15.59:0/3951112589 conn(0x56190f72bc60 msgr2=0x56190f731990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8ccc00640 0 client.18093 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 shutdown_connections 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 1 -- 172.21.15.59:0/3951112589 wait complete. 2024-09-05T12:13:38.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.820+0000 7ff8e8ada740 20 client.18093 trim_cache size 0 max 16384 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.838+0000 7ff8e8ada740 1 Processor -- start 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.838+0000 7ff8e8ada740 1 -- start start 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72fff0 0x56190f752a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 0x56190f734a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01af100 con 0x56190f72f6d0 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01b1980 con 0x56190f73a8b0 2024-09-05T12:13:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01af770 con 0x56190f72fff0 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42110/0 (socket says 172.21.15.59:42110) 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8df400640 1 -- 172.21.15.59:0/1611711699 learned_addr learned my addr 172.21.15.59:0/1611711699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8d7e00640 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72fff0 0x56190f752a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.839+0000 7ff8dfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 0x56190f734a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 -- 172.21.15.59:0/1611711699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72fff0 msgr2=0x56190f752a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72fff0 0x56190f752a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 -- 172.21.15.59:0/1611711699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 msgr2=0x56190f752520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 -- 172.21.15.59:0/1611711699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190f74d2d0 con 0x56190f73a8b0 2024-09-05T12:13:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8df400640 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 0x56190f734a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8012040 tx=0x7ff8d802bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8757f28fa08de25a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.840+0000 7ff8cf400640 1 -- 172.21.15.59:0/1611711699 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d803d030 con 0x56190f73a8b0 2024-09-05T12:13:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.841+0000 7ff8cf400640 1 -- 172.21.15.59:0/1611711699 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d8010cc0 con 0x56190f73a8b0 2024-09-05T12:13:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.841+0000 7ff8cf400640 1 -- 172.21.15.59:0/1611711699 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80229b0 con 0x56190f73a8b0 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1611711699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 msgr2=0x56190f734a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 0x56190f734a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8012040 tx=0x7ff8d802bf80 comp rx=0 tx=0).stop 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1611711699 shutdown_connections 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f73a8b0 0x56190f734a10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f72fff0 0x56190f752a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/1611711699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72f6d0 0x56190f752520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1611711699 >> 172.21.15.59:0/1611711699 conn(0x56190f72bc60 msgr2=0x56190f739830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1611711699 shutdown_connections 2024-09-05T12:13:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- 172.21.15.59:0/1611711699 wait complete. 2024-09-05T12:13:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.842+0000 7ff8e8ada740 1 -- start start 2024-09-05T12:13:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 Processor -- start 2024-09-05T12:13:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 0x56190f73eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 0x56190f73f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f7ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01afe40 con 0x56190f72fff0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01b12b0 con 0x56190f72f6d0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01b0510 con 0x56190f73a8b0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.843+0000 7ff8e8ada740 10 client.0 _ll_register_callbacks cb 0x7ff8d8013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 0x56190f73f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 0x56190f73f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42124/0 (socket says 172.21.15.59:42124) 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f7ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8d7e00640 1 -- 172.21.15.59:0/160561944 learned_addr learned my addr 172.21.15.59:0/160561944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8df400640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 0x56190f73eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 -- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 msgr2=0x56190f73eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 0x56190f73eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 -- 172.21.15.59:0/160561944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 msgr2=0x56190f73f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 0x56190f73f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 -- 172.21.15.59:0/160561944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190f7aec20 con 0x56190f73a8b0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8df400640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 0x56190f73eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8dfe00640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f7ae530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80134f0 tx=0x7ff8d8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825b2d58d691b656 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.844+0000 7ff8ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8046060 con 0x56190f73a8b0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d8010cc0 con 0x56190f73a8b0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff8d8022ba0 con 0x56190f73a8b0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff8d804d050 con 0x56190f73a8b0 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8e8ada740 10 client.28030 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56190f7b5f80 con 0x56190f73a8b0 2024-09-05T12:13:38.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.845+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d8010cc0 con 0x56190f73a8b0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff8d8003a00 con 0x56190f73a8b0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 10 client.28030 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 10 client.28030 resolve_mds: validated gid 24545 aka daemon mds.b 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 20 client.28030 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f7b65d0 0x56190f7b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 4 client.28030 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x56190f69ddc0 con 0x56190f7b65d0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.846+0000 7ff8df400640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f7b65d0 0x56190f7b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8df400640 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f7b65d0 0x56190f7b89f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97c35df0d7b6a80 server_cookie=a4d17dc757ede6c6 in_seq=0 out_seq=0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8ccc00640 10 client.28030 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8ccc00640 1 -- 172.21.15.59:0/160561944 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56190f69ddc0 con 0x56190f7b65d0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8ccc00640 10 client.28030 handle_command_reply: tid=0 2024-09-05T12:13:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 client.28030 shutdown 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 msgr2=0x56190f7ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f7ae530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80134f0 tx=0x7ff8d8005fd0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 shutdown_connections 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56190f73a8b0 0x56190f7ae530 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x56190f7b65d0 0x56190f7b89f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190f72fff0 0x56190f73f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 --2- 172.21.15.59:0/160561944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56190f72f6d0 0x56190f73eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.847+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 >> 172.21.15.59:0/160561944 conn(0x56190f72bc60 msgr2=0x56190f731990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8ccc00640 0 client.28030 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 shutdown_connections 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8e8ada740 1 -- 172.21.15.59:0/160561944 wait complete. 2024-09-05T12:13:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8e8ada740 20 client.28030 trim_cache size 0 max 16384 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 21, 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24545, 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 56, 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": {} 2024-09-05T12:13:38.852 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 msgr2=0x7ff8c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.848+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 0x7ff8c40a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80094d0 tx=0x7ff8d8004ab0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 msgr2=0x7ff8e019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000e9f0 tx=0x7ff8d000edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 shutdown_connections 2024-09-05T12:13:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8e010e3c0 0x7ff8e019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8e0074f40 0x7ff8e019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 --2- 172.21.15.59:0/4105596345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e0074620 0x7ff8e019a690 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 >> 172.21.15.59:0/4105596345 conn(0x7ff8e00fd8c0 msgr2=0x7ff8e00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 shutdown_connections 2024-09-05T12:13:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:38.849+0000 7ff8e6000640 1 -- 172.21.15.59:0/4105596345 wait complete. 2024-09-05T12:13:38.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:38 smithi059 ceph-mon[30025]: pgmap v873: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:13:39.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:38 smithi089 ceph-mon[34499]: pgmap v873: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:13:39.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:38 smithi189 ceph-mon[34772]: pgmap v873: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-09-05T12:13:39.643 INFO:tasks.quiescer:done joining Quiescers 2024-09-05T12:13:39.643 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-05T12:13:39.658 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-05T12:13:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 -- 172.21.15.59:0/3193036721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac1049a0 msgr2=0x7f30ac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 --2- 172.21.15.59:0/3193036721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac1049a0 0x7f30ac1089f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3098009860 tx=0x7f309802a340 comp rx=0 tx=0).stop 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 -- 172.21.15.59:0/3193036721 shutdown_connections 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 --2- 172.21.15.59:0/3193036721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac109120 0x7f30ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 --2- 172.21.15.59:0/3193036721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac1049a0 0x7f30ac1089f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 --2- 172.21.15.59:0/3193036721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac103ff0 0x7f30ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.264+0000 7f30b2400640 1 -- 172.21.15.59:0/3193036721 >> 172.21.15.59:0/3193036721 conn(0x7f30ac0ffa90 msgr2=0x7f30ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.265+0000 7f30b2400640 1 -- 172.21.15.59:0/3193036721 shutdown_connections 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.265+0000 7f30b2400640 1 -- 172.21.15.59:0/3193036721 wait complete. 2024-09-05T12:13:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.265+0000 7f30b2400640 1 Processor -- start 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 -- start start 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac1049a0 0x7f30ac19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac109120 0x7f30ac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac114120 con 0x7f30ac103ff0 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac113fa0 con 0x7f30ac1049a0 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30b2400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1142a0 con 0x7f30ac109120 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42146/0 (socket says 172.21.15.59:42146) 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30ab400640 1 -- 172.21.15.59:0/1583225575 learned_addr learned my addr 172.21.15.59:0/1583225575 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30abe00640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac109120 0x7f30ac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.266+0000 7f30aaa00640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac1049a0 0x7f30ac19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 -- 172.21.15.59:0/1583225575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac109120 msgr2=0x7f30ac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac109120 0x7f30ac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 -- 172.21.15.59:0/1583225575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac1049a0 msgr2=0x7f30ac19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac1049a0 0x7f30ac19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 -- 172.21.15.59:0/1583225575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac19f5d0 con 0x7f30ac103ff0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30ab400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9c0 tx=0x7f309c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30a3e00640 1 -- 172.21.15.59:0/1583225575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c00cd80 con 0x7f30ac103ff0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac19f8c0 con 0x7f30ac103ff0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30a3e00640 1 -- 172.21.15.59:0/1583225575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309c0035f0 con 0x7f30ac103ff0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30a3e00640 1 -- 172.21.15.59:0/1583225575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c010670 con 0x7f30ac103ff0 2024-09-05T12:13:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.267+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a71a0 con 0x7f30ac103ff0 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 Processor -- start 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30a3e00640 1 -- 172.21.15.59:0/1583225575 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f309c0037a0 con 0x7f30ac103ff0 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 -- start start 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f48435f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48440830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f4839c3e0 con 0x563f48440d70 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563f48343d80 con 0x563f484364b0 2024-09-05T12:13:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f483983a0 con 0x563f48435b90 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42150/0 (socket says 172.21.15.59:42150) 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.271+0000 7f30aaa00640 1 -- 172.21.15.59:0/4106017559 learned_addr learned my addr 172.21.15.59:0/4106017559 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30ab400640 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48440830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30abe00640 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f48435f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 -- 172.21.15.59:0/4106017559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 msgr2=0x563f48435f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f48435f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 -- 172.21.15.59:0/4106017559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 msgr2=0x563f48440830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48440830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 -- 172.21.15.59:0/4106017559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f48362ae0 con 0x563f48440d70 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30a3e00640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 0x7f30900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.272+0000 7f30aaa00640 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3098009780 tx=0x7f3098009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d539e68b43b97208 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.273+0000 7f30a3e00640 1 -- 172.21.15.59:0/1583225575 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f309c0cf740 con 0x7f30ac103ff0 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.273+0000 7f30abe00640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 0x7f30900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.273+0000 7f30a0c00640 1 -- 172.21.15.59:0/4106017559 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098041020 con 0x563f48440d70 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.273+0000 7f30a0c00640 1 -- 172.21.15.59:0/4106017559 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098030e50 con 0x563f48440d70 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.273+0000 7f30a0c00640 1 -- 172.21.15.59:0/4106017559 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098037040 con 0x563f48440d70 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 -- 172.21.15.59:0/4106017559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 msgr2=0x563f48443160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30abe00640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 0x7f30900a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005fd0 tx=0x7f30a4009500 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3098009780 tx=0x7f3098009510 comp rx=0 tx=0).stop 2024-09-05T12:13:40.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 -- 172.21.15.59:0/4106017559 shutdown_connections 2024-09-05T12:13:40.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48440d70 0x563f48443160 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48440830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/4106017559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f48435f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 -- 172.21.15.59:0/4106017559 >> 172.21.15.59:0/4106017559 conn(0x563f48432120 msgr2=0x563f48432500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 -- 172.21.15.59:0/4106017559 shutdown_connections 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.274+0000 7f30b4e3a740 1 -- 172.21.15.59:0/4106017559 wait complete. 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 -- start start 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 Processor -- start 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4843dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f48443980 con 0x563f484364b0 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563f483570f0 con 0x563f48440d70 2024-09-05T12:13:40.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f48443b00 con 0x563f48435b90 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.275+0000 7f30b4e3a740 10 client.0 _ll_register_callbacks cb 0x563f483489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42152/0 (socket says 172.21.15.59:42152) 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 -- 172.21.15.59:0/1206491275 learned_addr learned my addr 172.21.15.59:0/1206491275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30ab400640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4843dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30abe00640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 -- 172.21.15.59:0/1206491275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 msgr2=0x563f4843dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4843dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 -- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 msgr2=0x563f4843e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 -- 172.21.15.59:0/1206491275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4810d3c0 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f30aaa00640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f309802a930 tx=0x7f30980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f163a92446605bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.276+0000 7f3095600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098041030 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f3098030760 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3098030e50 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3098058050 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f30b4e3a740 10 client.18111 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f4844f680 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.277+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098030e50 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f309803d6e0 con 0x563f484364b0 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 10 client.18111 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 20 client.18111 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:40.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484509d0 0x563f48452dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 4 client.18111 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f483a9950 con 0x563f484509d0 2024-09-05T12:13:40.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.278+0000 7f30ab400640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484509d0 0x563f48452dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.279+0000 7f30ab400640 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484509d0 0x563f48452dc0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced4b76d235d3127 server_cookie=7ebea57b31cbd274 in_seq=0 out_seq=0 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.279+0000 7f3095600640 10 client.18111 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.279+0000 7f3095600640 1 -- 172.21.15.59:0/1206491275 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f483a9950 con 0x563f484509d0 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.279+0000 7f3095600640 10 client.18111 handle_command_reply: tid=0 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.279+0000 7f30b4e3a740 1 client.18111 shutdown 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 msgr2=0x563f4843e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f309802a930 tx=0x7f30980043b0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 shutdown_connections 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484509d0 0x563f48452dc0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843e1f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/1206491275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4843dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 >> 172.21.15.59:0/1206491275 conn(0x563f48432120 msgr2=0x563f48437e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f3095600640 0 client.18111 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 shutdown_connections 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 1 -- 172.21.15.59:0/1206491275 wait complete. 2024-09-05T12:13:40.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.280+0000 7f30b4e3a740 20 client.18111 trim_cache size 0 max 16384 2024-09-05T12:13:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 Processor -- start 2024-09-05T12:13:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 -- start start 2024-09-05T12:13:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4844c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1af0e0 con 0x563f484364b0 2024-09-05T12:13:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b1960 con 0x563f48440d70 2024-09-05T12:13:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.298+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1af750 con 0x563f48435b90 2024-09-05T12:13:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42166/0 (socket says 172.21.15.59:42166) 2024-09-05T12:13:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 -- 172.21.15.59:0/33969372 learned_addr learned my addr 172.21.15.59:0/33969372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:40.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30ab400640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4844c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 -- 172.21.15.59:0/33969372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 msgr2=0x563f4844c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4844c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 -- 172.21.15.59:0/33969372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 msgr2=0x563f4843b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 -- 172.21.15.59:0/33969372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f48443a00 con 0x563f484364b0 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30ab400640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4844c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30abe00640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.299+0000 7f30aaa00640 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f309803dca0 tx=0x7f30980056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445367946e24211 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f3097e00640 1 -- 172.21.15.59:0/33969372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309804a020 con 0x563f484364b0 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f3097e00640 1 -- 172.21.15.59:0/33969372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098030e50 con 0x563f484364b0 2024-09-05T12:13:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f3097e00640 1 -- 172.21.15.59:0/33969372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098041070 con 0x563f484364b0 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 -- 172.21.15.59:0/33969372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 msgr2=0x563f4843af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f309803dca0 tx=0x7f30980056d0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 -- 172.21.15.59:0/33969372 shutdown_connections 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f48440d70 0x563f4843b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f484364b0 0x563f4843af10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/33969372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48435b90 0x563f4844c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 -- 172.21.15.59:0/33969372 >> 172.21.15.59:0/33969372 conn(0x563f48432120 msgr2=0x563f4844bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 -- 172.21.15.59:0/33969372 shutdown_connections 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.300+0000 7f30b4e3a740 1 -- 172.21.15.59:0/33969372 wait complete. 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.301+0000 7f30b4e3a740 1 -- start start 2024-09-05T12:13:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.301+0000 7f30b4e3a740 1 Processor -- start 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.301+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.301+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48454090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30b4e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48440d70 0x563f484545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1afe20 con 0x563f48435b90 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b1290 con 0x563f484364b0 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30b4e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1b04f0 con 0x563f48440d70 2024-09-05T12:13:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30b4e3a740 10 client.0 _ll_register_callbacks cb 0x7f309c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42168/0 (socket says 172.21.15.59:42168) 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 -- 172.21.15.59:0/3642880523 learned_addr learned my addr 172.21.15.59:0/3642880523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30abe00640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48440d70 0x563f484545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30aaa00640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48454090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 -- 172.21.15.59:0/3642880523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48440d70 msgr2=0x563f484545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48440d70 0x563f484545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 -- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 msgr2=0x563f48454090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48454090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.302+0000 7f30ab400640 1 -- 172.21.15.59:0/3642880523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f484ba310 con 0x563f48435b90 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f30ab400640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c6a0 tx=0x7f309c0d54b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40568d09e331e9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f3095600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c09ce60 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f309c003b10 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3098030e50 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f309c09bb30 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f30b4e3a740 10 client.18117 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f484bb640 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.303+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098030e50 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f309c0981a0 con 0x563f48435b90 2024-09-05T12:13:40.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 10 client.18117 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 20 client.18117 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484bbd10 0x563f484be100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 4 client.18117 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-05T12:13:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f483a3e60 con 0x563f484bbd10 2024-09-05T12:13:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30aaa00640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484bbd10 0x563f484be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f30aaa00640 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484bbd10 0x563f484be100 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c2ad19b0dcd131 server_cookie=33a1cba337fa4006 in_seq=0 out_seq=0 2024-09-05T12:13:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.304+0000 7f3095600640 10 client.18117 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f3095600640 1 -- 172.21.15.59:0/3642880523 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f483a3e60 con 0x563f484bbd10 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f3095600640 10 client.18117 handle_command_reply: tid=0 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 client.18117 shutdown 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 msgr2=0x563f48453b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c6a0 tx=0x7f309c0d54b0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 shutdown_connections 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f48440d70 0x563f484545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563f484364b0 0x563f48454090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x563f484bbd10 0x563f484be100 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 --2- 172.21.15.59:0/3642880523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f48435b90 0x563f48453b50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 >> 172.21.15.59:0/3642880523 conn(0x563f48432120 msgr2=0x563f484375e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f3095600640 0 client.18117 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 shutdown_connections 2024-09-05T12:13:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 1 -- 172.21.15.59:0/3642880523 wait complete. 2024-09-05T12:13:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.305+0000 7f30b4e3a740 20 client.18117 trim_cache size 0 max 16384 2024-09-05T12:13:40.312 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:40.312 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-09-05T12:13:40.312 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-09-05T12:13:40.312 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 msgr2=0x7f30900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 0x7f30900a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005fd0 tx=0x7f30a4009500 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 msgr2=0x7f30ac19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9c0 tx=0x7f309c00edc0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 shutdown_connections 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f30900a2b80 0x7f30900a5040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac109120 0x7f30ac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30ac1049a0 0x7f30ac19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 --2- 172.21.15.59:0/1583225575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30ac103ff0 0x7f30ac19a620 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 >> 172.21.15.59:0/1583225575 conn(0x7f30ac0ffa90 msgr2=0x7f30ac100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 shutdown_connections 2024-09-05T12:13:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:40.306+0000 7f30b2400640 1 -- 172.21.15.59:0/1583225575 wait complete. 2024-09-05T12:13:40.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:40 smithi059 ceph-mon[30025]: pgmap v874: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:40.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:40 smithi059 ceph-mon[30025]: scrub summary: idle 2024-09-05T12:13:41.053 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-05T12:13:41.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:40 smithi089 ceph-mon[34499]: pgmap v874: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:41.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:40 smithi089 ceph-mon[34499]: scrub summary: idle 2024-09-05T12:13:41.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:40 smithi189 ceph-mon[34772]: pgmap v874: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:41.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:40 smithi189 ceph-mon[34772]: scrub summary: idle 2024-09-05T12:13:41.433 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.981+0000 7f5274000640 1 -- 172.21.15.59:0/3280023947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c074620 msgr2=0x7f526c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.981+0000 7f5274000640 1 --2- 172.21.15.59:0/3280023947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c074620 0x7f526c074a00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5260009900 tx=0x7f526002a480 comp rx=0 tx=0).stop 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 -- 172.21.15.59:0/3280023947 shutdown_connections 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 --2- 172.21.15.59:0/3280023947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c10e3c0 0x7f526c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 --2- 172.21.15.59:0/3280023947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074f40 0x7f526c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 --2- 172.21.15.59:0/3280023947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c074620 0x7f526c074a00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 -- 172.21.15.59:0/3280023947 >> 172.21.15.59:0/3280023947 conn(0x7f526c0fd8c0 msgr2=0x7f526c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 -- 172.21.15.59:0/3280023947 shutdown_connections 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.982+0000 7f5274000640 1 -- 172.21.15.59:0/3280023947 wait complete. 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 Processor -- start 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 -- start start 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074620 0x7f526c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 0x7f526c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1141a0 con 0x7f526c10e3c0 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114020 con 0x7f526c074620 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5274000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114320 con 0x7f526c074f40 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.983+0000 7f5270c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5272000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 0x7f526c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:54294/0 (socket says 172.21.15.59:54294) 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 -- 172.21.15.59:0/1603585438 learned_addr learned my addr 172.21.15.59:0/1603585438 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5271600640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074620 0x7f526c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 -- 172.21.15.59:0/1603585438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074620 msgr2=0x7f526c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074620 0x7f526c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 -- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 msgr2=0x7f526c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 0x7f526c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 -- 172.21.15.59:0/1603585438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19f5f0 con 0x7f526c074f40 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5272000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 0x7f526c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f5270c00640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9f0 tx=0x7f525800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.984+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525800cd80 con 0x7f526c074f40 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.985+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258010910 con 0x7f526c074f40 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.985+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19f8e0 con 0x7f526c074f40 2024-09-05T12:13:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.985+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258018990 con 0x7f526c074f40 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.985+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a71c0 con 0x7f526c074f40 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.986+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c074a00 con 0x7f526c074f40 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.988+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5258010ac0 con 0x7f526c074f40 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.989+0000 7f526a000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 0x7f52540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.989+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f52580cf7c0 con 0x7f526c074f40 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.989+0000 7f5271600640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 0x7f52540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.990+0000 7f5271600640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 0x7f52540a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5260000c00 tx=0x7f5260005b50 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:41.993+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f525809bc60 con 0x7f526c074f40 2024-09-05T12:13:42.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.214+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f526c108fb0 con 0x7f526c074f40 2024-09-05T12:13:42.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.215+0000 7f526a000640 1 -- 172.21.15.59:0/1603585438 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f525809ff60 con 0x7f526c074f40 2024-09-05T12:13:42.217 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:42.217 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:42.217 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.217+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 msgr2=0x7f52540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.217+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 0x7f52540a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5260000c00 tx=0x7f5260005b50 comp rx=0 tx=0).stop 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 msgr2=0x7f526c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9f0 tx=0x7f525800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 shutdown_connections 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f52540a2b80 0x7f52540a5040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c10e3c0 0x7f526c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f526c074f40 0x7f526c19ab80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 --2- 172.21.15.59:0/1603585438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f526c074620 0x7f526c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:42.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 >> 172.21.15.59:0/1603585438 conn(0x7f526c0fd8c0 msgr2=0x7f526c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:42.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 shutdown_connections 2024-09-05T12:13:42.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:42.218+0000 7f5274000640 1 -- 172.21.15.59:0/1603585438 wait complete. 2024-09-05T12:13:43.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-05T12:13:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:42 smithi089 ceph-mon[34499]: pgmap v875: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:43.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:42 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1603585438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:43.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:42 smithi059 ceph-mon[30025]: pgmap v875: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:43.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:42 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1603585438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:42 smithi189 ceph-mon[34772]: pgmap v875: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-05T12:13:43.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:42 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1603585438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:43.390 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 -- 172.21.15.59:0/2800928877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f18103e30 msgr2=0x7f5f18104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 --2- 172.21.15.59:0/2800928877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f18103e30 0x7f5f18104210 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009900 tx=0x7f5f0c02a410 comp rx=0 tx=0).stop 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 -- 172.21.15.59:0/2800928877 shutdown_connections 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 --2- 172.21.15.59:0/2800928877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 0x7f5f18110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 --2- 172.21.15.59:0/2800928877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181047e0 0x7f5f18108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 --2- 172.21.15.59:0/2800928877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f18103e30 0x7f5f18104210 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 -- 172.21.15.59:0/2800928877 >> 172.21.15.59:0/2800928877 conn(0x7f5f180ff8d0 msgr2=0x7f5f18101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 -- 172.21.15.59:0/2800928877 shutdown_connections 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.977+0000 7f5f1de00640 1 -- 172.21.15.59:0/2800928877 wait complete. 2024-09-05T12:13:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.978+0000 7f5f1de00640 1 Processor -- start 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.978+0000 7f5f1de00640 1 -- start start 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f18103e30 0x7f5f1819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 0x7f5f1819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18113fc0 con 0x7f5f181047e0 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18113e40 con 0x7f5f18108f60 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114140 con 0x7f5f18103e30 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59906/0 (socket says 172.21.15.59:59906) 2024-09-05T12:13:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f17e00640 1 -- 172.21.15.59:0/3282506731 learned_addr learned my addr 172.21.15.59:0/3282506731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f18103e30 0x7f5f1819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f1d400640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 0x7f5f1819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f17e00640 1 -- 172.21.15.59:0/3282506731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f18103e30 msgr2=0x7f5f1819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.979+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f18103e30 0x7f5f1819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f17e00640 1 -- 172.21.15.59:0/3282506731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 msgr2=0x7f5f1819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 0x7f5f1819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f17e00640 1 -- 172.21.15.59:0/3282506731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819f470 con 0x7f5f181047e0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400e9f0 tx=0x7f5f0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f15600640 1 -- 172.21.15.59:0/3282506731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400ccc0 con 0x7f5f181047e0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f15600640 1 -- 172.21.15.59:0/3282506731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0400ce60 con 0x7f5f181047e0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819f760 con 0x7f5f181047e0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f15600640 1 -- 172.21.15.59:0/3282506731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04010690 con 0x7f5f181047e0 2024-09-05T12:13:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.980+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a7040 con 0x7f5f181047e0 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.983+0000 7f5f15600640 1 -- 172.21.15.59:0/3282506731 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f040046c0 con 0x7f5f181047e0 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.983+0000 7f5f21fbb740 1 Processor -- start 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 -- start start 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabf1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1eab4d3e0 con 0x55f1eabf2020 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1eaaf4d80 con 0x55f1eabe7760 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1eab493a0 con 0x55f1eabe6e40 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f1d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59918/0 (socket says 172.21.15.59:59918) 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f17e00640 1 -- 172.21.15.59:0/2160760527 learned_addr learned my addr 172.21.15.59:0/2160760527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f1d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.59:54324/0 (socket says 172.21.15.59:54324) 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.984+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabf1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 -- 172.21.15.59:0/2160760527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 msgr2=0x55f1eabf1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabf1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 -- 172.21.15.59:0/2160760527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 msgr2=0x55f1eabf4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 -- 172.21.15.59:0/2160760527 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1eab13ae0 con 0x55f1eabe6e40 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f17e00640 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f1d400640 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009f10 tx=0x7f5f10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b902ebab7973ad0a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f09c00640 1 -- 172.21.15.59:0/2160760527 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10004d20 con 0x55f1eabe6e40 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f09c00640 1 -- 172.21.15.59:0/2160760527 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10004050 con 0x55f1eabe6e40 2024-09-05T12:13:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.985+0000 7f5f09c00640 1 -- 172.21.15.59:0/2160760527 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10005280 con 0x55f1eabe6e40 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f15600640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 0x7f5f000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f15600640 1 -- 172.21.15.59:0/3282506731 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5f040cf810 con 0x7f5f181047e0 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 0x7f5f000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 -- 172.21.15.59:0/2160760527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 msgr2=0x55f1eabe7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009f10 tx=0x7f5f10011810 comp rx=0 tx=0).stop 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 -- 172.21.15.59:0/2160760527 shutdown_connections 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabf2020 0x55f1eabf4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabf1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/2160760527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabe7220 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 -- 172.21.15.59:0/2160760527 >> 172.21.15.59:0/2160760527 conn(0x55f1eabe33d0 msgr2=0x55f1eabe37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.986+0000 7f5f21fbb740 1 -- 172.21.15.59:0/2160760527 shutdown_connections 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.987+0000 7f5f21fbb740 1 -- 172.21.15.59:0/2160760527 wait complete. 2024-09-05T12:13:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.987+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 0x7f5f000a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009230 tx=0x7f5f0c0058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.987+0000 7f5f21fbb740 1 -- start start 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.987+0000 7f5f21fbb740 1 Processor -- start 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eabef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f1eabf4bf0 con 0x55f1eabe6e40 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1eab080f0 con 0x55f1eabe7760 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1eabf4d70 con 0x55f1eabf2020 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f21fbb740 10 client.0 _ll_register_callbacks cb 0x55f1eaaefbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59928/0 (socket says 172.21.15.59:59928) 2024-09-05T12:13:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 -- 172.21.15.59:0/1886534854 learned_addr learned my addr 172.21.15.59:0/1886534854 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1d400640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eabef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f17e00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 -- 172.21.15.59:0/1886534854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 msgr2=0x55f1eabef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eabef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 -- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 msgr2=0x55f1eabef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f1ca00640 1 -- 172.21.15.59:0/1886534854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ea8be3c0 con 0x55f1eabe6e40 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.988+0000 7f5f17e00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004090 tx=0x7f5f0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c294cf84655d50f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5efe000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c04e070 con 0x55f1eabe6e40 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5f0c0037f0 con 0x55f1eabe6e40 2024-09-05T12:13:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f10004050 con 0x55f1eabe6e40 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c05a050 con 0x55f1eabe6e40 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5f21fbb740 10 client.18135 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.989+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1eac00c80 con 0x55f1eabe6e40 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f10004050 con 0x55f1eabe6e40 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5f0c04ac50 con 0x55f1eabe6e40 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 10 client.18135 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 20 client.18135 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac020e0 0x55f1eac04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 4 client.18135 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1eab5a950 con 0x55f1eac020e0 2024-09-05T12:13:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.990+0000 7f5f17e00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac020e0 0x55f1eac04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.991+0000 7f5f17e00640 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac020e0 0x55f1eac04550 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b680d436b174fe server_cookie=6293143341db9d19 in_seq=0 out_seq=0 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.991+0000 7f5efe000640 10 client.18135 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5efe000640 1 -- 172.21.15.59:0/1886534854 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1eab5a950 con 0x55f1eac020e0 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5efe000640 10 client.18135 handle_command_reply: tid=0 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 client.18135 shutdown 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 msgr2=0x55f1eabeef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004090 tx=0x7f5f0c009260 comp rx=0 tx=0).stop 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 shutdown_connections 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eabef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe7760 0x55f1eabef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac020e0 0x55f1eac04550 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/1886534854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe6e40 0x55f1eabeef50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 >> 172.21.15.59:0/1886534854 conn(0x55f1eabe33d0 msgr2=0x55f1eabe90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5efe000640 0 client.18135 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 shutdown_connections 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 1 -- 172.21.15.59:0/1886534854 wait complete. 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:43.992+0000 7f5f21fbb740 20 client.18135 trim_cache size 0 max 16384 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.006+0000 7f5f21fbb740 1 Processor -- start 2024-09-05T12:13:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.006+0000 7f5f21fbb740 1 -- start start 2024-09-05T12:13:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe6e40 0x55f1eabee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eac09e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181aee80 con 0x55f1eabe7760 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1700 con 0x55f1eabe6e40 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181af4f0 con 0x55f1eabf2020 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59934/0 (socket says 172.21.15.59:59934) 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f17e00640 1 -- 172.21.15.59:0/3077826505 learned_addr learned my addr 172.21.15.59:0/3077826505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe6e40 0x55f1eabee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f1d400640 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eac09e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f17e00640 1 -- 172.21.15.59:0/3077826505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 msgr2=0x55f1eac09e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.007+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eac09e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f17e00640 1 -- 172.21.15.59:0/3077826505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe6e40 msgr2=0x55f1eabee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe6e40 0x55f1eabee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f17e00640 1 -- 172.21.15.59:0/3077826505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1eabe7590 con 0x55f1eabe7760 2024-09-05T12:13:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f17e00640 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f040985d0 tx=0x7f5f04004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1142fedfc5caa01c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f09200640 1 -- 172.21.15.59:0/3077826505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0409c300 con 0x55f1eabe7760 2024-09-05T12:13:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f09200640 1 -- 172.21.15.59:0/3077826505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f040b2070 con 0x55f1eabe7760 2024-09-05T12:13:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.008+0000 7f5f09200640 1 -- 172.21.15.59:0/3077826505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04098270 con 0x55f1eabe7760 2024-09-05T12:13:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- 172.21.15.59:0/3077826505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 msgr2=0x55f1eabef9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f040985d0 tx=0x7f5f04004b00 comp rx=0 tx=0).stop 2024-09-05T12:13:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- 172.21.15.59:0/3077826505 shutdown_connections 2024-09-05T12:13:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabf2020 0x55f1eac09e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabef9e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/3077826505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabe6e40 0x55f1eabee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- 172.21.15.59:0/3077826505 >> 172.21.15.59:0/3077826505 conn(0x55f1eabe33d0 msgr2=0x55f1eabfd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- 172.21.15.59:0/3077826505 shutdown_connections 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- 172.21.15.59:0/3077826505 wait complete. 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.009+0000 7f5f21fbb740 1 -- start start 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 Processor -- start 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabf5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 0x55f1eabf6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181afbc0 con 0x55f1eabe7760 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1030 con 0x55f1eabf2020 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b0290 con 0x55f1eabe6e40 2024-09-05T12:13:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f21fbb740 10 client.0 _ll_register_callbacks cb 0x7f5f0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.010+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59950/0 (socket says 172.21.15.59:59950) 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f1d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 0x55f1eabf6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 -- 172.21.15.59:0/842809030 learned_addr learned my addr 172.21.15.59:0/842809030 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabf5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 -- 172.21.15.59:0/842809030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 msgr2=0x55f1eabf5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabf5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 -- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 msgr2=0x55f1eabf6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 0x55f1eabf6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 -- 172.21.15.59:0/842809030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1eac6bff0 con 0x55f1eabe7760 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabf5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f1d400640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 0x55f1eabf6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5f17e00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d010 tx=0x7f5f04009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ce0c59e806d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5efe000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.011+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0409bc00 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5f04098270 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f040b2070 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5f0409ffb0 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 10 client.18141 fetch_fsmap learned FSMap version 21 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f1eac6d270 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f040b2070 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5f040ac030 con 0x55f1eabe7760 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 10 client.18141 fetch_fsmap finished waiting for FSMap version 21 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 20 client.18141 populate_metadata read hostname 'smithi059' 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac6d9b0 0x55f1eac6fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 4 client.18141 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-05T12:13:44.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.012+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 --> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1eab54e60 con 0x55f1eac6d9b0 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac6d9b0 0x55f1eac6fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5f1ca00640 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac6d9b0 0x55f1eac6fdd0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5939d9bc6525c81c server_cookie=ecfa55de9a517308 in_seq=0 out_seq=0 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5efe000640 10 client.18141 ms_handle_connect on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5efe000640 1 -- 172.21.15.59:0/842809030 <== mds.0 v2:172.21.15.89:6838/4050620246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1eab54e60 con 0x55f1eac6d9b0 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5efe000640 10 client.18141 handle_command_reply: tid=0 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.013+0000 7f5f21fbb740 1 client.18141 shutdown 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 msgr2=0x55f1eabf6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d010 tx=0x7f5f04009310 comp rx=0 tx=0).stop 2024-09-05T12:13:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 shutdown_connections 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1eabf2020 0x55f1eabf6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.89:6838/4050620246,v1:172.21.15.89:6839/4050620246] conn(0x55f1eac6d9b0 0x55f1eac6fdd0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f1eabe7760 0x55f1eabf6410 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 --2- 172.21.15.59:0/842809030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1eabe6e40 0x55f1eabf5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 >> 172.21.15.59:0/842809030 conn(0x55f1eabe33d0 msgr2=0x55f1eabfd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5efe000640 0 client.18141 ms_handle_reset on v2:172.21.15.89:6838/4050620246 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 shutdown_connections 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 1 -- 172.21.15.59:0/842809030 wait complete. 2024-09-05T12:13:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.014+0000 7f5f21fbb740 20 client.18141 trim_cache size 0 max 16384 2024-09-05T12:13:44.020 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-09-05T12:13:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 msgr2=0x7f5f000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 0x7f5f000a50c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009230 tx=0x7f5f0c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 msgr2=0x7f5f1819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400e9f0 tx=0x7f5f0400edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 shutdown_connections 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f5f000a2c00 0x7f5f000a50c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f18108f60 0x7f5f1819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f181047e0 0x7f5f1819aa00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 --2- 172.21.15.59:0/3282506731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f18103e30 0x7f5f1819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 >> 172.21.15.59:0/3282506731 conn(0x7f5f180ff8d0 msgr2=0x7f5f18100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 shutdown_connections 2024-09-05T12:13:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.015+0000 7f5f1de00640 1 -- 172.21.15.59:0/3282506731 wait complete. 2024-09-05T12:13:44.737 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-05T12:13:44.737 INFO:tasks.fwd_scrub:done joining 2024-09-05T12:13:44.737 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-05T12:13:44.748 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T12:13:44.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 -- 172.21.15.59:0/2669223749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 msgr2=0x7f89fc076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 --2- 172.21.15.59:0/2669223749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc076fb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009900 tx=0x7f89ec02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 -- 172.21.15.59:0/2669223749 shutdown_connections 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 --2- 172.21.15.59:0/2669223749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 --2- 172.21.15.59:0/2669223749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 --2- 172.21.15.59:0/2669223749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc076fb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 -- 172.21.15.59:0/2669223749 >> 172.21.15.59:0/2669223749 conn(0x7f89fc0ff4b0 msgr2=0x7f89fc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 -- 172.21.15.59:0/2669223749 shutdown_connections 2024-09-05T12:13:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.911+0000 7f8a02000640 1 -- 172.21.15.59:0/2669223749 wait complete. 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 Processor -- start 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 -- start start 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc113e10 con 0x7f89fc077df0 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc113c90 con 0x7f89fc078790 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.912+0000 7f8a02000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc113f90 con 0x7f89fc076bb0 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89fbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59970/0 (socket says 172.21.15.59:59970) 2024-09-05T12:13:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 -- 172.21.15.59:0/4143019467 learned_addr learned my addr 172.21.15.59:0/4143019467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89fb400640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 -- 172.21.15.59:0/4143019467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 msgr2=0x7f89fc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 -- 172.21.15.59:0/4143019467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 msgr2=0x7f89fc1a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc1a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 -- 172.21.15.59:0/4143019467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc1a6030 con 0x7f89fc077df0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89fbe00640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.913+0000 7f89f3e00640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e9f0 tx=0x7f89e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.914+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400cd80 con 0x7f89fc077df0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.914+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e4010910 con 0x7f89fc077df0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.914+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4018990 con 0x7f89fc077df0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.914+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc1a6320 con 0x7f89fc077df0 2024-09-05T12:13:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.914+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a6820 con 0x7f89fc077df0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.915+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc1107c0 con 0x7f89fc077df0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.917+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89e4010ac0 con 0x7f89fc077df0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.918+0000 7f89f8c00640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 0x7f89e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.918+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f89e40cf720 con 0x7f89fc077df0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.918+0000 7f89fb400640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 0x7f89e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.919+0000 7f89fb400640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 0x7f89e00a50d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f89ec031440 tx=0x7f89ec0315b0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:44.922+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89e409bbc0 con 0x7f89fc077df0 2024-09-05T12:13:44.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:44 smithi059 ceph-mon[30025]: pgmap v876: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T12:13:45.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.112+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89fc062e20 con 0x7f89fc077df0 2024-09-05T12:13:45.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.113+0000 7f89f8c00640 1 -- 172.21.15.59:0/4143019467 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7f89e409fec0 con 0x7f89fc077df0 2024-09-05T12:13:45.115 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:45.115 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:13:45.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 msgr2=0x7f89e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 0x7f89e00a50d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f89ec031440 tx=0x7f89ec0315b0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 msgr2=0x7f89fc19e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e9f0 tx=0x7f89e400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 shutdown_connections 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f89e00a2c10 0x7f89e00a50d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89fc078790 0x7f89fc1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89fc077df0 0x7f89fc19e950 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 --2- 172.21.15.59:0/4143019467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89fc076bb0 0x7f89fc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 >> 172.21.15.59:0/4143019467 conn(0x7f89fc0ff4b0 msgr2=0x7f89fc100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 shutdown_connections 2024-09-05T12:13:45.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.116+0000 7f8a02000640 1 -- 172.21.15.59:0/4143019467 wait complete. 2024-09-05T12:13:45.134 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T11:49:37.375375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T11:54:28.033815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T11:54:30.500703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T11:54:37.110249+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T12:13:45.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T12:13:45.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:44 smithi089 ceph-mon[34499]: pgmap v876: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T12:13:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 -- 172.21.15.59:0/1757093104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 msgr2=0x7f8cd8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 --2- 172.21.15.59:0/1757093104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd8104120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009900 tx=0x7f8cc802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 -- 172.21.15.59:0/1757093104 shutdown_connections 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 --2- 172.21.15.59:0/1757093104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 0x7f8cd810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 --2- 172.21.15.59:0/1757093104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd8105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 --2- 172.21.15.59:0/1757093104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd8104120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 -- 172.21.15.59:0/1757093104 >> 172.21.15.59:0/1757093104 conn(0x7f8cd80ff4a0 msgr2=0x7f8cd81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 -- 172.21.15.59:0/1757093104 shutdown_connections 2024-09-05T12:13:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.295+0000 7f8cde400640 1 -- 172.21.15.59:0/1757093104 wait complete. 2024-09-05T12:13:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.296+0000 7f8cde400640 1 Processor -- start 2024-09-05T12:13:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.296+0000 7f8cde400640 1 -- start start 2024-09-05T12:13:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 0x7f8cd81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810f9a0 con 0x7f8cd8104f10 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810f820 con 0x7f8cd8103d20 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cde400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810fb20 con 0x7f8cd81058d0 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59980/0 (socket says 172.21.15.59:59980) 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 -- 172.21.15.59:0/3619269667 learned_addr learned my addr 172.21.15.59:0/3619269667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd7400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd7e00640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 0x7f8cd81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 -- 172.21.15.59:0/3619269667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 msgr2=0x7f8cd81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 0x7f8cd81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 -- 172.21.15.59:0/3619269667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 msgr2=0x7f8cd819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.297+0000 7f8cd6a00640 1 -- 172.21.15.59:0/3619269667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd81a1cd0 con 0x7f8cd8104f10 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8cd6a00640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cd80 con 0x7f8cd8104f10 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4010910 con 0x7f8cd8104f10 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4018990 con 0x7f8cd8104f10 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd81a1fc0 con 0x7f8cd8104f10 2024-09-05T12:13:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.298+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd81a2430 con 0x7f8cd8104f10 2024-09-05T12:13:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.299+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd810b1b0 con 0x7f8cd8104f10 2024-09-05T12:13:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.301+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8cc4010ac0 con 0x7f8cd8104f10 2024-09-05T12:13:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.302+0000 7f8ccfe00640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.303+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8cc40cf730 con 0x7f8cd8104f10 2024-09-05T12:13:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.303+0000 7f8cd7400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.304+0000 7f8cd7400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8005ab0 tx=0x7f8cc8000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.307+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cc409bbd0 con 0x7f8cd8104f10 2024-09-05T12:13:45.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:44 smithi189 ceph-mon[34772]: pgmap v876: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-09-05T12:13:45.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.492+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cd8062e90 con 0x7f8cd8104f10 2024-09-05T12:13:45.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.492+0000 7f8ccfe00640 1 -- 172.21.15.59:0/3619269667 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f8cc409fed0 con 0x7f8cd8104f10 2024-09-05T12:13:45.493 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 msgr2=0x7f8cbc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8005ab0 tx=0x7f8cc8000bc0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 msgr2=0x7f8cd819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e9f0 tx=0x7f8cc400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 shutdown_connections 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd81058d0 0x7f8cd81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cd8104f10 0x7f8cd819a5f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 --2- 172.21.15.59:0/3619269667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cd8103d20 0x7f8cd819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.494+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 >> 172.21.15.59:0/3619269667 conn(0x7f8cd80ff4a0 msgr2=0x7f8cd8108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.495+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 shutdown_connections 2024-09-05T12:13:45.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.495+0000 7f8cde400640 1 -- 172.21.15.59:0/3619269667 wait complete. 2024-09-05T12:13:45.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T12:13:45.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 -- 172.21.15.59:0/1503280545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 msgr2=0x7fa590104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 --2- 172.21.15.59:0/1503280545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590104870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa584009860 tx=0x7fa58402a340 comp rx=0 tx=0).stop 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 -- 172.21.15.59:0/1503280545 shutdown_connections 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 --2- 172.21.15.59:0/1503280545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 0x7fa590109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 --2- 172.21.15.59:0/1503280545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa5901071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 --2- 172.21.15.59:0/1503280545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590104870 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 -- 172.21.15.59:0/1503280545 >> 172.21.15.59:0/1503280545 conn(0x7fa5900fc0a0 msgr2=0x7fa5900fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 -- 172.21.15.59:0/1503280545 shutdown_connections 2024-09-05T12:13:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.672+0000 7fa596e00640 1 -- 172.21.15.59:0/1503280545 wait complete. 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.673+0000 7fa596e00640 1 Processor -- start 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.673+0000 7fa596e00640 1 -- start start 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.673+0000 7fa596e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.673+0000 7fa596e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.673+0000 7fa596e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 0x7fa590100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa596e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010d6f0 con 0x7fa590104db0 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa596e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010d570 con 0x7fa590102480 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa596e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa59010d870 con 0x7fa5901076e0 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60008/0 (socket says 172.21.15.59:60008) 2024-09-05T12:13:45.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595000640 1 -- 172.21.15.59:0/4291095234 learned_addr learned my addr 172.21.15.59:0/4291095234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa596400640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 0x7fa590100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 -- 172.21.15.59:0/4291095234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 msgr2=0x7fa590100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 0x7fa590100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 -- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 msgr2=0x7fa590100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595a00640 1 -- 172.21.15.59:0/4291095234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa590101200 con 0x7fa590102480 2024-09-05T12:13:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.674+0000 7fa595000640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa595a00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa584035040 tx=0x7fa584033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584037040 con 0x7fa590102480 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa584032770 con 0x7fa590102480 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584005520 con 0x7fa590102480 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5901a1e40 con 0x7fa590102480 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.675+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5901a2250 con 0x7fa590102480 2024-09-05T12:13:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.676+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5900fdb90 con 0x7fa590102480 2024-09-05T12:13:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.678+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa58406bd10 con 0x7fa590102480 2024-09-05T12:13:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.679+0000 7fa58e000640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 0x7fa5780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.680+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa5840eeec0 con 0x7fa590102480 2024-09-05T12:13:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.680+0000 7fa595000640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 0x7fa5780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:45.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.680+0000 7fa595000640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 0x7fa5780a5050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa590101b60 tx=0x7fa57c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.682+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa584049020 con 0x7fa590102480 2024-09-05T12:13:45.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.867+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa59002d950 con 0x7fa590102480 2024-09-05T12:13:45.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.868+0000 7fa58e000640 1 -- 172.21.15.59:0/4291095234 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fa5840b5e20 con 0x7fa590102480 2024-09-05T12:13:45.869 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-09-05T12:13:45.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 msgr2=0x7fa5780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 0x7fa5780a5050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa590101b60 tx=0x7fa57c009260 comp rx=0 tx=0).stop 2024-09-05T12:13:45.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 msgr2=0x7fa590101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa584035040 tx=0x7fa584033d90 comp rx=0 tx=0).stop 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 shutdown_connections 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fa5780a2b90 0x7fa5780a5050 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5901076e0 0x7fa590100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa590104db0 0x7fa590100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 --2- 172.21.15.59:0/4291095234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa590102480 0x7fa590101df0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 >> 172.21.15.59:0/4291095234 conn(0x7fa5900fc0a0 msgr2=0x7fa590102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 shutdown_connections 2024-09-05T12:13:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:45.870+0000 7fa596e00640 1 -- 172.21.15.59:0/4291095234 wait complete. 2024-09-05T12:13:45.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T12:13:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 -- 172.21.15.59:0/744355415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8104e10 msgr2=0x7f0dc8105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/744355415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8104e10 0x7f0dc8105290 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009900 tx=0x7f0db402a410 comp rx=0 tx=0).stop 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 -- 172.21.15.59:0/744355415 shutdown_connections 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/744355415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc81057d0 0x7f0dc810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/744355415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8104e10 0x7f0dc8105290 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/744355415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc8103c20 0x7f0dc8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 -- 172.21.15.59:0/744355415 >> 172.21.15.59:0/744355415 conn(0x7f0dc80ff3a0 msgr2=0x7f0dc81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 -- 172.21.15.59:0/744355415 shutdown_connections 2024-09-05T12:13:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.049+0000 7f0dc7e00640 1 -- 172.21.15.59:0/744355415 wait complete. 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 Processor -- start 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 -- start start 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 0x7f0dc819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc81057d0 0x7f0dc81a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810f880 con 0x7f0dc8103c20 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810f700 con 0x7f0dc8104e10 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.050+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810fa00 con 0x7f0dc81057d0 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60038/0 (socket says 172.21.15.59:60038) 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 -- 172.21.15.59:0/2959758119 learned_addr learned my addr 172.21.15.59:0/2959758119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6000640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 0x7f0dc819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc7400640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc81057d0 0x7f0dc81a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 -- 172.21.15.59:0/2959758119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc81057d0 msgr2=0x7f0dc81a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc81057d0 0x7f0dc81a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 -- 172.21.15.59:0/2959758119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 msgr2=0x7f0dc819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 0x7f0dc819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 -- 172.21.15.59:0/2959758119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc81a1b50 con 0x7f0dc8103c20 2024-09-05T12:13:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6000640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 0x7f0dc819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.051+0000 7f0dc6a00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d950 tx=0x7f0db800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.052+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8017020 con 0x7f0dc8103c20 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.052+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db8009c30 con 0x7f0dc8103c20 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.052+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db800f720 con 0x7f0dc8103c20 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.052+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc81a1e40 con 0x7f0dc8103c20 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.052+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a2250 con 0x7f0dc8103c20 2024-09-05T12:13:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.053+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc810b010 con 0x7f0dc8103c20 2024-09-05T12:13:46.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.055+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0db801a020 con 0x7f0dc8103c20 2024-09-05T12:13:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.056+0000 7f0dbf400640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.056+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0db80cf2f0 con 0x7f0dc8103c20 2024-09-05T12:13:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.056+0000 7f0dc6000640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.057+0000 7f0dc6000640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0db40314e0 tx=0x7f0db4000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.060+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0db8021020 con 0x7f0dc8103c20 2024-09-05T12:13:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4143019467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3619269667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:45 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4291095234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4143019467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:46.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3619269667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:45 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4291095234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.248+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0dc8062e90 con 0x7f0dc8103c20 2024-09-05T12:13:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.249+0000 7f0dbf400640 1 -- 172.21.15.59:0/2959758119 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f0db8009de0 con 0x7f0dc8103c20 2024-09-05T12:13:46.250 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 msgr2=0x7f0dac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0db40314e0 tx=0x7f0db4000bc0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 msgr2=0x7f0dc8199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d950 tx=0x7f0db800de10 comp rx=0 tx=0).stop 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 shutdown_connections 2024-09-05T12:13:46.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc81057d0 0x7f0dc81a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dc8104e10 0x7f0dc819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 --2- 172.21.15.59:0/2959758119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc8103c20 0x7f0dc8199f30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 >> 172.21.15.59:0/2959758119 conn(0x7f0dc80ff3a0 msgr2=0x7f0dc81081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 shutdown_connections 2024-09-05T12:13:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.251+0000 7f0dc7e00640 1 -- 172.21.15.59:0/2959758119 wait complete. 2024-09-05T12:13:46.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T12:13:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4143019467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3619269667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:45 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4291095234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 -- 172.21.15.59:0/2091711519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 msgr2=0x7ff234104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 --2- 172.21.15.59:0/2091711519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff234104120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff224009860 tx=0x7ff22402a340 comp rx=0 tx=0).stop 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 -- 172.21.15.59:0/2091711519 shutdown_connections 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 --2- 172.21.15.59:0/2091711519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 0x7ff23410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 --2- 172.21.15.59:0/2091711519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff234105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 --2- 172.21.15.59:0/2091711519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff234104120 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 -- 172.21.15.59:0/2091711519 >> 172.21.15.59:0/2091711519 conn(0x7ff2340ff4a0 msgr2=0x7ff2341018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 -- 172.21.15.59:0/2091711519 shutdown_connections 2024-09-05T12:13:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.429+0000 7ff23ae00640 1 -- 172.21.15.59:0/2091711519 wait complete. 2024-09-05T12:13:46.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.430+0000 7ff23ae00640 1 Processor -- start 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.430+0000 7ff23ae00640 1 -- start start 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff23419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 0x7ff2341a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff23410f9a0 con 0x7ff234104f10 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff23410f820 con 0x7ff234103d20 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff23ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff23410fb20 con 0x7ff2341058d0 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff22be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff23419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46546/0 (socket says 172.21.15.59:46546) 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 -- 172.21.15.59:0/971971437 learned_addr learned my addr 172.21.15.59:0/971971437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff238e00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 0x7ff2341a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 -- 172.21.15.59:0/971971437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 msgr2=0x7ff2341a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 0x7ff2341a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 -- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 msgr2=0x7ff23419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff23419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff233e00640 1 -- 172.21.15.59:0/971971437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2341a1cd0 con 0x7ff234103d20 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.431+0000 7ff22be00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff23419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff233e00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff224035040 tx=0x7ff224033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224037040 con 0x7ff234103d20 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2341a1f60 con 0x7ff234103d20 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff224030850 con 0x7ff234103d20 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224004920 con 0x7ff234103d20 2024-09-05T12:13:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.432+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2341a23c0 con 0x7ff234103d20 2024-09-05T12:13:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.433+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23410b1b0 con 0x7ff234103d20 2024-09-05T12:13:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.435+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff224004050 con 0x7ff234103d20 2024-09-05T12:13:46.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.436+0000 7ff231600640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 0x7ff2180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.437+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff2240ef5d0 con 0x7ff234103d20 2024-09-05T12:13:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.437+0000 7ff22be00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 0x7ff2180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.437+0000 7ff22be00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 0x7ff2180a5050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0096c0 tx=0x7ff21c009260 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.439+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2240bc0c0 con 0x7ff234103d20 2024-09-05T12:13:46.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.623+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff23402d9c0 con 0x7ff234103d20 2024-09-05T12:13:46.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.623+0000 7ff231600640 1 -- 172.21.15.59:0/971971437 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7ff2240bb9c0 con 0x7ff234103d20 2024-09-05T12:13:46.625 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-09-05T12:13:46.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 msgr2=0x7ff2180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 0x7ff2180a5050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0096c0 tx=0x7ff21c009260 comp rx=0 tx=0).stop 2024-09-05T12:13:46.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 msgr2=0x7ff23419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff224035040 tx=0x7ff224033d90 comp rx=0 tx=0).stop 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 shutdown_connections 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7ff2180a2b90 0x7ff2180a5050 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2341058d0 0x7ff2341a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff234104f10 0x7ff23419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 --2- 172.21.15.59:0/971971437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff234103d20 0x7ff23419a0b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 >> 172.21.15.59:0/971971437 conn(0x7ff2340ff4a0 msgr2=0x7ff234108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 shutdown_connections 2024-09-05T12:13:46.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.626+0000 7ff23ae00640 1 -- 172.21.15.59:0/971971437 wait complete. 2024-09-05T12:13:46.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-05T12:13:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 -- 172.21.15.59:0/2624796872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c103d30 msgr2=0x7fe39c104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 --2- 172.21.15.59:0/2624796872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c103d30 0x7fe39c104130 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 -- 172.21.15.59:0/2624796872 shutdown_connections 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 --2- 172.21.15.59:0/2624796872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c1058e0 0x7fe39c10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 --2- 172.21.15.59:0/2624796872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c104f20 0x7fe39c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 --2- 172.21.15.59:0/2624796872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c103d30 0x7fe39c104130 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 -- 172.21.15.59:0/2624796872 >> 172.21.15.59:0/2624796872 conn(0x7fe39c0ff4b0 msgr2=0x7fe39c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 -- 172.21.15.59:0/2624796872 shutdown_connections 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.800+0000 7fe3a3400640 1 -- 172.21.15.59:0/2624796872 wait complete. 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 Processor -- start 2024-09-05T12:13:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 -- start start 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c103d30 0x7fe39c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c1058e0 0x7fe39c1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10f990 con 0x7fe39c1058e0 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10f810 con 0x7fe39c104f20 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.801+0000 7fe3a3400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10fb10 con 0x7fe39c103d30 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe3a0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c103d30 0x7fe39c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 -- 172.21.15.59:0/256050525 learned_addr learned my addr 172.21.15.59:0/256050525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe3a1400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c1058e0 0x7fe39c1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 -- 172.21.15.59:0/256050525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c103d30 msgr2=0x7fe39c19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c103d30 0x7fe39c19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 -- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c1058e0 msgr2=0x7fe39c1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c1058e0 0x7fe39c1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 -- 172.21.15.59:0/256050525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a1c60 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe39be00640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe38800e9f0 tx=0x7fe38800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.802+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38800ccc0 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.803+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388010910 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.803+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a1f50 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.803+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3880188a0 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.803+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a2410 con 0x7fe39c104f20 2024-09-05T12:13:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.804+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c10b120 con 0x7fe39c104f20 2024-09-05T12:13:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.806+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe388010ac0 con 0x7fe39c104f20 2024-09-05T12:13:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.808+0000 7fe399600640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 0x7fe3840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.808+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe3880cfdb0 con 0x7fe39c104f20 2024-09-05T12:13:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.808+0000 7fe3a0a00640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 0x7fe3840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.809+0000 7fe3a0a00640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 0x7fe3840a5050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe390005ab0 tx=0x7fe390000bc0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:46.809+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe38809c8a0 con 0x7fe39c104f20 2024-09-05T12:13:47.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.047+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe39c062e90 con 0x7fe39c104f20 2024-09-05T12:13:47.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.056+0000 7fe399600640 1 -- 172.21.15.59:0/256050525 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740365 (secure 0 0 0) 0x7fe38809c1a0 con 0x7fe39c104f20 2024-09-05T12:13:47.068 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 18, 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "active_gid": 14146, 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "active_name": "x", 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addrs": { 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 23374249 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 23374249 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addr": "172.21.15.59:6801/23374249", 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "active_change": "2024-09-05T11:47:16.761249+0000", 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "active_mgr_features": 4540701547738038271, 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "available": true, 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: "standbys": [ 2024-09-05T12:13:47.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 24109, 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "z", 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.072 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.073 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.074 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.075 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.077 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.082 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.084 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.086 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-09-05T12:13:47.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.090 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:47.091 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-09-05T12:13:47.092 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.093 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-09-05T12:13:47.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.098 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.099 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.104 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-09-05T12:13:47.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.106 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.108 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:13:47.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-09-05T12:13:47.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-09-05T12:13:47.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.112 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.113 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:13:47.114 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.115 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.116 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.117 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-09-05T12:13:47.118 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.119 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-09-05T12:13:47.124 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.125 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:13:47.126 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-09-05T12:13:47.130 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-09-05T12:13:47.131 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:13:47.137 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:13:47.138 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.139 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-09-05T12:13:47.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-09-05T12:13:47.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.145 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-09-05T12:13:47.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.151 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.152 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.154 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.163 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.165 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.166 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:13:47.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.181 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.183 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.188 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.190 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:13:47.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.193 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.198 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.199 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.203 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.206 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.207 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.209 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.211 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.212 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.215 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:13:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:13:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:13:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.222 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.223 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-09-05T12:13:47.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-09-05T12:13:47.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.235 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:47.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:47.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.237 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.249 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:47.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.256 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.259 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-09-05T12:13:47.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-09-05T12:13:47.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-09-05T12:13:47.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-09-05T12:13:47.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.269 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-09-05T12:13:47.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-09-05T12:13:47.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-09-05T12:13:47.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-09-05T12:13:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:13:47.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.274 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-09-05T12:13:47.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-09-05T12:13:47.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.277 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.278 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.278 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-09-05T12:13:47.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-09-05T12:13:47.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-09-05T12:13:47.280 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.281 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:13:47.281 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-09-05T12:13:47.283 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.284 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.284 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.284 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.284 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.286 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.286 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.287 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.287 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.287 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.287 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.287 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.289 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.291 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.291 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.293 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.294 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-09-05T12:13:47.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.298 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.300 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-09-05T12:13:47.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-09-05T12:13:47.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-09-05T12:13:47.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.302 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.303 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.304 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.304 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.305 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.305 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.305 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-09-05T12:13:47.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-09-05T12:13:47.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-09-05T12:13:47.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.309 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.310 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-09-05T12:13:47.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.313 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.319 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.320 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.323 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-09-05T12:13:47.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-09-05T12:13:47.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.326 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.327 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.332 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.334 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.335 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.337 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.339 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:13:47.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.342 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-09-05T12:13:47.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.346 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.348 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.348 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.348 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.348 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.351 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.356 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-09-05T12:13:47.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:13:47.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-09-05T12:13:47.361 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-09-05T12:13:47.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-09-05T12:13:47.364 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.365 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.366 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.368 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.371 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.372 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.373 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.374 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.376 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.380 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.382 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.387 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.389 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-09-05T12:13:47.390 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.391 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.392 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.394 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.395 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.396 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.397 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.398 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.399 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.400 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.401 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.404 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.405 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.406 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.409 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.413 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.416 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.417 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.421 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.424 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-09-05T12:13:47.425 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:13:47.427 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.428 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.431 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-09-05T12:13:47.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.434 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.439 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-09-05T12:13:47.440 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:47.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.444 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.445 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.447 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.449 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.449 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.449 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.449 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.449 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:13:47.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:13:47.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.452 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.454 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.455 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.457 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.458 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.459 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.460 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.461 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.462 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.466 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.467 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.472 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.477 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.479 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.484 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.485 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.488 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.492 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.493 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.495 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.496 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.498 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-09-05T12:13:47.500 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-09-05T12:13:47.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.507 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.508 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.509 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.510 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.511 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.512 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.514 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.517 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.518 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-09-05T12:13:47.519 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.520 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.520 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.520 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.521 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.522 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.524 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.527 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.528 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.529 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.530 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.533 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.537 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.538 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-09-05T12:13:47.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-09-05T12:13:47.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.543 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.545 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.546 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.548 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-09-05T12:13:47.549 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.550 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-09-05T12:13:47.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.554 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.557 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.560 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-09-05T12:13:47.563 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.566 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.567 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.570 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.573 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.574 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-09-05T12:13:47.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:13:47.579 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-09-05T12:13:47.580 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.584 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.585 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.586 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.587 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.588 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.589 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.590 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.591 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.592 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 24113, 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "y", 2024-09-05T12:13:47.593 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.594 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.596 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.597 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.599 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.600 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.601 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-09-05T12:13:47.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.603 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-09-05T12:13:47.604 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.605 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.609 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.609 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.609 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.610 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:13:47.610 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.610 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.611 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.612 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.612 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.612 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.612 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:13:47.612 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.613 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.613 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-09-05T12:13:47.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-09-05T12:13:47.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.615 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-09-05T12:13:47.615 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.615 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-09-05T12:13:47.617 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.617 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.617 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.617 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-09-05T12:13:47.619 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.620 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.621 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:13:47.621 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:47.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.621 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:13:47.622 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:13:47.623 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.624 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.625 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.626 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.629 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.630 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.631 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.632 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.633 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-09-05T12:13:47.635 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.636 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.638 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.640 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.640 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.640 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.640 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.640 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.642 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.643 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:13:47.644 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.644 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.644 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.645 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.646 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-09-05T12:13:47.647 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.648 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.649 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.650 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.651 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:13:47.651 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:13:47.651 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.651 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.652 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.653 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.654 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.655 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.657 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-09-05T12:13:47.658 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.659 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:13:47.660 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.662 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.663 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.664 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.665 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-09-05T12:13:47.666 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.667 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-09-05T12:13:47.668 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.669 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:13:47.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-09-05T12:13:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.673 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-09-05T12:13:47.674 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-09-05T12:13:47.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.677 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-09-05T12:13:47.678 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.679 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:13:47.680 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.682 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.683 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.684 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.685 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.687 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.688 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.689 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:13:47.691 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.692 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.693 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.694 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.696 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.697 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.699 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.700 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:13:47.701 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.703 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:13:47.705 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.706 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.708 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.710 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.711 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:13:47.712 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.714 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.715 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-09-05T12:13:47.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.717 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.718 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.719 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.720 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.723 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:13:47.724 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.725 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:13:47.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.728 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.729 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.731 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.732 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.733 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.734 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.735 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.736 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.736 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.736 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.736 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.736 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-09-05T12:13:47.737 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.738 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-09-05T12:13:47.739 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.740 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.741 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:13:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:13:47.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.747 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.750 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.751 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.753 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:13:47.753 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:13:47.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.753 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:13:47.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:13:47.755 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:13:47.756 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.757 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.758 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.761 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.762 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.763 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.764 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.765 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.766 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.767 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.769 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.770 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.771 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.773 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.774 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.775 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.776 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.777 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.778 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.781 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:13:47.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:13:47.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.784 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.786 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.787 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.792 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.793 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.796 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.797 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-09-05T12:13:47.798 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.800 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.802 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-09-05T12:13:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:13:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.818 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.819 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.820 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.821 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.823 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.824 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.825 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.826 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.828 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-09-05T12:13:47.832 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-09-05T12:13:47.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.834 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:13:47.836 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.837 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.840 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.841 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.842 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.843 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:47.844 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.846 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.847 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.849 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.850 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.852 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.853 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.854 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.855 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.856 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-09-05T12:13:47.857 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.858 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.859 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.860 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.861 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.862 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.862 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.862 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.863 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.864 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.864 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.864 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.865 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.865 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.866 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.867 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.867 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.867 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.867 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.867 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.869 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.869 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.869 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.869 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.870 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.871 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.872 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.872 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-09-05T12:13:47.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:13:47.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.872 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.873 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-09-05T12:13:47.874 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-09-05T12:13:47.875 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.876 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.876 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:13:47.876 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.876 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.876 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.877 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.877 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.877 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.878 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.878 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.878 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.878 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.879 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.880 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.880 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.880 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.880 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.880 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.881 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.881 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.881 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.881 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.882 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.882 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.882 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.882 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.884 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.884 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.884 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.885 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.886 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.887 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:13:47.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.890 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.892 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.893 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:13:47.895 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.896 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.897 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-09-05T12:13:47.898 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:13:47.900 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.902 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.906 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.907 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.909 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.910 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.911 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.912 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.913 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.916 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.918 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.919 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.920 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.922 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.922 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.922 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.922 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.922 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.924 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:47.924 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.924 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.924 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.924 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.925 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.926 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.927 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.928 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.928 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.928 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.928 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.928 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.929 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.930 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.931 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.932 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.933 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-09-05T12:13:47.934 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.935 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.936 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.937 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.938 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.939 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.940 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.941 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-09-05T12:13:47.942 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.943 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.944 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.945 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.946 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.948 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.949 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.950 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.952 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.954 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.956 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.957 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.958 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.959 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-09-05T12:13:47.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.962 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.963 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.965 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.966 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.967 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.968 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.970 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.971 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.972 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.973 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.974 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.975 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.976 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.977 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.978 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.979 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.980 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.981 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-09-05T12:13:47.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:47.983 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.984 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.987 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.988 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.989 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:47.990 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.991 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.993 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:47.994 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.997 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:47.998 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:47.999 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-09-05T12:13:48.000 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.001 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.002 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.003 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.004 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.005 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.006 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.007 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.008 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.009 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.010 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.012 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.013 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.014 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.015 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.016 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.017 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.018 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.019 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.021 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-09-05T12:13:48.023 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.025 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.026 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.028 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.029 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-09-05T12:13:48.030 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-09-05T12:13:48.031 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-09-05T12:13:48.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-09-05T12:13:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-09-05T12:13:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.035 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.035 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.035 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.035 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.036 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-09-05T12:13:48.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-09-05T12:13:48.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-09-05T12:13:48.040 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.043 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.044 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.048 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.049 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.050 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.051 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.053 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.054 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.056 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.057 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.058 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.059 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.060 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.062 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.063 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.064 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.066 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.067 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.068 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.070 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.071 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.073 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:13:48.078 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-09-05T12:13:48.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:13:48.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.081 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-09-05T12:13:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-09-05T12:13:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-09-05T12:13:48.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-09-05T12:13:48.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.087 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.089 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.089 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-09-05T12:13:48.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.093 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-09-05T12:13:48.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.096 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-09-05T12:13:48.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.098 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.099 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.101 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.108 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.110 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.111 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.114 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.115 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.116 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.117 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-09-05T12:13:48.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.128 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.129 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.130 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.131 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.132 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.134 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.135 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "modules": [ 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard", 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "iostat", 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "nfs", 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "restful" 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.139 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.140 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.141 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-09-05T12:13:48.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.151 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-09-05T12:13:48.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:48.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:13:48.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.157 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-09-05T12:13:48.158 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.163 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.164 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-09-05T12:13:48.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:48.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-09-05T12:13:48.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:13:48.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:13:48.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.173 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:13:48.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.180 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-09-05T12:13:48.181 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.182 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.183 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:13:48.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:13:48.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.195 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.196 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:13:48.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:13:48.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:13:48.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.201 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:13:48.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.206 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:13:48.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.210 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.212 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:13:48.223 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.224 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.225 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-09-05T12:13:48.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.228 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:13:48.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.232 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.234 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-09-05T12:13:48.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.238 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.238 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:13:48.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.241 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.246 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.247 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.248 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-09-05T12:13:48.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.252 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:13:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:13:48.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.260 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.262 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.264 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:13:48.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.266 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:13:48.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:13:48.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.280 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.281 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.282 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:13:48.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.284 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.286 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.289 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.290 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:13:48.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:13:48.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.297 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.298 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-09-05T12:13:48.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-09-05T12:13:48.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.304 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.307 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.308 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.309 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.310 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-09-05T12:13:48.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:13:48.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-09-05T12:13:48.316 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.318 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:48.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.321 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.325 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.327 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.328 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-09-05T12:13:48.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-09-05T12:13:48.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:13:48.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.335 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-09-05T12:13:48.338 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.339 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.342 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.344 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-09-05T12:13:48.348 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.352 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.358 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:13:48.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-09-05T12:13:48.361 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-09-05T12:13:48.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.364 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.367 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.370 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.373 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.376 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.378 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.381 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.383 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-09-05T12:13:48.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.389 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.391 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.392 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.394 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-09-05T12:13:48.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-09-05T12:13:48.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.397 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.398 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.399 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.401 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.403 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.404 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.405 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.407 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.408 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.413 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.415 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.416 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.417 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-09-05T12:13:48.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.420 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.423 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.424 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.426 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.428 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.431 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.432 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.436 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.444 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:48.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-09-05T12:13:48.447 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.450 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.451 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.452 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.454 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.458 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.459 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.460 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.461 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.462 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.463 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.465 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.466 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.467 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.468 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.469 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:48.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:48.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.474 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-09-05T12:13:48.475 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-09-05T12:13:48.476 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-09-05T12:13:48.477 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.478 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.480 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.481 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:13:48.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.485 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.487 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-09-05T12:13:48.488 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.489 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.490 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.493 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.499 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.501 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.502 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.504 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.505 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.506 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.508 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.509 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.510 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.512 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.513 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.514 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.515 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-09-05T12:13:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.520 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.521 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-09-05T12:13:48.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-09-05T12:13:48.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.527 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.530 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.531 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.533 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.534 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.535 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.537 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.543 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.544 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.547 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.548 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.549 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.551 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.552 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.554 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.555 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.558 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.563 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-09-05T12:13:48.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.565 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.567 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.569 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-09-05T12:13:48.571 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.572 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.573 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.574 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.575 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.577 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.580 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.581 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.583 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.585 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-09-05T12:13:48.586 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.587 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.588 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.589 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.590 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.591 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.592 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.593 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.594 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.595 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.597 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.599 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.600 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:13:48.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.603 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.604 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-09-05T12:13:48.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.608 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.611 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.612 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.615 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.616 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-09-05T12:13:48.617 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-09-05T12:13:48.618 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: "services": { 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard": "https://172.21.15.59:8443/" 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: "always_on_modules": { 2024-09-05T12:13:48.619 INFO:teuthology.orchestra.run.smithi059.stdout: "octopus": [ 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-09-05T12:13:48.620 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "pacific": [ 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-09-05T12:13:48.621 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "quincy": [ 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-09-05T12:13:48.622 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "reef": [ 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-09-05T12:13:48.623 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "squid": [ 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-09-05T12:13:48.624 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "last_failure_osd_epoch": 3, 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: "active_clients": [ 2024-09-05T12:13:48.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3486746151 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "libcephsqlite", 2024-09-05T12:13:48.626 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1760103660 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-09-05T12:13:48.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3884315621 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-09-05T12:13:48.628 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3791005244 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-05T12:13:48.629 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:46 smithi189 ceph-mon[34772]: pgmap v877: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2959758119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:46 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/971971437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:47 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/256050525' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:46 smithi089 ceph-mon[34499]: pgmap v877: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-09-05T12:13:48.631 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2959758119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.632 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:46 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/971971437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.632 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:47 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/256050525' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:13:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 msgr2=0x7fe3840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 0x7fe3840a5050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe390005ab0 tx=0x7fe390000bc0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 msgr2=0x7fe39c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe38800e9f0 tx=0x7fe38800edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 shutdown_connections 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe3840a2b90 0x7fe3840a5050 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c1058e0 0x7fe39c1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c104f20 0x7fe39c19a580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 --2- 172.21.15.59:0/256050525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe39c103d30 0x7fe39c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 >> 172.21.15.59:0/256050525 conn(0x7fe39c0ff4b0 msgr2=0x7fe39c1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 shutdown_connections 2024-09-05T12:13:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:47.063+0000 7fe3a3400640 1 -- 172.21.15.59:0/256050525 wait complete. 2024-09-05T12:13:48.692 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-05T12:13:48.709 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-05T12:13:48.710 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T12:13:48.710 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:13:48.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:46 smithi059 ceph-mon[30025]: pgmap v877: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-09-05T12:13:48.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2959758119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:46 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/971971437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:13:48.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:47 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/256050525' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:13:48.751 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-05T12:13:48.751 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:13:48.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T12:13:49.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:48 smithi089 ceph-mon[34499]: pgmap v878: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-05T12:13:49.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:48 smithi059 ceph-mon[30025]: pgmap v878: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-05T12:13:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:49.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:48 smithi189 ceph-mon[34772]: pgmap v878: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-05T12:13:49.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2862180230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4109120 msgr2=0x7fe9c4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:49.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2862180230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4109120 0x7fe9c4110ca0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009ca0 tx=0x7fe9bc02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2862180230 shutdown_connections 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2862180230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4109120 0x7fe9c4110ca0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2862180230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c41049a0 0x7fe9c41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2862180230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c4103ff0 0x7fe9c41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2862180230 >> 172.21.15.59:0/2862180230 conn(0x7fe9c40ffa90 msgr2=0x7fe9c4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2862180230 shutdown_connections 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.825+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2862180230 wait complete. 2024-09-05T12:13:49.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.826+0000 7fe9cbe00640 1 Processor -- start 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 -- start start 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 0x7fe9c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 0x7fe9c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c4109120 0x7fe9c419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4114150 con 0x7fe9c4103ff0 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113fd0 con 0x7fe9c4109120 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9cbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41142d0 con 0x7fe9c41049a0 2024-09-05T12:13:49.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 0x7fe9c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 0x7fe9c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60098/0 (socket says 172.21.15.59:60098) 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9c8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 0x7fe9c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9c9400640 1 -- 172.21.15.59:0/2916698684 learned_addr learned my addr 172.21.15.59:0/2916698684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.827+0000 7fe9c9e00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c4109120 0x7fe9c419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 -- 172.21.15.59:0/2916698684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c4109120 msgr2=0x7fe9c419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c4109120 0x7fe9c419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 -- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 msgr2=0x7fe9c419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 0x7fe9c419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 -- 172.21.15.59:0/2916698684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c419f5e0 con 0x7fe9c41049a0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c8a00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 0x7fe9c419ab70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e9f0 tx=0x7fe9b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000ccc0 con 0x7fe9c41049a0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b0010910 con 0x7fe9c41049a0 2024-09-05T12:13:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.828+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c419f8d0 con 0x7fe9c41049a0 2024-09-05T12:13:49.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.829+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0018960 con 0x7fe9c41049a0 2024-09-05T12:13:49.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.829+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c41a71b0 con 0x7fe9c41049a0 2024-09-05T12:13:49.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.830+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c41043d0 con 0x7fe9c41049a0 2024-09-05T12:13:49.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.832+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9b0010ac0 con 0x7fe9c41049a0 2024-09-05T12:13:49.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.833+0000 7fe9c2000640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 0x7fe9ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:49.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.833+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe9b00cfbc0 con 0x7fe9c41049a0 2024-09-05T12:13:49.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.833+0000 7fe9c9400640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 0x7fe9ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:49.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.834+0000 7fe9c9400640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 0x7fe9ac0a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009cd0 tx=0x7fe9b8005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:49.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:49.835+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9b009c060 con 0x7fe9c41049a0 2024-09-05T12:13:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.033+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9c419b990 con 0x7fe9c41049a0 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.033+0000 7fe9c2000640 1 -- 172.21.15.59:0/2916698684 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+592 (secure 0 0 0) 0x7fe9c419b990 con 0x7fe9c41049a0 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3566679520 2024-09-06T11:47:16.761138+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1131370065 2024-09-06T11:47:16.761138+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1895165557 2024-09-06T11:47:16.761138+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2307158254 2024-09-06T11:46:49.566628+0000 2024-09-05T12:13:50.035 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3602565252 2024-09-06T11:46:49.566628+0000 2024-09-05T12:13:50.036 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1380942177 2024-09-06T11:46:49.566628+0000 2024-09-05T12:13:50.036 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/2057764847 2024-09-06T11:47:16.761138+0000 2024-09-05T12:13:50.036 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1592551985 2024-09-06T11:46:49.566628+0000 2024-09-05T12:13:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-09-05T12:13:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 msgr2=0x7fe9ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 0x7fe9ac0a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009cd0 tx=0x7fe9b8005f90 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 msgr2=0x7fe9c419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 0x7fe9c419ab70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e9f0 tx=0x7fe9b000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 shutdown_connections 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fe9ac0a2c00 0x7fe9ac0a50c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c4109120 0x7fe9c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9c41049a0 0x7fe9c419ab70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 --2- 172.21.15.59:0/2916698684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c4103ff0 0x7fe9c419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 >> 172.21.15.59:0/2916698684 conn(0x7fe9c40ffa90 msgr2=0x7fe9c4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 shutdown_connections 2024-09-05T12:13:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:50.036+0000 7fe9cbe00640 1 -- 172.21.15.59:0/2916698684 wait complete. 2024-09-05T12:13:50.863 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-05T12:13:50.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:13:50.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-05T12:13:50.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T12:13:50.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:13:50.962 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-05T12:13:51.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:13:51.003 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-09-05T12:13:51.080 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-05T12:13:51.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:13:51.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T12:13:51.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:50 smithi059 ceph-mon[30025]: pgmap v879: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:13:51.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:50 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2916698684' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:13:51.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:13:51.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-09-05T12:13:51.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:50 smithi089 ceph-mon[34499]: pgmap v879: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:13:51.184 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:50 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2916698684' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:13:51.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:50 smithi189 ceph-mon[34772]: pgmap v879: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-05T12:13:51.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:50 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2916698684' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:13:51.497 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-05T12:13:51.507 INFO:tasks.cephadm:Teardown begin 2024-09-05T12:13:51.508 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-05T12:13:51.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:51.890 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.494+0000 7f1dd6000640 1 -- 172.21.15.59:0/3081548337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd0108f60 msgr2=0x7f1dd0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.494+0000 7f1dd6000640 1 --2- 172.21.15.59:0/3081548337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd0108f60 0x7f1dd0110ae0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009ca0 tx=0x7f1dc802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 -- 172.21.15.59:0/3081548337 shutdown_connections 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 --2- 172.21.15.59:0/3081548337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd0108f60 0x7f1dd0110ae0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 --2- 172.21.15.59:0/3081548337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd01047e0 0x7f1dd0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 --2- 172.21.15.59:0/3081548337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 -- 172.21.15.59:0/3081548337 >> 172.21.15.59:0/3081548337 conn(0x7f1dd00ff910 msgr2=0x7f1dd0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 -- 172.21.15.59:0/3081548337 shutdown_connections 2024-09-05T12:13:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.495+0000 7f1dd6000640 1 -- 172.21.15.59:0/3081548337 wait complete. 2024-09-05T12:13:52.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 Processor -- start 2024-09-05T12:13:52.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 -- start start 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd01047e0 0x7f1dd019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd0108f60 0x7f1dd019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0113fc0 con 0x7f1dd01047e0 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0113e40 con 0x7f1dd0103e30 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.496+0000 7f1dd6000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0114140 con 0x7f1dd0108f60 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd01047e0 0x7f1dd019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:45998/0 (socket says 172.21.15.59:45998) 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 -- 172.21.15.59:0/1035947425 learned_addr learned my addr 172.21.15.59:0/1035947425 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd5600640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd0108f60 0x7f1dd019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 -- 172.21.15.59:0/1035947425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd0108f60 msgr2=0x7f1dd019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd0108f60 0x7f1dd019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 -- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd01047e0 msgr2=0x7f1dd019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd01047e0 0x7f1dd019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 -- 172.21.15.59:0/1035947425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd019f3d0 con 0x7f1dd0103e30 2024-09-05T12:13:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.497+0000 7f1dd4c00640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9f0 tx=0x7f1dc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.498+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cd80 con 0x7f1dd0103e30 2024-09-05T12:13:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.498+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc4010910 con 0x7f1dd0103e30 2024-09-05T12:13:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.498+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd019f6c0 con 0x7f1dd0103e30 2024-09-05T12:13:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.498+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4018990 con 0x7f1dd0103e30 2024-09-05T12:13:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.498+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01a6fa0 con 0x7f1dd0103e30 2024-09-05T12:13:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.499+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd0105310 con 0x7f1dd0103e30 2024-09-05T12:13:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.501+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1dc4010ac0 con 0x7f1dd0103e30 2024-09-05T12:13:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.502+0000 7f1dcd600640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 0x7f1db80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.502+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1dc40cf7c0 con 0x7f1dd0103e30 2024-09-05T12:13:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.503+0000 7f1dcfe00640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 0x7f1db80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.503+0000 7f1dcfe00640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 0x7f1db80a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00ffd30 tx=0x7f1dbc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:52.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.506+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1dc409bc60 con 0x7f1dd0103e30 2024-09-05T12:13:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.731+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd0104210 con 0x7f1dd0103e30 2024-09-05T12:13:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.732+0000 7f1dcd600640 1 -- 172.21.15.59:0/1035947425 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f1dc409ff60 con 0x7f1dd0103e30 2024-09-05T12:13:52.734 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:52.734 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:52.735 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:52.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 msgr2=0x7f1db80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 0x7f1db80a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00ffd30 tx=0x7f1dbc005f90 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 msgr2=0x7f1dd019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9f0 tx=0x7f1dc400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 shutdown_connections 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f1db80a2b80 0x7f1db80a5040 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd0108f60 0x7f1dd019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dd01047e0 0x7f1dd019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 --2- 172.21.15.59:0/1035947425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dd0103e30 0x7f1dd019a420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 >> 172.21.15.59:0/1035947425 conn(0x7f1dd00ff910 msgr2=0x7f1dd0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.734+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 shutdown_connections 2024-09-05T12:13:52.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:52.735+0000 7f1dd6000640 1 -- 172.21.15.59:0/1035947425 wait complete. 2024-09-05T12:13:53.077 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:52 smithi059 ceph-mon[30025]: pgmap v880: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:13:53.077 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:52 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1035947425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:52 smithi189 ceph-mon[34772]: pgmap v880: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:13:53.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:52 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1035947425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:53.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:52 smithi089 ceph-mon[34499]: pgmap v880: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-05T12:13:53.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:52 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1035947425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:53.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:13:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:54.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.501+0000 7f0091000640 1 -- 172.21.15.59:0/2836510723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c1049c0 msgr2=0x7f008c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:54.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.501+0000 7f0091000640 1 --2- 172.21.15.59:0/2836510723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c1049c0 0x7f008c108a10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0074009900 tx=0x7f007402a410 comp rx=0 tx=0).stop 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 -- 172.21.15.59:0/2836510723 shutdown_connections 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 --2- 172.21.15.59:0/2836510723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c109140 0x7f008c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 --2- 172.21.15.59:0/2836510723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c1049c0 0x7f008c108a10 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 --2- 172.21.15.59:0/2836510723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c104010 0x7f008c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 -- 172.21.15.59:0/2836510723 >> 172.21.15.59:0/2836510723 conn(0x7f008c0ffa50 msgr2=0x7f008c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 -- 172.21.15.59:0/2836510723 shutdown_connections 2024-09-05T12:13:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.502+0000 7f0091000640 1 -- 172.21.15.59:0/2836510723 wait complete. 2024-09-05T12:13:54.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 Processor -- start 2024-09-05T12:13:54.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 -- start start 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 0x7f008c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 0x7f008c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f008c114160 con 0x7f008c104010 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f008c113fe0 con 0x7f008c1049c0 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.503+0000 7f0091000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1142e0 con 0x7f008c109140 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0081400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 0x7f008c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45746/0 (socket says 172.21.15.59:45746) 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 -- 172.21.15.59:0/1699080914 learned_addr learned my addr 172.21.15.59:0/1699080914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f008a800640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 0x7f008c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 -- 172.21.15.59:0/1699080914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 msgr2=0x7f008c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:54.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 0x7f008c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 -- 172.21.15.59:0/1699080914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 msgr2=0x7f008c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:54.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 0x7f008c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0089e00640 1 -- 172.21.15.59:0/1699080914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c19f690 con 0x7f008c104010 2024-09-05T12:13:54.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f0081400640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 0x7f008c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:54.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.504+0000 7f008a800640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 0x7f008c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0089e00640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f007c00e9f0 tx=0x7f007c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c00cd80 con 0x7f008c104010 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c19f980 con 0x7f008c104010 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007c005500 con 0x7f008c104010 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1a7260 con 0x7f008c104010 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.505+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c018630 con 0x7f008c104010 2024-09-05T12:13:54.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.507+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c1043f0 con 0x7f008c104010 2024-09-05T12:13:54.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.508+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f007c0046c0 con 0x7f008c104010 2024-09-05T12:13:54.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.510+0000 7f0083400640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 0x7f00700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:54.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.510+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f007c0cf990 con 0x7f008c104010 2024-09-05T12:13:54.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.510+0000 7f0081400640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 0x7f00700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:54.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.511+0000 7f0081400640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 0x7f00700a5040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0074009230 tx=0x7f00740058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:54.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.514+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f007c09c480 con 0x7f008c104010 2024-09-05T12:13:54.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.750+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008c10c380 con 0x7f008c104010 2024-09-05T12:13:54.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.751+0000 7f0083400640 1 -- 172.21.15.59:0/1699080914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11849 (secure 0 0 0) 0x7f007c09bd80 con 0x7f008c104010 2024-09-05T12:13:54.753 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"btime":"2024-09-05T11:55:15:722116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1152129733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1152129733},{"type":"v1","addr":"172.21.15.59:6835","nonce":1152129733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1365446032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1365446032},{"type":"v1","addr":"172.21.15.59:6837","nonce":1365446032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/3325685690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":3325685690},{"type":"v1","addr":"172.21.15.59:6839","nonce":3325685690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2374945780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2374945780},{"type":"v1","addr":"172.21.15.59:6841","nonce":2374945780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3760489514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3760489514},{"type":"v1","addr":"172.21.15.189:6833","nonce":3760489514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/2738174591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2738174591},{"type":"v1","addr":"172.21.15.89:6833","nonce":2738174591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4161006032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4161006032},{"type":"v1","addr":"172.21.15.189:6835","nonce":4161006032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/827690495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":827690495},{"type":"v1","addr":"172.21.15.189:6837","nonce":827690495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/318370648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":318370648},{"type":"v1","addr":"172.21.15.89:6835","nonce":318370648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3241376158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3241376158},{"type":"v1","addr":"172.21.15.189:6839","nonce":3241376158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T11:54:32.492433+0000","modified":"2024-09-05T11:55:14.725014+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6837/2748476304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":2748476304},{"type":"v1","addr":"172.21.15.89:6837","nonce":2748476304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4050620246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4050620246},{"type":"v1","addr":"172.21.15.89:6839","nonce":4050620246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545]},"id":1}]} 2024-09-05T12:13:54.754 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 21 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 msgr2=0x7f00700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 0x7f00700a5040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0074009230 tx=0x7f00740058d0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 msgr2=0x7f008c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f007c00e9f0 tx=0x7f007c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 shutdown_connections 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f00700a2b80 0x7f00700a5040 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f008c109140 0x7f008c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f008c1049c0 0x7f008c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 --2- 172.21.15.59:0/1699080914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f008c104010 0x7f008c19a6e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:54.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.753+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 >> 172.21.15.59:0/1699080914 conn(0x7f008c0ffa50 msgr2=0x7f008c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:54.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.754+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 shutdown_connections 2024-09-05T12:13:54.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:54.754+0000 7f0091000640 1 -- 172.21.15.59:0/1699080914 wait complete. 2024-09-05T12:13:55.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:54 smithi059 ceph-mon[30025]: pgmap v881: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-05T12:13:55.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:54 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:54 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/1699080914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:54 smithi189 ceph-mon[34772]: pgmap v881: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-05T12:13:55.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:54 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:54 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/1699080914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:54 smithi089 ceph-mon[34499]: pgmap v881: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-05T12:13:55.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:54 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:54 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/1699080914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:13:55.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:13:55.964 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:13:56.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:55 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:13:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:13:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:13:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:55 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:13:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-05T12:13:56.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-05T12:13:56.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi189 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-05T12:13:56.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi089 to 184530841: error parsing value: Value '184530841' is below minimum 939524096 2024-09-05T12:13:56.436 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:13:56.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:13:56.437 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:13:56.438 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:55 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' 2024-09-05T12:13:56.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.518+0000 7f72b1c00640 1 -- 172.21.15.59:0/3294994134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 msgr2=0x7f72ac108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.518+0000 7f72b1c00640 1 --2- 172.21.15.59:0/3294994134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac108830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7298009900 tx=0x7f729802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 -- 172.21.15.59:0/3294994134 shutdown_connections 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 --2- 172.21.15.59:0/3294994134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac108f60 0x7f72ac110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 --2- 172.21.15.59:0/3294994134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac108830 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 --2- 172.21.15.59:0/3294994134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac103e30 0x7f72ac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 -- 172.21.15.59:0/3294994134 >> 172.21.15.59:0/3294994134 conn(0x7f72ac0ff8d0 msgr2=0x7f72ac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:56.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 -- 172.21.15.59:0/3294994134 shutdown_connections 2024-09-05T12:13:56.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.519+0000 7f72b1c00640 1 -- 172.21.15.59:0/3294994134 wait complete. 2024-09-05T12:13:56.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 Processor -- start 2024-09-05T12:13:56.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 -- start start 2024-09-05T12:13:56.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:56.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac108f60 0x7f72ac19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac113fc0 con 0x7f72ac108f60 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac113e40 con 0x7f72ac103e30 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.520+0000 7f72b1c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac114140 con 0x7f72ac1047e0 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46034/0 (socket says 172.21.15.59:46034) 2024-09-05T12:13:56.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 -- 172.21.15.59:0/4214602516 learned_addr learned my addr 172.21.15.59:0/4214602516 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72ab400640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72b1200640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac108f60 0x7f72ac19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 -- 172.21.15.59:0/4214602516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 msgr2=0x7f72ac19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 -- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac108f60 msgr2=0x7f72ac19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac108f60 0x7f72ac19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.521+0000 7f72abe00640 1 -- 172.21.15.59:0/4214602516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac19f410 con 0x7f72ac103e30 2024-09-05T12:13:56.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72abe00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f729c00e9f0 tx=0x7f729c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:56.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c00cd80 con 0x7f72ac103e30 2024-09-05T12:13:56.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c010910 con 0x7f72ac103e30 2024-09-05T12:13:56.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c018990 con 0x7f72ac103e30 2024-09-05T12:13:56.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac19f700 con 0x7f72ac103e30 2024-09-05T12:13:56.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.522+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a6fe0 con 0x7f72ac103e30 2024-09-05T12:13:56.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.523+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac105310 con 0x7f72ac103e30 2024-09-05T12:13:56.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.526+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f729c010ac0 con 0x7f72ac103e30 2024-09-05T12:13:56.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.527+0000 7f72a8c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 0x7f72900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:56.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.527+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f729c0cfb20 con 0x7f72ac103e30 2024-09-05T12:13:56.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.527+0000 7f72ab400640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 0x7f72900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:56.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.528+0000 7f72ab400640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 0x7f72900a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7298009230 tx=0x7f72980058d0 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:56.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.529+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f729c09bfc0 con 0x7f72ac103e30 2024-09-05T12:13:56.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.714+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72ac10c140 con 0x7f72ac103e30 2024-09-05T12:13:56.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.716+0000 7f72a8c00640 1 -- 172.21.15.59:0/4214602516 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24957 (secure 0 0 0) 0x7f729c0a02c0 con 0x7f72ac103e30 2024-09-05T12:13:56.720 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-05T12:13:56.720 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"fsid":"53d560d4-6b7c-11ef-bcd6-c7b262605968","created":"2024-09-05T11:46:24.063849+0000","modified":"2024-09-05T11:56:19.566525+0000","last_up_change":"2024-09-05T11:53:34.769407+0000","last_in_change":"2024-09-05T11:53:11.374935+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T11:49:37.375375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T11:54:28.033815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T11:54:30.500703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T11:54:37.110249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8d274feb-cee7-4082-a14e-2c4c9ed0bc40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6803","nonce":2347559609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6805","nonce":2347559609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6809","nonce":2347559609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2347559609},{"type":"v1","addr":"172.21.15.59:6807","nonce":2347559609}]},"public_addr":"172.21.15.59:6803/2347559609","cluster_addr":"172.21.15.59:6805/2347559609","heartbeat_back_addr":"172.21.15.59:6809/2347559609","heartbeat_front_addr":"172.21.15.59:6807/2347559609","state":["exists","up"]},{"osd":1,"uuid":"02b53955-a3d9-45d3-9a4f-c36d00907942","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6801","nonce":3265971901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6803","nonce":3265971901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6807","nonce":3265971901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":3265971901},{"type":"v1","addr":"172.21.15.89:6805","nonce":3265971901}]},"public_addr":"172.21.15.89:6801/3265971901","cluster_addr":"172.21.15.89:6803/3265971901","heartbeat_back_addr":"172.21.15.89:6807/3265971901","heartbeat_front_addr":"172.21.15.89:6805/3265971901","state":["exists","up"]},{"osd":2,"uuid":"b03ca7ba-23bc-4f72-b35e-1784080f9550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6801","nonce":3282762650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6803","nonce":3282762650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6807","nonce":3282762650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3282762650},{"type":"v1","addr":"172.21.15.189:6805","nonce":3282762650}]},"public_addr":"172.21.15.189:6801/3282762650","cluster_addr":"172.21.15.189:6803/3282762650","heartbeat_back_addr":"172.21.15.189:6807/3282762650","heartbeat_front_addr":"172.21.15.189:6805/3282762650","state":["exists","up"]},{"osd":3,"uuid":"fe023f0b-749c-4eed-ae29-f58180c46a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6811","nonce":2051936210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6813","nonce":2051936210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6817","nonce":2051936210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2051936210},{"type":"v1","addr":"172.21.15.59:6815","nonce":2051936210}]},"public_addr":"172.21.15.59:6811/2051936210","cluster_addr":"172.21.15.59:6813/2051936210","heartbeat_back_addr":"172.21.15.59:6817/2051936210","heartbeat_front_addr":"172.21.15.59:6815/2051936210","state":["exists","up"]},{"osd":4,"uuid":"1ff44594-2df9-4bb2-8154-402f258fab7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6809","nonce":200597957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6811","nonce":200597957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6815","nonce":200597957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":200597957},{"type":"v1","addr":"172.21.15.89:6813","nonce":200597957}]},"public_addr":"172.21.15.89:6809/200597957","cluster_addr":"172.21.15.89:6811/200597957","heartbeat_back_addr":"172.21.15.89:6815/200597957","heartbeat_front_addr":"172.21.15.89:6813/200597957","state":["exists","up"]},{"osd":5,"uuid":"89d7954d-f1d1-4ac8-b1df-e6c542fe0104","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6809","nonce":3482558888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6811","nonce":3482558888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6815","nonce":3482558888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3482558888},{"type":"v1","addr":"172.21.15.189:6813","nonce":3482558888}]},"public_addr":"172.21.15.189:6809/3482558888","cluster_addr":"172.21.15.189:6811/3482558888","heartbeat_back_addr":"172.21.15.189:6815/3482558888","heartbeat_front_addr":"172.21.15.189:6813/3482558888","state":["exists","up"]},{"osd":6,"uuid":"3748a7e4-df25-42ae-b0cf-a50a43bb3ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6819","nonce":3110186175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6821","nonce":3110186175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6825","nonce":3110186175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":3110186175},{"type":"v1","addr":"172.21.15.59:6823","nonce":3110186175}]},"public_addr":"172.21.15.59:6819/3110186175","cluster_addr":"172.21.15.59:6821/3110186175","heartbeat_back_addr":"172.21.15.59:6825/3110186175","heartbeat_front_addr":"172.21.15.59:6823/3110186175","state":["exists","up"]},{"osd":7,"uuid":"20387607-714b-47c7-af78-daf2be8da827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6817","nonce":2237080020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6819","nonce":2237080020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6823","nonce":2237080020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2237080020},{"type":"v1","addr":"172.21.15.89:6821","nonce":2237080020}]},"public_addr":"172.21.15.89:6817/2237080020","cluster_addr":"172.21.15.89:6819/2237080020","heartbeat_back_addr":"172.21.15.89:6823/2237080020","heartbeat_front_addr":"172.21.15.89:6821/2237080020","state":["exists","up"]},{"osd":8,"uuid":"7405b92c-3969-427a-b954-31e466aef539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6817","nonce":2486650777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6819","nonce":2486650777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6823","nonce":2486650777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2486650777},{"type":"v1","addr":"172.21.15.189:6821","nonce":2486650777}]},"public_addr":"172.21.15.189:6817/2486650777","cluster_addr":"172.21.15.189:6819/2486650777","heartbeat_back_addr":"172.21.15.189:6823/2486650777","heartbeat_front_addr":"172.21.15.189:6821/2486650777","state":["exists","up"]},{"osd":9,"uuid":"85bf4014-3af1-4004-ac79-add151bbd6df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6827","nonce":1620616877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6829","nonce":1620616877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6833","nonce":1620616877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1620616877},{"type":"v1","addr":"172.21.15.59:6831","nonce":1620616877}]},"public_addr":"172.21.15.59:6827/1620616877","cluster_addr":"172.21.15.59:6829/1620616877","heartbeat_back_addr":"172.21.15.59:6833/1620616877","heartbeat_front_addr":"172.21.15.59:6831/1620616877","state":["exists","up"]},{"osd":10,"uuid":"f83d4779-7bf1-47e6-93a7-e54379bc692f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6825","nonce":1393916736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6827","nonce":1393916736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6831","nonce":1393916736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":1393916736},{"type":"v1","addr":"172.21.15.89:6829","nonce":1393916736}]},"public_addr":"172.21.15.89:6825/1393916736","cluster_addr":"172.21.15.89:6827/1393916736","heartbeat_back_addr":"172.21.15.89:6831/1393916736","heartbeat_front_addr":"172.21.15.89:6829/1393916736","state":["exists","up"]},{"osd":11,"uuid":"fedf8ed4-fc52-4603-b9cd-19837038c324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6825","nonce":1305059632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6827","nonce":1305059632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6831","nonce":1305059632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1305059632},{"type":"v1","addr":"172.21.15.189:6829","nonce":1305059632}]},"public_addr":"172.21.15.189:6825/1305059632","cluster_addr":"172.21.15.189:6827/1305059632","heartbeat_back_addr":"172.21.15.189:6831/1305059632","heartbeat_front_addr":"172.21.15.189:6829/1305059632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:48:45.403037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:10.526079+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:49:34.535106+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:02.725316+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:29.014826+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:50:53.921000+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:21.990064+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:51:48.742041+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:12.999352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:52:41.023877+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:07.232647+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T11:53:32.814254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6801/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/3566679520":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1131370065":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1895165557":"2024-09-06T11:47:16.761138+0000","172.21.15.59:6801/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2307158254":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/3602565252":"2024-09-06T11:46:49.566628+0000","172.21.15.59:0/1380942177":"2024-09-06T11:46:49.566628+0000","172.21.15.59:6800/2057764847":"2024-09-06T11:47:16.761138+0000","172.21.15.59:0/1592551985":"2024-09-06T11:46:49.566628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:13:56.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 msgr2=0x7f72900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:56.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 0x7f72900a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7298009230 tx=0x7f72980058d0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 msgr2=0x7f72ac19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f729c00e9f0 tx=0x7f729c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 shutdown_connections 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f72900a2c00 0x7f72900a50c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ac108f60 0x7f72ac19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72ac1047e0 0x7f72ac19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 --2- 172.21.15.59:0/4214602516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ac103e30 0x7f72ac19a460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.718+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 >> 172.21.15.59:0/4214602516 conn(0x7f72ac0ff8d0 msgr2=0x7f72ac100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.719+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 shutdown_connections 2024-09-05T12:13:56.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:56.719+0000 7f72b1c00640 1 -- 172.21.15.59:0/4214602516 wait complete. 2024-09-05T12:13:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:56 smithi059 ceph-mon[30025]: pgmap v882: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:56 smithi059 ceph-mon[30025]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:13:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:56 smithi059 ceph-mon[30025]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:13:57.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:56 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/4214602516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:56 smithi189 ceph-mon[34772]: pgmap v882: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:56 smithi189 ceph-mon[34772]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:13:57.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:56 smithi189 ceph-mon[34772]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:13:57.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:56 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/4214602516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:57.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:56 smithi089 ceph-mon[34499]: pgmap v882: 129 pgs: 129 active+clean; 7.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:57.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:56 smithi089 ceph-mon[34499]: Adjusting osd_memory_target on smithi059 to 177.5M 2024-09-05T12:13:57.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:56 smithi089 ceph-mon[34499]: Unable to set osd_memory_target on smithi059 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-05T12:13:57.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:56 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/4214602516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:13:57.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs fail cephfs 2024-09-05T12:13:57.830 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:13:58.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 -- 172.21.15.59:0/2960980372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 msgr2=0x7f9310114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:58.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 --2- 172.21.15.59:0/2960980372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f9310114e40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9308009ca0 tx=0x7f930802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 -- 172.21.15.59:0/2960980372 shutdown_connections 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 --2- 172.21.15.59:0/2960980372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f9310114e40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 --2- 172.21.15.59:0/2960980372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310077040 0x7f9310112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 --2- 172.21.15.59:0/2960980372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310076720 0x7f9310076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 -- 172.21.15.59:0/2960980372 >> 172.21.15.59:0/2960980372 conn(0x7f93100ffa50 msgr2=0x7f9310101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 -- 172.21.15.59:0/2960980372 shutdown_connections 2024-09-05T12:13:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.404+0000 7f9317a00640 1 -- 172.21.15.59:0/2960980372 wait complete. 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.405+0000 7f9317a00640 1 Processor -- start 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.405+0000 7f9317a00640 1 -- start start 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.405+0000 7f9317a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.405+0000 7f9317a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310077040 0x7f93101a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.405+0000 7f9317a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f931019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9317a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93101185f0 con 0x7f9310112a50 2024-09-05T12:13:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9317a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9310118470 con 0x7f9310076720 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9317a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9310118770 con 0x7f9310077040 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f930fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310077040 0x7f93101a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.59:46042/0 (socket says 172.21.15.59:46042) 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 -- 172.21.15.59:0/949565996 learned_addr learned my addr 172.21.15.59:0/949565996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:13:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f931019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 -- 172.21.15.59:0/949565996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310077040 msgr2=0x7f93101a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310077040 0x7f93101a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 -- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 msgr2=0x7f931019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f931019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.406+0000 7f9315000640 1 -- 172.21.15.59:0/949565996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931019eff0 con 0x7f9310076720 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f9315000640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9f0 tx=0x7f930400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930400cd80 con 0x7f9310076720 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931019f2e0 con 0x7f9310076720 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304005500 con 0x7f9310076720 2024-09-05T12:13:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304010690 con 0x7f9310076720 2024-09-05T12:13:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.407+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101ab5d0 con 0x7f9310076720 2024-09-05T12:13:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.408+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9310076b00 con 0x7f9310076720 2024-09-05T12:13:58.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.410+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f93040046c0 con 0x7f9310076720 2024-09-05T12:13:58.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.411+0000 7f930d600640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 0x7f92f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:13:58.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.411+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f93040cf890 con 0x7f9310076720 2024-09-05T12:13:58.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.411+0000 7f930fe00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 0x7f92f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:13:58.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.412+0000 7f930fe00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 0x7f92f80a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0095f0 tx=0x7f92fc005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:13:58.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.415+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f930409c380 con 0x7f9310076720 2024-09-05T12:13:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.641+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f931019fdf0 con 0x7f9310076720 2024-09-05T12:13:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.649+0000 7f930d600640 1 -- 172.21.15.59:0/949565996 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f930409bc80 con 0x7f9310076720 2024-09-05T12:13:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-05T12:13:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 msgr2=0x7f92f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 0x7f92f80a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0095f0 tx=0x7f92fc005f90 comp rx=0 tx=0).stop 2024-09-05T12:13:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 msgr2=0x7f93101a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:13:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9f0 tx=0x7f930400edf0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 shutdown_connections 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f92f80a2b80 0x7f92f80a5040 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310112a50 0x7f931019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9310077040 0x7f93101a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 --2- 172.21.15.59:0/949565996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9310076720 0x7f93101a4700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 >> 172.21.15.59:0/949565996 conn(0x7f93100ffa50 msgr2=0x7f9310101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 shutdown_connections 2024-09-05T12:13:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:13:58.653+0000 7f9317a00640 1 -- 172.21.15.59:0/949565996 wait complete. 2024-09-05T12:13:58.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a[30021]: 2024-09-05T12:13:58.642+0000 7f7ad8600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:13:58.949 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: ignoring --setuser ceph since I am not root 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: ignoring --setgroup ceph since I am not root 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mds[64019]: -- 172.21.15.89:0/986396862 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556736ee5860 con 0x556736ec3000 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: starting mds.b at 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: ignoring --setuser ceph since I am not root 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: ignoring --setgroup ceph since I am not root 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mds[63718]: -- 172.21.15.89:0/2867732975 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562cd8f01860 con 0x562cd8ede800 2024-09-05T12:13:58.950 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: starting mds.h at 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: pgmap v883: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/949565996' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:13:59.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:13:59.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:13:59.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:13:58 smithi059 ceph-mon[30025]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-05T12:13:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: pgmap v883: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:59.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/949565996' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:13:59.339 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:13:58 smithi189 ceph-mon[34772]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-05T12:13:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: pgmap v883: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-05T12:13:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/949565996' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:13:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:13:59.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:13:59.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:13:59.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:13:59.435 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:13:58 smithi089 ceph-mon[34499]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-05T12:13:59.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-05T12:13:59.903 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 -- 172.21.15.59:0/652498694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd140074ee0 msgr2=0x7fd14010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 --2- 172.21.15.59:0/652498694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd140074ee0 0x7fd14010dd20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009900 tx=0x7fd12c02a480 comp rx=0 tx=0).stop 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 -- 172.21.15.59:0/652498694 shutdown_connections 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 --2- 172.21.15.59:0/652498694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14010e260 0x7fd140110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 --2- 172.21.15.59:0/652498694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd140074ee0 0x7fd14010dd20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 --2- 172.21.15.59:0/652498694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1400745c0 0x7fd1400749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 -- 172.21.15.59:0/652498694 >> 172.21.15.59:0/652498694 conn(0x7fd1400fd740 msgr2=0x7fd1400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 -- 172.21.15.59:0/652498694 shutdown_connections 2024-09-05T12:14:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.447+0000 7fd145c00640 1 -- 172.21.15.59:0/652498694 wait complete. 2024-09-05T12:14:00.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.448+0000 7fd145c00640 1 Processor -- start 2024-09-05T12:14:00.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.448+0000 7fd145c00640 1 -- start start 2024-09-05T12:14:00.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1400745c0 0x7fd14019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 0x7fd14019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd140114000 con 0x7fd140074ee0 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd140113e80 con 0x7fd14010e260 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd140114180 con 0x7fd1400745c0 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd13f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd13f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45792/0 (socket says 172.21.15.59:45792) 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd13f400640 1 -- 172.21.15.59:0/190780750 learned_addr learned my addr 172.21.15.59:0/190780750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd13fe00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1400745c0 0x7fd14019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.449+0000 7fd145200640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 0x7fd14019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 -- 172.21.15.59:0/190780750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1400745c0 msgr2=0x7fd14019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1400745c0 0x7fd14019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 -- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 msgr2=0x7fd14019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 -- 172.21.15.59:0/190780750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14019f450 con 0x7fd14010e260 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd13f400640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145200640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 0x7fd14019ed70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd138005fd0 tx=0x7fd13800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13800cdf0 con 0x7fd14010e260 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd138005020 con 0x7fd14010e260 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380106d0 con 0x7fd14010e260 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14019f740 con 0x7fd14010e260 2024-09-05T12:14:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.450+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401a7020 con 0x7fd14010e260 2024-09-05T12:14:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.451+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1400749a0 con 0x7fd14010e260 2024-09-05T12:14:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.453+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1380054d0 con 0x7fd14010e260 2024-09-05T12:14:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.455+0000 7fd13cc00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 0x7fd1240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.455+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9352+0+0 (secure 0 0 0) 0x7fd1380cfac0 con 0x7fd14010e260 2024-09-05T12:14:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.455+0000 7fd13fe00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 0x7fd1240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.456+0000 7fd13fe00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 0x7fd1240a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd130009bd0 tx=0x7fd130005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.457+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd13809c560 con 0x7fd14010e260 2024-09-05T12:14:00.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:00.678+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd14002d7c0 con 0x7fd14010e260 2024-09-05T12:14:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: pgmap v885: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6836/595669453,v1:172.21.15.89:6838/595669453] up:boot 2024-09-05T12:14:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: mds.? [v2:172.21.15.89:6837/724812823,v1:172.21.15.89:6839/724812823] up:boot 2024-09-05T12:14:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-05T12:14:00.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:14:00.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:00 smithi089 ceph-mon[34499]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: pgmap v885: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6836/595669453,v1:172.21.15.89:6838/595669453] up:boot 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: mds.? [v2:172.21.15.89:6837/724812823,v1:172.21.15.89:6839/724812823] up:boot 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:14:00.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:00 smithi059 ceph-mon[30025]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:14:01.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: pgmap v885: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6836/595669453,v1:172.21.15.89:6838/595669453] up:boot 2024-09-05T12:14:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: mds.? [v2:172.21.15.89:6837/724812823,v1:172.21.15.89:6839/724812823] up:boot 2024-09-05T12:14:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-05T12:14:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:14:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:00 smithi189 ceph-mon[34772]: from='mgr.14146 172.21.15.59:0/450977689' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:14:01.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.662+0000 7fd13cc00640 1 -- 172.21.15.59:0/190780750 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7fd13809be60 con 0x7fd14010e260 2024-09-05T12:14:01.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 msgr2=0x7fd1240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 0x7fd1240a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd130009bd0 tx=0x7fd130005f90 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 msgr2=0x7fd14019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 0x7fd14019ed70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd138005fd0 tx=0x7fd13800ef10 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 shutdown_connections 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fd1240a2b80 0x7fd1240a5040 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd14010e260 0x7fd14019ed70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd140074ee0 0x7fd14019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 --2- 172.21.15.59:0/190780750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1400745c0 0x7fd14019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 >> 172.21.15.59:0/190780750 conn(0x7fd1400fd740 msgr2=0x7fd1400fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 shutdown_connections 2024-09-05T12:14:01.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:01.665+0000 7fd145c00640 1 -- 172.21.15.59:0/190780750 wait complete. 2024-09-05T12:14:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:01 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/190780750' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:01.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:01 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:01.959 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:01 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/190780750' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:01.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:01 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:02.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:01 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/190780750' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:02.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:01 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:14:02.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-05T12:14:02.810 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:14:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: pgmap v886: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:02.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:14:02.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:14:02.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: Cluster is now healthy 2024-09-05T12:14:02.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:14:02.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:02 smithi089 ceph-mon[34499]: fsmap 12 up:standby 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: pgmap v886: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: Cluster is now healthy 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:14:02.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:02 smithi059 ceph-mon[30025]: fsmap 12 up:standby 2024-09-05T12:14:03.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: pgmap v886: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-05T12:14:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:14:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:14:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: Cluster is now healthy 2024-09-05T12:14:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:14:03.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:02 smithi189 ceph-mon[34772]: fsmap 12 up:standby 2024-09-05T12:14:03.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 -- 172.21.15.59:0/2373680915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 msgr2=0x7f558c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 --2- 172.21.15.59:0/2373680915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c110ca0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5588009ca0 tx=0x7f558802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 -- 172.21.15.59:0/2373680915 shutdown_connections 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 --2- 172.21.15.59:0/2373680915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c110ca0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 --2- 172.21.15.59:0/2373680915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c1049a0 0x7f558c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 --2- 172.21.15.59:0/2373680915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c103ff0 0x7f558c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 -- 172.21.15.59:0/2373680915 >> 172.21.15.59:0/2373680915 conn(0x7f558c0ffa90 msgr2=0x7f558c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 -- 172.21.15.59:0/2373680915 shutdown_connections 2024-09-05T12:14:03.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.350+0000 7f5594800640 1 -- 172.21.15.59:0/2373680915 wait complete. 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.351+0000 7f5594800640 1 Processor -- start 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.351+0000 7f5594800640 1 -- start start 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c103ff0 0x7f558c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c1049a0 0x7f558c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f558c114180 con 0x7f558c109120 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f558c114000 con 0x7f558c103ff0 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5594800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f558c114300 con 0x7f558c1049a0 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5592800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:03.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5592800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42626/0 (socket says 172.21.15.59:42626) 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5592800640 1 -- 172.21.15.59:0/499711827 learned_addr learned my addr 172.21.15.59:0/499711827 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5591e00640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c103ff0 0x7f558c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.352+0000 7f5591400640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c1049a0 0x7f558c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 -- 172.21.15.59:0/499711827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c1049a0 msgr2=0x7f558c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c1049a0 0x7f558c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 -- 172.21.15.59:0/499711827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c103ff0 msgr2=0x7f558c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:03.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c103ff0 0x7f558c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 -- 172.21.15.59:0/499711827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c19f550 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5592800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5588003f10 tx=0x7f5588003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588031710 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c19f7e0 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55880318b0 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.353+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a7120 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.354+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558802fbb0 con 0x7f558c109120 2024-09-05T12:14:03.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.355+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c1043d0 con 0x7f558c109120 2024-09-05T12:14:03.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.356+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5588004040 con 0x7f558c109120 2024-09-05T12:14:03.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.358+0000 7f5586a00640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 0x7f55740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:03.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.358+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9352+0+0 (secure 0 0 0) 0x7f55880f0b20 con 0x7f558c109120 2024-09-05T12:14:03.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.358+0000 7f5591e00640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 0x7f55740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:03.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.359+0000 7f5591e00640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 0x7f55740a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5580005d00 tx=0x7f558000b410 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:03.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.362+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55880bd5c0 con 0x7f558c109120 2024-09-05T12:14:03.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.548+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f558c1a7380 con 0x7f558c109120 2024-09-05T12:14:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.659+0000 7f5586a00640 1 -- 172.21.15.59:0/499711827 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f55880bcec0 con 0x7f558c109120 2024-09-05T12:14:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-09-05T12:14:03.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.661+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 msgr2=0x7f55740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:03.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.661+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 0x7f55740a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5580005d00 tx=0x7f558000b410 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.661+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 msgr2=0x7f558c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.661+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5588003f10 tx=0x7f5588003f40 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 shutdown_connections 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f55740a2b80 0x7f55740a5040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f558c109120 0x7f558c19ee70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558c1049a0 0x7f558c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 --2- 172.21.15.59:0/499711827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c103ff0 0x7f558c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 >> 172.21.15.59:0/499711827 conn(0x7f558c0ffa90 msgr2=0x7f558c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 shutdown_connections 2024-09-05T12:14:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:03.662+0000 7f5594800640 1 -- 172.21.15.59:0/499711827 wait complete. 2024-09-05T12:14:03.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:03 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:03.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:03 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:03.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:03 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:04.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-05T12:14:04.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:04 smithi059 ceph-mon[30025]: pgmap v887: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-05T12:14:04.747 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:14:04.931 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:04 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:04.932 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:04 smithi059 ceph-mon[30025]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:14:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:04 smithi089 ceph-mon[34499]: pgmap v887: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-05T12:14:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:04 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:04.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:04 smithi089 ceph-mon[34499]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:14:05.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:04 smithi189 ceph-mon[34772]: pgmap v887: 129 pgs: 129 active+clean; 7.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-05T12:14:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:04 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/499711827' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:05.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:04 smithi189 ceph-mon[34772]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 -- 172.21.15.59:0/3375477580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee181079c0 msgr2=0x7fee18107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 --2- 172.21.15.59:0/3375477580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee181079c0 0x7fee18107da0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fee08009930 tx=0x7fee0802a420 comp rx=0 tx=0).stop 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 -- 172.21.15.59:0/3375477580 shutdown_connections 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 --2- 172.21.15.59:0/3375477580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18102360 0x7fee1810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 --2- 172.21.15.59:0/3375477580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181019c0 0x7fee18101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 --2- 172.21.15.59:0/3375477580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee181079c0 0x7fee18107da0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 -- 172.21.15.59:0/3375477580 >> 172.21.15.59:0/3375477580 conn(0x7fee180fd740 msgr2=0x7fee180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 -- 172.21.15.59:0/3375477580 shutdown_connections 2024-09-05T12:14:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.302+0000 7fee17200640 1 -- 172.21.15.59:0/3375477580 wait complete. 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.303+0000 7fee17200640 1 Processor -- start 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 -- start start 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee18102360 0x7fee1819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181079c0 0x7fee1819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee18111dc0 con 0x7fee181019c0 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee18111c40 con 0x7fee181079c0 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee17200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee18111f40 con 0x7fee18102360 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42638/0 (socket says 172.21.15.59:42638) 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee15e00640 1 -- 172.21.15.59:0/2954056519 learned_addr learned my addr 172.21.15.59:0/2954056519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:14:05.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee16800640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181079c0 0x7fee1819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.304+0000 7fee15400640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee18102360 0x7fee1819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 -- 172.21.15.59:0/2954056519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee18102360 msgr2=0x7fee1819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee18102360 0x7fee1819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 -- 172.21.15.59:0/2954056519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181079c0 msgr2=0x7fee1819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181079c0 0x7fee1819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 -- 172.21.15.59:0/2954056519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1819f410 con 0x7fee181019c0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee15e00640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fee08009230 tx=0x7fee08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0803f030 con 0x7fee181019c0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee08004440 con 0x7fee181019c0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.305+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1819f6a0 con 0x7fee181019c0 2024-09-05T12:14:05.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.306+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08035070 con 0x7fee181019c0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.306+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181a6fe0 con 0x7fee181019c0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.307+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee18103100 con 0x7fee181019c0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.308+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee08042030 con 0x7fee181019c0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.310+0000 7fee0ea00640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 0x7fedfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.310+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8769+0+0 (secure 0 0 0) 0x7fee080ef5a0 con 0x7fee181019c0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.310+0000 7fee15400640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 0x7fedfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.311+0000 7fee15400640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 0x7fedfc0a5040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fee180764a0 tx=0x7fee04005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:05.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.314+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee080bbc30 con 0x7fee181019c0 2024-09-05T12:14:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.504+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee181a71b0 con 0x7fee181019c0 2024-09-05T12:14:05.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.668+0000 7fee0ea00640 1 -- 172.21.15.59:0/2954056519 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fee080bff30 con 0x7fee181019c0 2024-09-05T12:14:05.670 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.672+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 msgr2=0x7fedfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.672+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 0x7fedfc0a5040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fee180764a0 tx=0x7fee04005f90 comp rx=0 tx=0).stop 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.672+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 msgr2=0x7fee1819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fee08009230 tx=0x7fee08004040 comp rx=0 tx=0).stop 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 shutdown_connections 2024-09-05T12:14:05.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7fedfc0a2b80 0x7fedfc0a5040 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee181079c0 0x7fee1819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee18102360 0x7fee1819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 --2- 172.21.15.59:0/2954056519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee181019c0 0x7fee1819a460 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 >> 172.21.15.59:0/2954056519 conn(0x7fee180fd740 msgr2=0x7fee180fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 shutdown_connections 2024-09-05T12:14:05.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:05.673+0000 7fee17200640 1 -- 172.21.15.59:0/2954056519 wait complete. 2024-09-05T12:14:05.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:05 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:05.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:05 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:06.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:05 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:06.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-05T12:14:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:14:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:06 smithi089 ceph-mon[34499]: pgmap v889: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:06 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:06.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:06 smithi089 ceph-mon[34499]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:14:06.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:06 smithi059 ceph-mon[30025]: pgmap v889: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:06.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:06 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:06.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:06 smithi059 ceph-mon[30025]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:14:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:06 smithi189 ceph-mon[34772]: pgmap v889: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:06 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/2954056519' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:06 smithi189 ceph-mon[34772]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 -- 172.21.15.59:0/580356313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 msgr2=0x7f658410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 --2- 172.21.15.59:0/580356313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658410e890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f657c009c10 tx=0x7f657c02a6f0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 -- 172.21.15.59:0/580356313 shutdown_connections 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 --2- 172.21.15.59:0/580356313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658410e890 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 --2- 172.21.15.59:0/580356313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 0x7f6584101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 --2- 172.21.15.59:0/580356313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 0x7f6584107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 -- 172.21.15.59:0/580356313 >> 172.21.15.59:0/580356313 conn(0x7f65840fd740 msgr2=0x7f65840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 -- 172.21.15.59:0/580356313 shutdown_connections 2024-09-05T12:14:07.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.425+0000 7f6588e00640 1 -- 172.21.15.59:0/580356313 wait complete. 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.426+0000 7f6588e00640 1 Processor -- start 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.426+0000 7f6588e00640 1 -- start start 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 0x7f658419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 0x7f658419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111dc0 con 0x7f6584102360 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111c40 con 0x7f65841019c0 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6588e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111f40 con 0x7f65841079c0 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6582a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6582a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42660/0 (socket says 172.21.15.59:42660) 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6582a00640 1 -- 172.21.15.59:0/3351423131 learned_addr learned my addr 172.21.15.59:0/3351423131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6583400640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 0x7f658419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.427+0000 7f6583e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 0x7f658419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 -- 172.21.15.59:0/3351423131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 msgr2=0x7f658419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 0x7f658419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 -- 172.21.15.59:0/3351423131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 msgr2=0x7f658419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 0x7f658419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 -- 172.21.15.59:0/3351423131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658419f470 con 0x7f6584102360 2024-09-05T12:14:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6582a00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9c0 tx=0x7f657000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657000ccc0 con 0x7f6584102360 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6570005500 con 0x7f6584102360 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658419f760 con 0x7f6584102360 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.428+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570010430 con 0x7f6584102360 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.429+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65841a7040 con 0x7f6584102360 2024-09-05T12:14:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.430+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6584101e20 con 0x7f6584102360 2024-09-05T12:14:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.432+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65700046c0 con 0x7f6584102360 2024-09-05T12:14:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.434+0000 7f657be00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 0x7f65680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:14:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.434+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8152+0+0 (secure 0 0 0) 0x7f65700cf290 con 0x7f6584102360 2024-09-05T12:14:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.435+0000 7f6583400640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 0x7f65680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:14:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.435+0000 7f6583400640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 0x7f65680a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6574009b50 tx=0x7f6574005f90 comp rx=0 tx=0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:14:07.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.438+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f657009c1f0 con 0x7f6584102360 2024-09-05T12:14:07.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.627+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65841a7340 con 0x7f6584102360 2024-09-05T12:14:07.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.677+0000 7f657be00640 1 -- 172.21.15.59:0/3351423131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7f657009baf0 con 0x7f6584102360 2024-09-05T12:14:07.678 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' removed 2024-09-05T12:14:07.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 msgr2=0x7f65680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:07.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 0x7f65680a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6574009b50 tx=0x7f6574005f90 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 msgr2=0x7f658419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9c0 tx=0x7f657000ee80 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 shutdown_connections 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:6800/23374249,v1:172.21.15.59:6801/23374249] conn(0x7f65680a2c00 0x7f65680a50c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65841079c0 0x7f658419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6584102360 0x7f658419aa00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 --2- 172.21.15.59:0/3351423131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65841019c0 0x7f658419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.679+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 >> 172.21.15.59:0/3351423131 conn(0x7f65840fd740 msgr2=0x7f65840fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:14:07.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.680+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 shutdown_connections 2024-09-05T12:14:07.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-05T12:14:07.680+0000 7f6588e00640 1 -- 172.21.15.59:0/3351423131 wait complete. 2024-09-05T12:14:07.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:07 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:07.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:07 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:08.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:07 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:14:08.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:14:08.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:14:08.480 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:14:08.514 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-05T12:14:08.515 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:14:08.550 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:14:08.567 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:14:08.585 INFO:tasks.cephadm:Stopping all daemons... 2024-09-05T12:14:08.585 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-05T12:14:08.586 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a 2024-09-05T12:14:08.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:08 smithi189 ceph-mon[34772]: pgmap v891: 33 pgs: 33 active+clean; 577 KiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:08 smithi189 ceph-mon[34772]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:08.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:08 smithi189 ceph-mon[34772]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:14:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:08 smithi089 ceph-mon[34499]: pgmap v891: 33 pgs: 33 active+clean; 577 KiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:08 smithi089 ceph-mon[34499]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:08.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:08 smithi089 ceph-mon[34499]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:14:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:08 smithi059 systemd[1]: Stopping Ceph mon.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:08 smithi059 ceph-mon[30025]: pgmap v891: 33 pgs: 33 active+clean; 577 KiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:14:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:08 smithi059 ceph-mon[30025]: from='client.? 172.21.15.59:0/3351423131' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:14:08.958 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:08 smithi059 ceph-mon[30025]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:14:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a[30021]: 2024-09-05T12:14:09.095+0000 7f7adf400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a[30021]: 2024-09-05T12:14:09.095+0000 7f7adf400640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-05T12:14:09.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 podman[116527]: 2024-09-05 12:14:09.209176344 +0000 UTC m=+0.327445738 container died 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:14:09.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 podman[116527]: 2024-09-05 12:14:09.643348763 +0000 UTC m=+0.761618156 container cleanup 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T12:14:09.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 bash[116527]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a 2024-09-05T12:14:10.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Sep 05 12:14:09 smithi059 podman[116539]: 2024-09-05 12:14:09.911332478 +0000 UTC m=+0.691568475 container remove 080e238eff3c9205aeb849110aea5b5be49234fbb2b753aa488ad2573e16cae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-05T12:14:10.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.a.service' 2024-09-05T12:14:10.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:10.287 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-05T12:14:10.287 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-05T12:14:10.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.b 2024-09-05T12:14:10.609 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:10 smithi089 systemd[1]: Stopping Ceph mon.b for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-b[34495]: 2024-09-05T12:14:10.608+0000 7f37d3347640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-b[34495]: 2024-09-05T12:14:10.608+0000 7f37d3347640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-05T12:14:10.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:10 smithi089 podman[69308]: 2024-09-05 12:14:10.833432289 +0000 UTC m=+0.352946283 container died 576bd771f338f6dc0e8772e5384b597b1af7df96e143a164e3ab52c9d3dd8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-b, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:14:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:11 smithi089 podman[69308]: 2024-09-05 12:14:11.309233099 +0000 UTC m=+0.828747089 container cleanup 576bd771f338f6dc0e8772e5384b597b1af7df96e143a164e3ab52c9d3dd8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T12:14:11.434 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 05 12:14:11 smithi089 bash[69308]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-b 2024-09-05T12:14:11.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.b.service' 2024-09-05T12:14:11.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:11.828 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-05T12:14:11.828 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-05T12:14:11.828 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.c 2024-09-05T12:14:12.142 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:11 smithi189 systemd[1]: Stopping Ceph mon.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:12.560 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c[34767]: 2024-09-05T12:14:12.140+0000 7fb969d6e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:12.560 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c[34767]: 2024-09-05T12:14:12.140+0000 7fb969d6e640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-05T12:14:12.561 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:12 smithi189 podman[69512]: 2024-09-05 12:14:12.243404637 +0000 UTC m=+0.221383285 container died 71c5ef535066c20ac73bf738235203241c1a878122df8c904c1f7d1314feb17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:14:12.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:12 smithi189 podman[69512]: 2024-09-05 12:14:12.76721711 +0000 UTC m=+0.745195758 container cleanup 71c5ef535066c20ac73bf738235203241c1a878122df8c904c1f7d1314feb17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T12:14:12.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:12 smithi189 bash[69512]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c 2024-09-05T12:14:13.251 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.c.service' 2024-09-05T12:14:13.273 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:13 smithi189 podman[69523]: 2024-09-05 12:14:13.002514251 +0000 UTC m=+0.755589620 container remove 71c5ef535066c20ac73bf738235203241c1a878122df8c904c1f7d1314feb17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T12:14:13.274 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:13 smithi189 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-05T12:14:13.274 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:13 smithi189 systemd[1]: Stopped Ceph mon.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T12:14:13.274 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 05 12:14:13 smithi189 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mon.c.service: Consumed 16.034s CPU time. 2024-09-05T12:14:13.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:13.303 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-05T12:14:13.303 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-05T12:14:13.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T12:14:13.708 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:13 smithi059 systemd[1]: Stopping Ceph mgr.x for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:14.060 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x[30253]: 2024-09-05T12:14:13.726+0000 7faa8d800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:14.060 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:13 smithi059 podman[116648]: 2024-09-05 12:14:13.808366972 +0000 UTC m=+0.210277950 container died 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-05T12:14:14.442 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:14 smithi059 podman[116648]: 2024-09-05 12:14:14.206605345 +0000 UTC m=+0.608516322 container cleanup 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-05T12:14:14.442 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:14 smithi059 bash[116648]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x 2024-09-05T12:14:14.698 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Sep 05 12:14:14 smithi059 podman[116660]: 2024-09-05 12:14:14.441716412 +0000 UTC m=+0.624417153 container remove 48d3a55566b07a0dfec4e5e94a2df73e42d7e0f6680fc9d79604bbab564fa93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-05T12:14:14.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.x.service' 2024-09-05T12:14:14.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:14.779 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-05T12:14:14.780 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-05T12:14:14.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.y 2024-09-05T12:14:15.118 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 12:14:14 smithi089 systemd[1]: Stopping Ceph mgr.y for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:15.408 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 12:14:15 smithi089 podman[69431]: 2024-09-05 12:14:15.118262409 +0000 UTC m=+0.151859354 container died 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T12:14:15.684 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 12:14:15 smithi089 podman[69431]: 2024-09-05 12:14:15.506702684 +0000 UTC m=+0.540299632 container cleanup 62c3ab2e75c4c7c799cdc59d6b7df72cedc46552b01666dae1eca7857faed12c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2) 2024-09-05T12:14:15.684 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 05 12:14:15 smithi089 bash[69431]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-y 2024-09-05T12:14:15.944 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.y.service' 2024-09-05T12:14:15.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:15.979 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-05T12:14:15.980 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-05T12:14:15.980 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.z 2024-09-05T12:14:16.318 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 12:14:16 smithi189 systemd[1]: Stopping Ceph mgr.z for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:16.588 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 12:14:16 smithi189 podman[69638]: 2024-09-05 12:14:16.318453937 +0000 UTC m=+0.143490342 container died 19ca017281fc758b858ab4ac1a2c77304bdca285e3e87ece790a62740dac2136 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T12:14:16.967 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 12:14:16 smithi189 podman[69638]: 2024-09-05 12:14:16.70668745 +0000 UTC m=+0.531723852 container cleanup 19ca017281fc758b858ab4ac1a2c77304bdca285e3e87ece790a62740dac2136 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:14:16.967 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 05 12:14:16 smithi189 bash[69638]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mgr-z 2024-09-05T12:14:17.133 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mgr.z.service' 2024-09-05T12:14:17.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:17.168 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-05T12:14:17.169 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-05T12:14:17.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.0 2024-09-05T12:14:17.557 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:17 smithi059 systemd[1]: Stopping Ceph osd.0 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:17.958 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T12:14:17.555+0000 7f98e8e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:17.958 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T12:14:17.555+0000 7f98e8e00640 -1 osd.0 97 *** Got signal Terminated *** 2024-09-05T12:14:17.958 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0[43607]: 2024-09-05T12:14:17.555+0000 7f98e8e00640 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:23.331 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:22 smithi059 podman[116766]: 2024-09-05 12:14:22.99276277 +0000 UTC m=+5.532207582 container died 0e1779df18c6de303b079a703bb39168efcb40e2e052937c3c162a90ab3d03cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-05T12:14:23.708 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:23 smithi059 podman[116766]: 2024-09-05 12:14:23.545280918 +0000 UTC m=+6.084725719 container cleanup 0e1779df18c6de303b079a703bb39168efcb40e2e052937c3c162a90ab3d03cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-09-05T12:14:23.708 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:23 smithi059 bash[116766]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0 2024-09-05T12:14:23.960 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:23 smithi059 podman[116784]: 2024-09-05 12:14:23.746724747 +0000 UTC m=+0.738891076 container remove 0e1779df18c6de303b079a703bb39168efcb40e2e052937c3c162a90ab3d03cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T12:14:24.601 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.232251025 +0000 UTC m=+0.027504048 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:14:24.601 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.357955807 +0000 UTC m=+0.153208803 container create fa85c9e238d33e7485d40af7288f2f736cc7ef06a073fbd2b095ad7cb464dcf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T12:14:24.873 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.601018628 +0000 UTC m=+0.396271650 container init fa85c9e238d33e7485d40af7288f2f736cc7ef06a073fbd2b095ad7cb464dcf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T12:14:24.873 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.608355014 +0000 UTC m=+0.403608022 container start fa85c9e238d33e7485d40af7288f2f736cc7ef06a073fbd2b095ad7cb464dcf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-05T12:14:24.873 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.683966285 +0000 UTC m=+0.479219300 container attach fa85c9e238d33e7485d40af7288f2f736cc7ef06a073fbd2b095ad7cb464dcf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T12:14:25.208 INFO:journalctl@ceph.osd.0.smithi059.stdout:Sep 05 12:14:24 smithi059 podman[116876]: 2024-09-05 12:14:24.872613103 +0000 UTC m=+0.667866114 container died fa85c9e238d33e7485d40af7288f2f736cc7ef06a073fbd2b095ad7cb464dcf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:14:25.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.0.service' 2024-09-05T12:14:25.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:25.878 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-05T12:14:25.878 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-05T12:14:25.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.1 2024-09-05T12:14:26.184 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:25 smithi089 systemd[1]: Stopping Ceph osd.1 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:26.684 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T12:14:26.185+0000 7fc3197cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:26.684 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T12:14:26.185+0000 7fc3197cb640 -1 osd.1 97 *** Got signal Terminated *** 2024-09-05T12:14:26.684 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1[38916]: 2024-09-05T12:14:26.185+0000 7fc3197cb640 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:31.934 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:31 smithi089 podman[69559]: 2024-09-05 12:14:31.648355449 +0000 UTC m=+5.579557482 container died e056e7cb8fb02f261dbdd5ae21c8d0f09fce04d45bc4177186e1966c1cfdab08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD) 2024-09-05T12:14:32.314 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69559]: 2024-09-05 12:14:32.045131806 +0000 UTC m=+5.976333825 container cleanup e056e7cb8fb02f261dbdd5ae21c8d0f09fce04d45bc4177186e1966c1cfdab08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-05T12:14:32.314 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 bash[69559]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1 2024-09-05T12:14:32.660 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69576]: 2024-09-05 12:14:32.313433086 +0000 UTC m=+0.661062165 container remove e056e7cb8fb02f261dbdd5ae21c8d0f09fce04d45bc4177186e1966c1cfdab08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-05T12:14:32.934 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69666]: 2024-09-05 12:14:32.562727194 +0000 UTC m=+0.015888677 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:14:32.934 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69666]: 2024-09-05 12:14:32.697838417 +0000 UTC m=+0.150999899 container create 0f01df68fcd6e1470cad5517fb4cdedfa12c960021dd4bae499a85b097bbe406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:14:33.396 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69666]: 2024-09-05 12:14:32.933932492 +0000 UTC m=+0.387093977 container init 0f01df68fcd6e1470cad5517fb4cdedfa12c960021dd4bae499a85b097bbe406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True) 2024-09-05T12:14:33.396 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:32 smithi089 podman[69666]: 2024-09-05 12:14:32.937329243 +0000 UTC m=+0.390490726 container start 0f01df68fcd6e1470cad5517fb4cdedfa12c960021dd4bae499a85b097bbe406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:14:33.396 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:33 smithi089 podman[69666]: 2024-09-05 12:14:33.042231775 +0000 UTC m=+0.495393267 container attach 0f01df68fcd6e1470cad5517fb4cdedfa12c960021dd4bae499a85b097bbe406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:14:33.396 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 05 12:14:33 smithi089 podman[69666]: 2024-09-05 12:14:33.125164856 +0000 UTC m=+0.578326341 container died 0f01df68fcd6e1470cad5517fb4cdedfa12c960021dd4bae499a85b097bbe406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T12:14:33.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.1.service' 2024-09-05T12:14:33.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:33.935 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-05T12:14:33.935 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-05T12:14:33.935 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.2 2024-09-05T12:14:34.270 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:33 smithi189 systemd[1]: Stopping Ceph osd.2 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:34.537 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T12:14:34.269+0000 7f20ddfbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:34.537 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T12:14:34.269+0000 7f20ddfbd640 -1 osd.2 97 *** Got signal Terminated *** 2024-09-05T12:14:34.537 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2[39697]: 2024-09-05T12:14:34.269+0000 7f20ddfbd640 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:39.970 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:39 smithi189 podman[69761]: 2024-09-05 12:14:39.622064302 +0000 UTC m=+5.496355635 container died 0f47b40b638438955445af126f0890b5c8d63c8ab076973fc120d9db9f30838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T12:14:40.338 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:39 smithi189 podman[69761]: 2024-09-05 12:14:39.969843914 +0000 UTC m=+5.844135279 container cleanup 0f47b40b638438955445af126f0890b5c8d63c8ab076973fc120d9db9f30838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T12:14:40.338 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:39 smithi189 bash[69761]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2 2024-09-05T12:14:40.735 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:40 smithi189 podman[69789]: 2024-09-05 12:14:40.355002822 +0000 UTC m=+0.730135995 container remove 0f47b40b638438955445af126f0890b5c8d63c8ab076973fc120d9db9f30838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T12:14:40.992 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:40 smithi189 podman[69879]: 2024-09-05 12:14:40.637254121 +0000 UTC m=+0.016389391 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:14:40.993 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:40 smithi189 podman[69879]: 2024-09-05 12:14:40.764351746 +0000 UTC m=+0.143487009 container create 20f118ecc7953bfa6bfcea6d089d092884f56c173ab6d71d0655fff0fb812517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-05T12:14:40.993 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:40 smithi189 podman[69879]: 2024-09-05 12:14:40.99205332 +0000 UTC m=+0.371188593 container init 20f118ecc7953bfa6bfcea6d089d092884f56c173ab6d71d0655fff0fb812517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:14:41.338 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:40 smithi189 podman[69879]: 2024-09-05 12:14:40.995723006 +0000 UTC m=+0.374858272 container start 20f118ecc7953bfa6bfcea6d089d092884f56c173ab6d71d0655fff0fb812517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:14:41.339 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:41 smithi189 podman[69879]: 2024-09-05 12:14:41.100495166 +0000 UTC m=+0.479630433 container attach 20f118ecc7953bfa6bfcea6d089d092884f56c173ab6d71d0655fff0fb812517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T12:14:41.339 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 05 12:14:41 smithi189 podman[69879]: 2024-09-05 12:14:41.180536257 +0000 UTC m=+0.559671523 container died 20f118ecc7953bfa6bfcea6d089d092884f56c173ab6d71d0655fff0fb812517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-2-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T12:14:41.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.2.service' 2024-09-05T12:14:41.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:41.874 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-05T12:14:41.874 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-05T12:14:41.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.3 2024-09-05T12:14:42.208 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:41 smithi059 systemd[1]: Stopping Ceph osd.3 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:42.708 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:42 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T12:14:42.443+0000 7f2d43200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:42.708 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:42 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T12:14:42.443+0000 7f2d43200640 -1 osd.3 97 *** Got signal Terminated *** 2024-09-05T12:14:42.708 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:42 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T12:14:42.443+0000 7f2d43200640 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:43.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:43.656+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:44.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:44 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:44.161+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:44.531 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:44.246+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:44.838 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:44.530+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:44.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:44 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:44.626+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:45.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:45 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:45.134+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:45.547 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:45.255+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:45.838 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:45.545+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:45.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:45 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:45.627+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:46.180 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:45 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T12:14:45.786+0000 7f2d39c00640 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:19.282684+0000 front 2024-09-05T12:14:19.282743+0000 (oldest deadline 2024-09-05T12:14:45.683375+0000) 2024-09-05T12:14:46.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:46.179+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:46.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:46.305+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:46.504+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:46.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:46 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:46.676+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:47.143 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:46 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3[50348]: 2024-09-05T12:14:46.737+0000 7f2d39c00640 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:19.282684+0000 front 2024-09-05T12:14:19.282743+0000 (oldest deadline 2024-09-05T12:14:45.683375+0000) 2024-09-05T12:14:47.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:47 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:47.142+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:47.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:47.287+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:47.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:47.517+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:48.168 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:47 smithi059 podman[116988]: 2024-09-05 12:14:47.821299473 +0000 UTC m=+5.646974605 container died 43d2ff916f998c8e399768645327a13a9fd0cb99f00508df0cfa379296e5412b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T12:14:48.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:47 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:47.700+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:48.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:48 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:48.166+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:48.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:48.253+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:48.468+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:48.777 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:48 smithi059 podman[116988]: 2024-09-05 12:14:48.500160417 +0000 UTC m=+6.325835547 container cleanup 43d2ff916f998c8e399768645327a13a9fd0cb99f00508df0cfa379296e5412b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-05T12:14:48.777 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:48 smithi059 bash[116988]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3 2024-09-05T12:14:49.154 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:48 smithi059 podman[117004]: 2024-09-05 12:14:48.776784642 +0000 UTC m=+0.943927356 container remove 43d2ff916f998c8e399768645327a13a9fd0cb99f00508df0cfa379296e5412b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T12:14:49.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:48 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:48.720+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:49.428 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:49 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:49.152+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:49.429 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.136930262 +0000 UTC m=+0.027737491 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:14:49.429 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.302643846 +0000 UTC m=+0.193451064 container create 0aeda122ff8b3c3524e9ccd3787f5b595e64ccb85a0e304df7262db9405a0ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T12:14:49.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:49.301+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:49.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:49.425+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:49.707 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.64700676 +0000 UTC m=+0.537813989 container init 0aeda122ff8b3c3524e9ccd3787f5b595e64ccb85a0e304df7262db9405a0ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:14:49.708 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.653922157 +0000 UTC m=+0.544729376 container start 0aeda122ff8b3c3524e9ccd3787f5b595e64ccb85a0e304df7262db9405a0ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-05T12:14:50.149 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.763698442 +0000 UTC m=+0.654505676 container attach 0aeda122ff8b3c3524e9ccd3787f5b595e64ccb85a0e304df7262db9405a0ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-05T12:14:50.150 INFO:journalctl@ceph.osd.3.smithi059.stdout:Sep 05 12:14:49 smithi059 podman[117095]: 2024-09-05 12:14:49.920189301 +0000 UTC m=+0.810996529 container died 0aeda122ff8b3c3524e9ccd3787f5b595e64ccb85a0e304df7262db9405a0ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T12:14:50.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:49 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:49.725+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:50.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:50 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:50.148+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:50.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.3.service' 2024-09-05T12:14:50.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:50.684 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-05T12:14:50.684 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-05T12:14:50.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.4 2024-09-05T12:14:50.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:50.351+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:50.838 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:50.470+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:51.001 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:50 smithi089 systemd[1]: Stopping Ceph osd.4 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:51.001 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:50 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:50.753+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:51.434 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:51 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:50.999+0000 7f2b7c2b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:51.434 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:51 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:50.999+0000 7f2b7c2b0640 -1 osd.4 97 *** Got signal Terminated *** 2024-09-05T12:14:51.434 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:51 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:50.999+0000 7f2b7c2b0640 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:51.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:51 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:51.141+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:51.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:51.379+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:51.455+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:52.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:51 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:51.712+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:52.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:52.122+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:52.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:52 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:52.122+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:52.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:52.419+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:52.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:52.530+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:52.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:52.483+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:53.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:52 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:52.702+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:53.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:52 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:52.702+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:53.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:53 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:53.168+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:53.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:53 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:53.168+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:53.684 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:53 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:53.416+0000 7f2b748be640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:25.364899+0000 front 2024-09-05T12:14:25.364927+0000 (oldest deadline 2024-09-05T12:14:52.965586+0000) 2024-09-05T12:14:53.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:53 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:53.422+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:53.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:53 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:53.494+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:53.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:53 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:53.469+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:53.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:53 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:53.469+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:54.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:53 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:53.752+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:54.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:53 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:53.752+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:54.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:54.121+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:54.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:54.121+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:54.684 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:54 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:54.397+0000 7f2b748be640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:25.364899+0000 front 2024-09-05T12:14:25.364927+0000 (oldest deadline 2024-09-05T12:14:52.965586+0000) 2024-09-05T12:14:54.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:54 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:54.459+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:54.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:54 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:54.463+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:54.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:54 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:54.456+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:54.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:54 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:54.456+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:54.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:54 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:54.504+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:14:55.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:54 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:54.709+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:55.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:54 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:54.709+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:55.380 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:55 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:55.078+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:55.381 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:55 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:55.078+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:55.684 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:55 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4[45339]: 2024-09-05T12:14:55.421+0000 7f2b748be640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:25.364899+0000 front 2024-09-05T12:14:25.364927+0000 (oldest deadline 2024-09-05T12:14:52.965586+0000) 2024-09-05T12:14:55.706 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:55 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:55.536+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:14:55.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:55.435+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:55.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:55.470+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:55.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:55.422+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:55.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:55 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:55.422+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:56.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:14:55 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:14:55.762+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:14:56.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:55 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:55.755+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:56.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:55 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:55.755+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:56.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:56 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:56.080+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:56.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:56 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:56.080+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:56.684 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:56 smithi089 podman[69782]: 2024-09-05 12:14:56.414698968 +0000 UTC m=+5.534000958 container died 5f701c526ae60c717eb052b6d54829883c5070cac41c3f49a812277c421f1364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-05T12:14:56.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:56 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:56.470+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:56.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:56 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:56.429+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:56.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:56 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:56.429+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:14:56.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:56 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:56.393+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:56.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:56 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:56.393+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:56.893 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:56 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:56.540+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:14:57.052 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:56 smithi089 podman[69782]: 2024-09-05 12:14:56.82496436 +0000 UTC m=+5.944266362 container cleanup 5f701c526ae60c717eb052b6d54829883c5070cac41c3f49a812277c421f1364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-05T12:14:57.052 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:56 smithi089 bash[69782]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4 2024-09-05T12:14:57.052 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:56 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:56.748+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:57.052 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:56 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:56.748+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:57.052 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:14:56 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:14:56.802+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:14:57.207 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:57 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:57.122+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:57.208 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:57 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:57.122+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:57.434 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69797]: 2024-09-05 12:14:57.051061874 +0000 UTC m=+0.633018825 container remove 5f701c526ae60c717eb052b6d54829883c5070cac41c3f49a812277c421f1364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T12:14:57.764 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:57 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:57.733+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:57.764 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:57 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:57.733+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:57.764 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.390716229 +0000 UTC m=+0.015845801 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:14:57.764 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.493793108 +0000 UTC m=+0.118922670 container create 8bfd1ae1c6d47e497f1ebb5c2a6d5e4cd8b265766f74d12c476023aa1071793c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:14:57.764 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.763369897 +0000 UTC m=+0.388499459 container init 8bfd1ae1c6d47e497f1ebb5c2a6d5e4cd8b265766f74d12c476023aa1071793c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-05T12:14:57.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:57 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:57.435+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:57.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:57 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:57.396+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:57.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:57 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:57.396+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:14:57.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:57 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:57.358+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:57.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:57 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:57.358+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:57.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:57 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:57.556+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:14:58.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:14:57 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:14:57.773+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:14:58.184 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.767572468 +0000 UTC m=+0.392702033 container start 8bfd1ae1c6d47e497f1ebb5c2a6d5e4cd8b265766f74d12c476023aa1071793c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T12:14:58.184 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.863331636 +0000 UTC m=+0.488461198 container attach 8bfd1ae1c6d47e497f1ebb5c2a6d5e4cd8b265766f74d12c476023aa1071793c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-05T12:14:58.184 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 05 12:14:57 smithi089 podman[69888]: 2024-09-05 12:14:57.954833172 +0000 UTC m=+0.579962743 container died 8bfd1ae1c6d47e497f1ebb5c2a6d5e4cd8b265766f74d12c476023aa1071793c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:14:58.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:58.160+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:58.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:58.160+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:58.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.4.service' 2024-09-05T12:14:58.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:14:58.695 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-05T12:14:58.695 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-05T12:14:58.695 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.5 2024-09-05T12:14:58.722 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:58 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:58.470+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:58.722 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:58 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:58.418+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:58.722 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:58 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:58.418+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:14:58.723 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:58 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:58.374+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:58.723 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:58 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:58.374+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:58.934 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:14:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:14:58.742+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:14:58.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:58.710+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:14:58.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:58 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:58.710+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:14:58.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:58.604+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:14:59.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:58 smithi189 systemd[1]: Stopping Ceph osd.5 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:14:59.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:59 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:59.152+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:14:59.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:14:59 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:14:59.152+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:14:59.506 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:59.389+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:14:59.506 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:14:59.389+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:14:59.506 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:59.165+0000 7fbd3c8fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:14:59.507 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:59.165+0000 7fbd3c8fc640 -1 osd.5 97 *** Got signal Terminated *** 2024-09-05T12:14:59.507 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:59.165+0000 7fbd3c8fc640 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:14:59.507 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:59.370+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:14:59.507 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:14:59.370+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:14:59.840 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:14:59 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:14:59.505+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:14:59.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:14:59 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:14:59.573+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:00.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:14:59 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:14:59.721+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:00.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:59 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:59.746+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:00.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:14:59 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:14:59.746+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:00.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:00 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:00.132+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:00.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:00 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:00.132+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:00.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:00.466+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:00.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:00.466+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:00.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:00.351+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:15:00.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:00.351+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:15:00.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:00.405+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:00.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:00 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:00.405+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:00.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:00 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:00.737+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:00.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:00 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:00.737+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:00.934 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:00 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:00.676+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:00.934 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:00 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:00.676+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:00.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:00 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:00.529+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:00.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:00 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:00.529+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:01.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:01.143+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:01.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:01.143+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:01.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:01.143+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:01.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:01.363+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:01.363+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:01.363+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:36.286293+0000 front 2024-09-05T12:14:36.286316+0000 (oldest deadline 2024-09-05T12:15:00.986474+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:01.433+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:01.433+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:01.430+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:01.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:01 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:01.430+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:01.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:01.564+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:01.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:01 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:01.564+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:02.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:01 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:01.701+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:02.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:01 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:01.701+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:02.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:01 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:01.692+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:02.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:01 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:01.692+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:02.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:01 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:01.692+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:02.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:02.190+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:02.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:02.190+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:02.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:02.190+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:02.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:02.385+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:15:02.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:02.385+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:02.385+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:36.286293+0000 front 2024-09-05T12:14:36.286316+0000 (oldest deadline 2024-09-05T12:15:00.986474+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:02.414+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:02.414+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:02.478+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:02.478+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:02.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:02 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:02.478+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:02.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:02.567+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:02.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:02.567+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:03.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:02 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:02.690+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:03.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:02 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:02.690+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:03.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:02 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:02.738+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:03.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:02 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:02.738+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:03.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:02 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:02.738+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:03.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:03 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:03.197+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:03.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:03 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:03.197+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:03.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:03 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:03.197+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:03.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:03.342+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.284453+0000 front 2024-09-05T12:14:22.284309+0000 (oldest deadline 2024-09-05T12:14:43.984411+0000) 2024-09-05T12:15:03.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:03.342+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.285406+0000 front 2024-09-05T12:14:29.285583+0000 (oldest deadline 2024-09-05T12:14:56.285929+0000) 2024-09-05T12:15:03.839 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5[46123]: 2024-09-05T12:15:03.342+0000 7fbd34f0a640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:36.286293+0000 front 2024-09-05T12:14:36.286316+0000 (oldest deadline 2024-09-05T12:15:00.986474+0000) 2024-09-05T12:15:03.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:03.425+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:03.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:03.425+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:03.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:03.436+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:03.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:03.436+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:03.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:03 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:03.436+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:03.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:03 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:03.520+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:03.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:03 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:03.520+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:04.085 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:03 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:03.709+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:04.085 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:03 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:03.709+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:04.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:03 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:03.720+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:04.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:03 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:03.720+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:04.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:03 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:03.720+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:04.477 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:04.219+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:04.477 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:04.219+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:04.478 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:04.219+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:04.740 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:04 smithi189 podman[69982]: 2024-09-05 12:15:04.481105507 +0000 UTC m=+5.592581471 container died 6874a77f2840b825a5d8d92e52c93418f487b3af46ef18fb510ad2fa444c5ba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T12:15:04.740 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:04.450+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:04.740 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:04.450+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:04.740 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:04.388+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:04.741 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:04.388+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:04.741 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:04 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:04.388+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:04.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:04.475+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:04.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:04 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:04.475+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:05.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:04 smithi189 podman[69982]: 2024-09-05 12:15:04.857834778 +0000 UTC m=+5.969310742 container cleanup 6874a77f2840b825a5d8d92e52c93418f487b3af46ef18fb510ad2fa444c5ba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T12:15:05.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:04 smithi189 bash[69982]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5 2024-09-05T12:15:05.098 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:04 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:04.701+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:05.098 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:04 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:04.701+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:05.098 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:04 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:04.732+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:05.098 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:04 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:04.732+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:05.098 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:04 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:04.732+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:05.414 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70000]: 2024-09-05 12:15:05.134739617 +0000 UTC m=+0.643355667 container remove 6874a77f2840b825a5d8d92e52c93418f487b3af46ef18fb510ad2fa444c5ba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T12:15:05.497 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:05 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:05.241+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:05.497 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:05 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:05.241+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:05.497 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:05 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:05.241+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:05.721 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.379592386 +0000 UTC m=+0.016286788 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.493926192 +0000 UTC m=+0.130620582 container create 2cee1b3d46eb11922357f18ce40db54542ef6063744344ec1cb3f3bbb4b7739d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:05 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:05.413+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:05 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:05.413+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:05 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:05.413+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:05 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:05.413+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:05.722 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:05 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:05.413+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:05.934 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:05 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:05.676+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:05.934 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:05 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:05.676+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:05.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:05 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:05.759+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:05.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:05 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:05.759+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:05.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:05 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:05.759+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:05.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:05 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:05.495+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:05.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:05 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:05.495+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:06.089 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.72133552 +0000 UTC m=+0.358029910 container init 2cee1b3d46eb11922357f18ce40db54542ef6063744344ec1cb3f3bbb4b7739d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-05T12:15:06.089 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.724697979 +0000 UTC m=+0.361392373 container start 2cee1b3d46eb11922357f18ce40db54542ef6063744344ec1cb3f3bbb4b7739d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T12:15:06.089 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.796453007 +0000 UTC m=+0.433147404 container attach 2cee1b3d46eb11922357f18ce40db54542ef6063744344ec1cb3f3bbb4b7739d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:15:06.089 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 05 12:15:05 smithi189 podman[70088]: 2024-09-05 12:15:05.916923539 +0000 UTC m=+0.553617939 container died 2cee1b3d46eb11922357f18ce40db54542ef6063744344ec1cb3f3bbb4b7739d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:15:06.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:06.194+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:06.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:06.194+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:06.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:06.194+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:06.509 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:06 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:06.390+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:06.509 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:06 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:06.390+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:06.510 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:06 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:06.421+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:06.510 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:06 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:06.421+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:06.510 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:06 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:06.421+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:06.532 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.5.service' 2024-09-05T12:15:06.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:06.613 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-05T12:15:06.613 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-05T12:15:06.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.6 2024-09-05T12:15:06.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:06.508+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:06.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:06.508+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:06.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:06 smithi059 systemd[1]: Stopping Ceph osd.6 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:07.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:06.686+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:07.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:06.686+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:07.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:06.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:07.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:06.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:07.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:06 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:06.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:07.160+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:07.160+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:07.160+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:07.054+0000 7ff28a600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:07.054+0000 7ff28a600640 -1 osd.6 97 *** Got signal Terminated *** 2024-09-05T12:15:07.458 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:07.054+0000 7ff28a600640 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:07.740 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:07 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:07.436+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:07.741 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:07 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:07.436+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:07.741 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:07 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:07.383+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:07.741 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:07 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:07.383+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:07.741 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:07 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:07.383+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:07.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:07.507+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:07.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:07 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:07.507+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:08.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:07.731+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:08.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:07.731+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:08.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:07.804+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:08.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:07.804+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:08.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:07 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:07.804+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:08.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:08.123+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:08.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:08.123+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:08.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:08.123+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:08.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:08.123+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:08.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:08 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:08.453+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:08.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:08 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:08.453+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:08.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:08 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:08.412+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:08.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:08 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:08.412+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:08.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:08 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:08.412+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:08.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:08.489+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:08.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:08 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:08.489+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:09.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:08.745+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:09.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:08.745+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:09.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:08.745+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:09.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:08.811+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:09.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:08.811+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:09.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:08.811+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:09.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:08 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:08.811+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:09.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:09.106+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:09.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:09.106+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:09.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:09.106+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:09.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:09.106+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:09.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:09 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:09.451+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:09.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:09 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:09.451+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:09.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:09 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:09.392+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:09.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:09 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:09.392+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:09.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:09 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:09.392+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:09.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:09.486+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:09.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:09 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:09.486+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:10.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:09.707+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:10.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:09.707+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:10.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:09.707+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:10.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:09.768+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:10.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:09.768+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:10.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:09.768+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:10.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:09.768+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:10.450 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:10.102+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:10.451 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:10.102+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:10.451 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:10.102+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:10.451 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:10.102+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:10.707 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:10.449+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:10.708 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:10 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:10.449+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:10.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:10.473+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:10.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:10.473+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:10.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:10.376+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:10.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:10.376+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:10.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:10 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:10.376+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:11.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:10.737+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:11.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:10.737+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:11.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:10.737+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:11.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:10.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:11.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:10.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:11.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:10.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:11.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:10 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:10.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:11.456 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:11.084+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:11.456 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:11.084+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:11.456 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:11.084+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:11.456 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:11.084+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:11.708 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:11.454+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.902882+0000 front 2024-09-05T12:14:30.902710+0000 (oldest deadline 2024-09-05T12:14:53.803007+0000) 2024-09-05T12:15:11.708 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:11 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6[57268]: 2024-09-05T12:15:11.454+0000 7ff286000640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:39.004944+0000 front 2024-09-05T12:14:39.004607+0000 (oldest deadline 2024-09-05T12:15:00.005410+0000) 2024-09-05T12:15:11.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:11.475+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:11.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:11.475+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:11.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:11.372+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:11.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:11.372+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:11.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:11 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:11.372+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:12.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:11.776+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:12.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:11.776+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:12.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:11.776+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:12.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:11.834+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:12.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:11.834+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:12.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:11.834+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:12.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:11 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:11.834+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:12.391 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:12.119+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:12.392 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:12.119+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:12.392 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:12.119+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:12.392 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:12 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:12.119+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:12.672 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:12 smithi059 podman[117211]: 2024-09-05 12:15:12.391104852 +0000 UTC m=+5.471483603 container died 92c7c646493a32d51de8175ef4de35af9674fa8a916530f072969aeeee363e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:15:12.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:12.486+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:12.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:12.486+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:12.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:12.486+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:12.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:12.346+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:12.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:12.346+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:12.839 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:12 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:12.346+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:12.957 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:12 smithi059 podman[117211]: 2024-09-05 12:15:12.886631519 +0000 UTC m=+5.967010269 container cleanup 92c7c646493a32d51de8175ef4de35af9674fa8a916530f072969aeeee363e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T12:15:12.958 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:12 smithi059 bash[117211]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6 2024-09-05T12:15:13.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:12.728+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:13.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:12.728+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:13.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:12.728+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:13.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:12.822+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:13.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:12.822+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:13.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:12.822+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:13.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:12 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:12.822+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:13.458 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117227]: 2024-09-05 12:15:13.163262773 +0000 UTC m=+0.761096180 container remove 92c7c646493a32d51de8175ef4de35af9674fa8a916530f072969aeeee363e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:15:13.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:13.075+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:13.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:13.075+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:13.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:13.075+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:13.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:13 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:13.075+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:13.588 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:13.526+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:13.588 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:13.526+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:13.588 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:13.526+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:13.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:13.320+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:13.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:13.320+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:13.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:13 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:13.320+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:13.885 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117316]: 2024-09-05 12:15:13.516235596 +0000 UTC m=+0.027947198 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:13.885 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117316]: 2024-09-05 12:15:13.639259044 +0000 UTC m=+0.150970616 container create 9ee07899eba1a91afc48aa9cb2ed6cc9527bf41a9c6c0e58f5e545d01e41925e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T12:15:13.885 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117316]: 2024-09-05 12:15:13.884278473 +0000 UTC m=+0.395990045 container init 9ee07899eba1a91afc48aa9cb2ed6cc9527bf41a9c6c0e58f5e545d01e41925e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T12:15:14.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117316]: 2024-09-05 12:15:13.89141297 +0000 UTC m=+0.403124543 container start 9ee07899eba1a91afc48aa9cb2ed6cc9527bf41a9c6c0e58f5e545d01e41925e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T12:15:14.155 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:13 smithi059 podman[117316]: 2024-09-05 12:15:13.983168715 +0000 UTC m=+0.494880309 container attach 9ee07899eba1a91afc48aa9cb2ed6cc9527bf41a9c6c0e58f5e545d01e41925e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T12:15:14.155 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:14.101+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:14.155 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:14.101+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:14.155 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:14.101+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:14.155 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:14 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:14.101+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:13.682+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:13.682+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:13.682+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:13.870+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:13.870+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:14.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:13.870+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:14.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:13 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:13.870+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:14.457 INFO:journalctl@ceph.osd.6.smithi059.stdout:Sep 05 12:15:14 smithi059 podman[117316]: 2024-09-05 12:15:14.154371216 +0000 UTC m=+0.666082794 container died 9ee07899eba1a91afc48aa9cb2ed6cc9527bf41a9c6c0e58f5e545d01e41925e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:15:14.588 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:14.554+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:14.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:14.554+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:14.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:14.554+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:14.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:14.323+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:14.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:14.323+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:14.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:14 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:14.323+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:14.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.6.service' 2024-09-05T12:15:15.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:15.073 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-05T12:15:15.073 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-05T12:15:15.073 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.7 2024-09-05T12:15:15.101 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:14.710+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:14.710+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:14.710+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:14.829+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:14.829+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:14.829+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:15.102 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:14 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:14.829+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:15.379 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 systemd[1]: Stopping Ceph osd.7 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:15.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:15.089+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:15.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:15.089+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:15.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:15.089+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:15.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:15 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:15.089+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:15.571 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:15.300+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:15.571 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:15.300+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:15.571 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:15.300+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:15.683 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.377+0000 7fe25a089640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:15.684 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.377+0000 7fe25a089640 -1 osd.7 97 *** Got signal Terminated *** 2024-09-05T12:15:15.684 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.377+0000 7fe25a089640 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:15.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:15.569+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:15.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:15.569+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:15.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:15 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:15.569+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:16.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.741+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:16.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.741+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:16.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:15.741+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:16.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:15.780+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:16.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:15.780+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:16.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:15.780+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:16.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:15.780+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:16.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:16.067+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:16.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:16.067+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:16.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:16.067+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:16.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:16 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:16.067+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:16.547+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:16.547+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:16.548+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:16.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:16.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:16.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:16 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:16.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:17.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:16.801+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:17.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:16.801+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:17.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:16.801+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:17.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:16.801+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:17.185 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:16.755+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:17.185 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:16.755+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:17.185 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:16 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:16.755+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:17.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:17.069+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:17.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:17.069+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:17.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:17.069+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:17.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:17 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:17.069+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:17.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:17.305+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:17.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:17.305+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:17.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:17.305+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:18.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:17.587+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:18.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:17.587+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:18.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:17 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:17.587+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:18.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:17.760+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:18.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:17.760+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:18.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:17.760+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:18.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:17.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:18.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:17.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:18.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:17.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:18.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:17 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:17.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:18.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:18.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:18.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:18.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:18.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:18.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:18.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:18.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:18.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:18 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:18.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:18.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:18.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:18.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:18.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:18.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:18.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:18.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:18.313+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:19.088 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:18.590+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:19.088 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:18.590+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:19.088 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:18.590+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:19.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:18 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:18.590+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:19.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:18.784+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:19.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:18.784+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:19.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:18.784+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:19.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:18.784+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.143270+0000 front 2024-09-05T12:14:55.143306+0000 (oldest deadline 2024-09-05T12:15:18.643317+0000) 2024-09-05T12:15:19.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:18.818+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:19.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:18.818+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:19.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:18.818+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:19.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:18 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:18.818+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:19.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:19 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:19.086+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:19.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:19 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:19.086+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:19.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:19 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:19.086+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:19.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:19 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:19.086+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:19.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:19 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:19.086+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:19.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:19.291+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:19.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:19.291+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:19.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:19.291+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:19.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:19.291+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:20.088 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:19.596+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:20.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:19.596+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:20.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:19.596+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:20.089 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:19 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:19.596+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:19.805+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:30.739032+0000 front 2024-09-05T12:14:30.738987+0000 (oldest deadline 2024-09-05T12:14:55.439191+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:19.805+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.140395+0000 front 2024-09-05T12:14:37.140567+0000 (oldest deadline 2024-09-05T12:15:00.040430+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:19.805+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:42.842013+0000 front 2024-09-05T12:14:42.842055+0000 (oldest deadline 2024-09-05T12:15:08.141927+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7[52089]: 2024-09-05T12:15:19.805+0000 7fe25669f640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.143270+0000 front 2024-09-05T12:14:55.143306+0000 (oldest deadline 2024-09-05T12:15:18.643317+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:19.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:20.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:19.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:20.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:19.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:20.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:19.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:20.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:19 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:19.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:20.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:20 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:20.115+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:20.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:20 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:20.115+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:20.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:20 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:20.115+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:20.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:20 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:20.115+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:20.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:20 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:20.115+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:20.551 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:20.242+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:20.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:20.242+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:20.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:20.242+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:20.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:20.242+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:20.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:20.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:20.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:20.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:20.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:20.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:20.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:20 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:20.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:20 smithi089 podman[70003]: 2024-09-05 12:15:20.88705228 +0000 UTC m=+5.618695221 container died f6f02dc77deaf11ae796a9c1abac2ceef442945512277af181884587435cfca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:20.749+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:20.749+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:20.749+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:20.749+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:21.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:20.749+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:21.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:21.149+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:21.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:21.149+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:21.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:21.149+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:21.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:21.149+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:21.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:21 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:21.149+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:21.522 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:21 smithi089 podman[70003]: 2024-09-05 12:15:21.286214038 +0000 UTC m=+6.017856965 container cleanup f6f02dc77deaf11ae796a9c1abac2ceef442945512277af181884587435cfca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-05T12:15:21.522 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:21 smithi089 bash[70003]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7 2024-09-05T12:15:21.551 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:21.246+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:21.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:21.246+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:21.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:21.246+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:21.552 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:21.246+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:21.791 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:21 smithi089 podman[70017]: 2024-09-05 12:15:21.521171534 +0000 UTC m=+0.632316611 container remove f6f02dc77deaf11ae796a9c1abac2ceef442945512277af181884587435cfca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T12:15:21.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:21.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:21.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:21.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:21.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:21.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:21.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:21 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:21.550+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:22.083 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:21 smithi089 podman[70108]: 2024-09-05 12:15:21.730640734 +0000 UTC m=+0.015500085 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:22.083 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:21 smithi089 podman[70108]: 2024-09-05 12:15:21.863783455 +0000 UTC m=+0.148642834 container create 43d99a77870dc898d0c2deac1ffeb70b5b2693d70aff0b3e4ca71e1c6a884122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T12:15:22.083 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:22 smithi089 podman[70108]: 2024-09-05 12:15:22.082864141 +0000 UTC m=+0.367723488 container init 43d99a77870dc898d0c2deac1ffeb70b5b2693d70aff0b3e4ca71e1c6a884122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:15:22.083 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:21 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:21.790+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:22.084 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:21 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:21.790+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:22.084 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:21 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:21.790+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:22.084 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:21 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:21.790+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:22.084 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:21 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:21.790+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:22.434 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:22 smithi089 podman[70108]: 2024-09-05 12:15:22.086158754 +0000 UTC m=+0.371018094 container start 43d99a77870dc898d0c2deac1ffeb70b5b2693d70aff0b3e4ca71e1c6a884122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:15:22.434 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:22 smithi089 podman[70108]: 2024-09-05 12:15:22.174481731 +0000 UTC m=+0.459341073 container attach 43d99a77870dc898d0c2deac1ffeb70b5b2693d70aff0b3e4ca71e1c6a884122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T12:15:22.434 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 05 12:15:22 smithi089 podman[70131]: 2024-09-05 12:15:22.30230432 +0000 UTC m=+0.016559598 container died 43d99a77870dc898d0c2deac1ffeb70b5b2693d70aff0b3e4ca71e1c6a884122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:15:22.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:22.120+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:22.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:22.120+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:22.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:22.120+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:22.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:22.120+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:22.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:22 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:22.120+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:22.544 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:22.209+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:22.544 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:22.209+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:22.544 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:22.209+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:22.544 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:22.209+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:22.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:22.542+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:22.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:22.542+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:22.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:22.542+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:22.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:22 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:22.542+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:22.887 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:22.833+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:22.887 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:22.833+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:22.887 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:22.833+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:22.887 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:22.833+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:22.887 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:22 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:22.833+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:22.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.7.service' 2024-09-05T12:15:22.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:22.989 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-05T12:15:22.989 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-05T12:15:22.989 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.8 2024-09-05T12:15:23.321 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 systemd[1]: Stopping Ceph osd.8 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:23.321 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:23.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:23.321 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:23.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:23.322 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:23.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:23.322 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:23.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:23.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:23 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:23.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:23.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:23 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:23.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:23.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:23 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:23.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:23.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:23 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:23.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:23.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:23 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:23.112+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.319+0000 7fd6d8468640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.319+0000 7fd6d8468640 -1 osd.8 97 *** Got signal Terminated *** 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.319+0000 7fd6d8468640 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.531+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.531+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.531+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:23.589 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:23 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:23.531+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:24.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:23.843+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:24.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:23.843+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:24.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:23.843+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:24.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:23.843+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:24.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:23.843+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:24.359 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:24 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:24.096+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:24.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:24 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:24.096+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:24.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:24 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:24.096+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:24.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:24 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:24.096+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:24.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:24 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:24.096+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:24.566 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:24.128+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:24.566 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:24.128+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:24.566 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:24.128+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:24.566 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:24.128+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:24.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:24.565+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:24.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:24.565+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:24.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:24.565+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:24.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:24 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:24.565+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:25.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:24.797+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:25.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:24.797+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:25.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:24.797+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:25.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:24.797+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:25.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:24 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:24.797+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:25.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:25 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:25.056+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:25.540 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:25.096+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:25.541 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:25.096+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:25.541 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:25.096+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:25.541 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:25.096+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:25.541 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:25.096+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:25.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:25.539+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:25.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:25.539+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:25.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:25.539+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:25.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:25.539+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:26.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:25.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:26.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:25.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:26.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:25.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:26.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:25.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:26.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:25 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:25.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:26.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:26.054+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:26.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:26.054+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:26.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:26.054+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:26.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:26.054+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:26.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:26.054+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:26.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.009+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:26.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:26.510+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:26.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:26.510+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:26.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:26.510+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:26.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:26.510+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:26.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:26 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:26.510+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.917032+0000 front 2024-09-05T12:15:01.917075+0000 (oldest deadline 2024-09-05T12:15:26.017101+0000) 2024-09-05T12:15:27.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:26.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:27.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:26.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:27.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:26.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:27.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:26.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:27.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:26 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:26.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:27.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:27.055+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:27.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:27.055+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:27.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:27.055+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:27.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:27.055+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:27.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:27.055+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:27.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:26 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:26.970+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:27.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:27.485+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:27.612141+0000 front 2024-09-05T12:14:27.612306+0000 (oldest deadline 2024-09-05T12:14:51.712110+0000) 2024-09-05T12:15:27.838 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:27.485+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:37.413746+0000 front 2024-09-05T12:14:37.413717+0000 (oldest deadline 2024-09-05T12:14:59.713880+0000) 2024-09-05T12:15:27.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:27.485+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:46.615595+0000 front 2024-09-05T12:14:46.615668+0000 (oldest deadline 2024-09-05T12:15:11.915635+0000) 2024-09-05T12:15:27.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:27.485+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.916235+0000 front 2024-09-05T12:14:51.916253+0000 (oldest deadline 2024-09-05T12:15:17.816180+0000) 2024-09-05T12:15:27.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:27 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8[52589]: 2024-09-05T12:15:27.485+0000 7fd6d0a76640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.917032+0000 front 2024-09-05T12:15:01.917075+0000 (oldest deadline 2024-09-05T12:15:26.017101+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:28.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:27 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:27.770+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:28.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:28.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:28.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:28.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:28.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:28.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:27 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:27.962+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:28.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:28 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:28.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:28.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:28 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:28.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:28.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:28 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:28.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:28.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:28 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:28.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:28.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:28 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:28.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:29.072 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:28 smithi189 podman[70206]: 2024-09-05 12:15:28.804237642 +0000 UTC m=+5.609233140 container died 9528e2c16f84c053a85e21b0bacf77551a466f92f367e78e304856b6dbe8f962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T12:15:29.073 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:29 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:29.045+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:29.073 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:29 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:29.045+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:29.073 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:29 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:29.045+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:29.073 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:29 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:29.045+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:29.073 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:29 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:29.045+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:29.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:28 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:28.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:29.338 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:29 smithi189 podman[70206]: 2024-09-05 12:15:29.16972571 +0000 UTC m=+5.974721198 container cleanup 9528e2c16f84c053a85e21b0bacf77551a466f92f367e78e304856b6dbe8f962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T12:15:29.338 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:29 smithi189 bash[70206]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:29.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:29.785 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:29 smithi189 podman[70222]: 2024-09-05 12:15:29.446113171 +0000 UTC m=+0.638982412 container remove 9528e2c16f84c053a85e21b0bacf77551a466f92f367e78e304856b6dbe8f962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-05T12:15:30.061 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:29 smithi189 podman[70315]: 2024-09-05 12:15:29.687420095 +0000 UTC m=+0.015909182 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:30.061 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:29 smithi189 podman[70315]: 2024-09-05 12:15:29.830536452 +0000 UTC m=+0.159025569 container create 76a472dabce9eb19d5354542b970a1da3d760556337443938ac8a2cbffb73e65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T12:15:30.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:30.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:30.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:30.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:30.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:30.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:29 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:29.791+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:30.331 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:30 smithi189 podman[70315]: 2024-09-05 12:15:30.093129932 +0000 UTC m=+0.421619006 container init 76a472dabce9eb19d5354542b970a1da3d760556337443938ac8a2cbffb73e65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:15:30.331 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:30 smithi189 podman[70315]: 2024-09-05 12:15:30.096412799 +0000 UTC m=+0.424901873 container start 76a472dabce9eb19d5354542b970a1da3d760556337443938ac8a2cbffb73e65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-05T12:15:30.331 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:30 smithi189 podman[70315]: 2024-09-05 12:15:30.18310926 +0000 UTC m=+0.511598342 container attach 76a472dabce9eb19d5354542b970a1da3d760556337443938ac8a2cbffb73e65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T12:15:30.331 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 05 12:15:30 smithi189 podman[70315]: 2024-09-05 12:15:30.299725 +0000 UTC m=+0.628214084 container died 76a472dabce9eb19d5354542b970a1da3d760556337443938ac8a2cbffb73e65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T12:15:30.332 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:30.059+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:30.332 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:30.059+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:30.332 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:30.059+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:30.332 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:30.059+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:30.332 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:30.059+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:30.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:29 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:29.971+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:30.979 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.8.service' 2024-09-05T12:15:31.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:31.018 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-05T12:15:31.018 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-05T12:15:31.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.9 2024-09-05T12:15:31.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:31.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:31.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:31.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:31.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:31.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:30 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:30.806+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:31.298 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:31.044+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:31.299 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:31.044+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:31.299 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:31.044+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:31.299 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:31.044+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:31.299 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:31 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:31.044+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:31.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:30 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:30.989+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:31.459 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:31 smithi059 systemd[1]: Stopping Ceph osd.9 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:31.957 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:31 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:31.517+0000 7fc43be00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:31.958 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:31 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:31.517+0000 7fc43be00640 -1 osd.9 97 *** Got signal Terminated *** 2024-09-05T12:15:31.958 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:31 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:31.517+0000 7fc43be00640 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:32.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:32.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:32.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:32.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:32.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:32.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:31 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:31.810+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:32.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:32 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:32.014+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:32.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:32 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:32.014+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:32.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:32 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:32.014+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:32.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:32 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:32.014+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:32.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:32 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:32.014+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:32.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:32.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:32.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:32.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:32.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:32.459 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.011+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:33.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:32 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:32.786+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:33.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:33.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:33.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:33.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:33.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:33.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:33.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:33.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:33.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:33 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:33.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:33.457 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:33.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:33.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:33.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:33.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:33.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:32 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:32.964+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:34.085 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:34.086 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:33 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:33.830+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:34.262 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:34.262 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:34.262 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:34.262 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:34.262 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:34.263 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.008+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:34.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:33 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:33.966+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:35.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:35.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:34 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:34.855+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:35.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:35.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:35.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:35.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:35.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:35.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:34 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:34.973+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:35.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:34 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:34.993+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:36.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:36.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:36.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:36.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:36.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:36.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:36.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:35 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:35.813+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:36.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:36.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:36.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:36.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:36.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:36.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:36 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:36.007+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.870208+0000 front 2024-09-05T12:14:18.870397+0000 (oldest deadline 2024-09-05T12:14:43.570377+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.172192+0000 front 2024-09-05T12:14:28.172145+0000 (oldest deadline 2024-09-05T12:14:51.672045+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:35.773082+0000 front 2024-09-05T12:14:35.773117+0000 (oldest deadline 2024-09-05T12:15:00.473067+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:44.574056+0000 front 2024-09-05T12:14:44.574029+0000 (oldest deadline 2024-09-05T12:15:08.074077+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:51.476367+0000 front 2024-09-05T12:14:51.476304+0000 (oldest deadline 2024-09-05T12:15:17.376529+0000) 2024-09-05T12:15:36.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9[64199]: 2024-09-05T12:15:36.010+0000 7fc432800640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:01.477952+0000 front 2024-09-05T12:15:01.477864+0000 (oldest deadline 2024-09-05T12:15:24.977652+0000) 2024-09-05T12:15:37.142 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:36 smithi059 podman[117431]: 2024-09-05 12:15:36.890588417 +0000 UTC m=+5.580579042 container died bdd4efd8669144bd4d808e0174699ebef17a1ef7ee9c2e0c6ab7b6607af66911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2) 2024-09-05T12:15:37.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:37.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:37.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:37.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:37.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:37.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:37.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:36 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:36.784+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:37.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:37.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:37.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:37.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:37.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:37.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:37 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:37.000+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:37.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:37 smithi059 podman[117431]: 2024-09-05 12:15:37.33053667 +0000 UTC m=+6.020527331 container cleanup bdd4efd8669144bd4d808e0174699ebef17a1ef7ee9c2e0c6ab7b6607af66911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-05T12:15:37.458 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:37 smithi059 bash[117431]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9 2024-09-05T12:15:37.957 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:37 smithi059 podman[117448]: 2024-09-05 12:15:37.490572667 +0000 UTC m=+0.586883489 container remove bdd4efd8669144bd4d808e0174699ebef17a1ef7ee9c2e0c6ab7b6607af66911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:38.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:37 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:37.793+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:38.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:38.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:38.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:38.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:38.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:38.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.033+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:38.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:37.95058546 +0000 UTC m=+0.027217386 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:38.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:38.057424386 +0000 UTC m=+0.134056311 container create 3168736cf6072473524d2d57211f3ce3d5d8732411a68694744d915fb8078dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T12:15:38.360 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:38.27679583 +0000 UTC m=+0.353427756 container init 3168736cf6072473524d2d57211f3ce3d5d8732411a68694744d915fb8078dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T12:15:38.361 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:38.283542939 +0000 UTC m=+0.360174865 container start 3168736cf6072473524d2d57211f3ce3d5d8732411a68694744d915fb8078dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:15:38.361 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:38.359740775 +0000 UTC m=+0.436372710 container attach 3168736cf6072473524d2d57211f3ce3d5d8732411a68694744d915fb8078dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:15:38.707 INFO:journalctl@ceph.osd.9.smithi059.stdout:Sep 05 12:15:38 smithi059 podman[117537]: 2024-09-05 12:15:38.546683391 +0000 UTC m=+0.623315328 container died 3168736cf6072473524d2d57211f3ce3d5d8732411a68694744d915fb8078dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T12:15:39.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.9.service' 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:39.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:38 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:38.795+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:39.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:39.242 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-05T12:15:39.242 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-05T12:15:39.242 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.10 2024-09-05T12:15:39.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:39.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:39.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:39.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:39.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:39.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:38 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:38.988+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:39.547 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 systemd[1]: Stopping Ceph osd.10 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:39.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.546+0000 7fd1bd65c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:39.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.546+0000 7fd1bd65c640 -1 osd.10 97 *** Got signal Terminated *** 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.546+0000 7fd1bd65c640 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:39.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:39 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:39.771+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:40.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:40.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:40.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:40.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:40.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:40.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:39.945+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:41.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:41.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:41.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:41.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:41.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:41.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:41.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:40 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:40.742+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:41.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:41.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:41.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:41.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:41.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:41.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:40 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:40.983+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:42.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:42.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:42.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:42.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:42.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:42.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:42.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:41 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:41.695+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:42.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:42.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:42.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:42.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:42.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:42.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:42 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:42.032+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:43.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:43.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:43.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:43.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:43.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:43.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:43.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:42 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:42.730+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:43.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:43.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:43.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:43.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:43.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:43.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:43 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:43.001+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:44.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:18.490330+0000 front 2024-09-05T12:14:18.490423+0000 (oldest deadline 2024-09-05T12:14:43.190455+0000) 2024-09-05T12:15:44.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:29.591715+0000 front 2024-09-05T12:14:29.591828+0000 (oldest deadline 2024-09-05T12:14:51.891996+0000) 2024-09-05T12:15:44.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.193972+0000 front 2024-09-05T12:14:38.194015+0000 (oldest deadline 2024-09-05T12:15:00.994296+0000) 2024-09-05T12:15:44.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-09-05T12:14:45.595949+0000 front 2024-09-05T12:14:45.595977+0000 (oldest deadline 2024-09-05T12:15:08.495797+0000) 2024-09-05T12:15:44.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:55.397528+0000 front 2024-09-05T12:14:55.397581+0000 (oldest deadline 2024-09-05T12:15:18.897596+0000) 2024-09-05T12:15:44.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:14:58.898271+0000 front 2024-09-05T12:14:58.898357+0000 (oldest deadline 2024-09-05T12:15:27.698464+0000) 2024-09-05T12:15:44.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:08.199790+0000 front 2024-09-05T12:15:08.199747+0000 (oldest deadline 2024-09-05T12:15:32.899490+0000) 2024-09-05T12:15:44.185 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:43 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10[58787]: 2024-09-05T12:15:43.707+0000 7fd1b5c6a640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.100772+0000 front 2024-09-05T12:15:18.100808+0000 (oldest deadline 2024-09-05T12:15:43.400829+0000) 2024-09-05T12:15:44.300 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:44.300 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:44.300 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:44.300 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:44.300 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:44.301 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:44.301 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:44 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:44.046+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:45.281 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:44 smithi089 podman[70222]: 2024-09-05 12:15:44.999348147 +0000 UTC m=+5.560406541 container died 215703ad25d27011a83f25d297ec13713f203147e234e38839a108f8a21f96fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-05T12:15:45.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:45.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:45.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:45.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:45.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:45.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:45.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:45 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:45.087+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:45.656 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:45 smithi089 podman[70222]: 2024-09-05 12:15:45.371362093 +0000 UTC m=+5.932420481 container cleanup 215703ad25d27011a83f25d297ec13713f203147e234e38839a108f8a21f96fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T12:15:45.656 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:45 smithi089 bash[70222]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10 2024-09-05T12:15:45.934 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:45 smithi089 podman[70239]: 2024-09-05 12:15:45.655913736 +0000 UTC m=+0.657866849 container remove 215703ad25d27011a83f25d297ec13713f203147e234e38839a108f8a21f96fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-05T12:15:46.243 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:45 smithi089 podman[70326]: 2024-09-05 12:15:45.865961535 +0000 UTC m=+0.016097755 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:46.243 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:45 smithi089 podman[70326]: 2024-09-05 12:15:45.964883437 +0000 UTC m=+0.115019655 container create ceada494d73a511512bce1256bc8b5dd5500aea1819f611f0b5452a5ebf006f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-05T12:15:46.243 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:46 smithi089 podman[70326]: 2024-09-05 12:15:46.242309851 +0000 UTC m=+0.392446069 container init ceada494d73a511512bce1256bc8b5dd5500aea1819f611f0b5452a5ebf006f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:46.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:46 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:46.121+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:46.684 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:46 smithi089 podman[70326]: 2024-09-05 12:15:46.245890437 +0000 UTC m=+0.396026658 container start ceada494d73a511512bce1256bc8b5dd5500aea1819f611f0b5452a5ebf006f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T12:15:46.684 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:46 smithi089 podman[70326]: 2024-09-05 12:15:46.342590164 +0000 UTC m=+0.492726402 container attach ceada494d73a511512bce1256bc8b5dd5500aea1819f611f0b5452a5ebf006f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-05T12:15:46.684 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 05 12:15:46 smithi089 podman[70326]: 2024-09-05 12:15:46.44135863 +0000 UTC m=+0.591494854 container died ceada494d73a511512bce1256bc8b5dd5500aea1819f611f0b5452a5ebf006f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-10-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:15:47.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.10.service' 2024-09-05T12:15:47.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:47.058 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-05T12:15:47.058 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-05T12:15:47.059 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.11 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.108+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:47.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 systemd[1]: Stopping Ceph osd.11 for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:47.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.381+0000 7fa41b966640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:47.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.381+0000 7fa41b966640 -1 osd.11 97 *** Got signal Terminated *** 2024-09-05T12:15:47.840 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:47 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:47.381+0000 7fa41b966640 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:15:48.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:48.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:48 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:48.135+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:49.590 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:49.590 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:49.590 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:49.590 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:49.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:49.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:49.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:49 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:49.171+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:50.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:50.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:50 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:50.160+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:51.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:51.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:51 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:51.168+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:52.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-09-05T12:14:22.174390+0000 front 2024-09-05T12:14:22.174341+0000 (oldest deadline 2024-09-05T12:14:43.874438+0000) 2024-09-05T12:15:52.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-05T12:14:28.475920+0000 front 2024-09-05T12:14:28.475911+0000 (oldest deadline 2024-09-05T12:14:53.175768+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-05T12:14:38.778142+0000 front 2024-09-05T12:14:38.778169+0000 (oldest deadline 2024-09-05T12:15:02.278334+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-05T12:14:53.980641+0000 front 2024-09-05T12:14:53.980728+0000 (oldest deadline 2024-09-05T12:15:18.080538+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-05T12:15:00.981455+0000 front 2024-09-05T12:15:00.981525+0000 (oldest deadline 2024-09-05T12:15:25.081496+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-09-05T12:15:10.982448+0000 front 2024-09-05T12:15:10.982498+0000 (oldest deadline 2024-09-05T12:15:33.882377+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-05T12:15:18.583542+0000 front 2024-09-05T12:15:18.583579+0000 (oldest deadline 2024-09-05T12:15:43.883445+0000) 2024-09-05T12:15:52.589 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11[59238]: 2024-09-05T12:15:52.211+0000 7fa417f7c640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-09-05T12:15:23.883748+0000 front 2024-09-05T12:15:23.883763+0000 (oldest deadline 2024-09-05T12:15:51.484386+0000) 2024-09-05T12:15:53.015 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:52 smithi189 podman[70428]: 2024-09-05 12:15:52.73691449 +0000 UTC m=+5.479520708 container died ea5578454a13f8db866530f6f18a11796b230f3e0bd9e33e9c97c386593955f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:15:53.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70428]: 2024-09-05 12:15:53.105977608 +0000 UTC m=+5.848583826 container cleanup ea5578454a13f8db866530f6f18a11796b230f3e0bd9e33e9c97c386593955f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:15:53.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 bash[70428]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11 2024-09-05T12:15:53.716 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70445]: 2024-09-05 12:15:53.35714368 +0000 UTC m=+0.618005523 container remove ea5578454a13f8db866530f6f18a11796b230f3e0bd9e33e9c97c386593955f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-05T12:15:54.061 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70533]: 2024-09-05 12:15:53.716233064 +0000 UTC m=+0.111995498 container create 76f21b1ca4abaa40c4ed2865ad70159664b3e31ab002f323e53f64ccaa872299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T12:15:54.061 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70533]: 2024-09-05 12:15:53.619894518 +0000 UTC m=+0.015656953 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:15:54.062 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70533]: 2024-09-05 12:15:53.927196963 +0000 UTC m=+0.322959389 container init 76f21b1ca4abaa40c4ed2865ad70159664b3e31ab002f323e53f64ccaa872299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T12:15:54.062 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:53 smithi189 podman[70533]: 2024-09-05 12:15:53.931459792 +0000 UTC m=+0.327222227 container start 76f21b1ca4abaa40c4ed2865ad70159664b3e31ab002f323e53f64ccaa872299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T12:15:54.062 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:54 smithi189 podman[70533]: 2024-09-05 12:15:54.060521811 +0000 UTC m=+0.456284250 container attach 76f21b1ca4abaa40c4ed2865ad70159664b3e31ab002f323e53f64ccaa872299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T12:15:54.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 05 12:15:54 smithi189 podman[70533]: 2024-09-05 12:15:54.122542984 +0000 UTC m=+0.518305421 container died 76f21b1ca4abaa40c4ed2865ad70159664b3e31ab002f323e53f64ccaa872299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-05T12:15:54.736 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@osd.11.service' 2024-09-05T12:15:54.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:54.774 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-05T12:15:54.774 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-05T12:15:54.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.a 2024-09-05T12:15:55.169 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:54 smithi059 systemd[1]: Stopping Ceph mds.a for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:55.458 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:55 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a[70918]: 2024-09-05T12:15:55.167+0000 7fe17da00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:55.458 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:55 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a[70918]: 2024-09-05T12:15:55.167+0000 7fe17da00640 -1 mds.a *** got signal Terminated *** 2024-09-05T12:15:57.359 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:57 smithi059 podman[117638]: 2024-09-05 12:15:57.072654715 +0000 UTC m=+2.001129941 container died 7acf1b0de9afdf6a9314c376d446c475b41aae7f243ae6d31f6b34ad677c0783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-05T12:15:57.708 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:57 smithi059 podman[117638]: 2024-09-05 12:15:57.455319011 +0000 UTC m=+2.383794237 container cleanup 7acf1b0de9afdf6a9314c376d446c475b41aae7f243ae6d31f6b34ad677c0783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-05T12:15:57.708 INFO:journalctl@ceph.mds.a.smithi059.stdout:Sep 05 12:15:57 smithi059 bash[117638]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-a 2024-09-05T12:15:57.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.a.service' 2024-09-05T12:15:57.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:15:57.966 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-05T12:15:57.966 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-05T12:15:57.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.d 2024-09-05T12:15:58.346 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:15:58 smithi059 systemd[1]: Stopping Ceph mds.d for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:15:58.707 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:15:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d[69753]: 2024-09-05T12:15:58.345+0000 7f481d800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:15:58.707 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:15:58 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d[69753]: 2024-09-05T12:15:58.345+0000 7f481d800640 -1 mds.d *** got signal Terminated *** 2024-09-05T12:16:01.016 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:16:00 smithi059 podman[117771]: 2024-09-05 12:16:00.754165881 +0000 UTC m=+2.518643816 container died f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T12:16:01.319 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:16:01 smithi059 podman[117771]: 2024-09-05 12:16:01.150828642 +0000 UTC m=+2.915306567 container cleanup f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True) 2024-09-05T12:16:01.319 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:16:01 smithi059 bash[117771]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d 2024-09-05T12:16:01.570 INFO:journalctl@ceph.mds.d.smithi059.stdout:Sep 05 12:16:01 smithi059 podman[117783]: 2024-09-05 12:16:01.318581605 +0000 UTC m=+0.557246815 container remove f8eccdd37f66ff29233121fdf2ba084beffba844f217ae50c325d4e64da5d380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:16:01.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.d.service' 2024-09-05T12:16:01.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:01.659 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-05T12:16:01.659 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-05T12:16:01.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.g 2024-09-05T12:16:01.958 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:01 smithi059 systemd[1]: Stopping Ceph mds.g for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:02.458 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g[70553]: 2024-09-05T12:16:02.041+0000 7f8d4f000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:02.458 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:02 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g[70553]: 2024-09-05T12:16:02.041+0000 7f8d4f000640 -1 mds.g *** got signal Terminated *** 2024-09-05T12:16:05.882 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:05 smithi059 podman[117892]: 2024-09-05 12:16:05.614266413 +0000 UTC m=+3.675099162 container died 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, ceph=True) 2024-09-05T12:16:06.152 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:06 smithi059 podman[117892]: 2024-09-05 12:16:06.029553327 +0000 UTC m=+4.090386132 container cleanup 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:06.152 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:06 smithi059 bash[117892]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g 2024-09-05T12:16:06.457 INFO:journalctl@ceph.mds.g.smithi059.stdout:Sep 05 12:16:06 smithi059 podman[117905]: 2024-09-05 12:16:06.206208327 +0000 UTC m=+0.580659541 container remove 30aa7770c1568cb53744aa46ba9ed442ba1e0414fed9a67003c4b8279888a1c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:06.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.g.service' 2024-09-05T12:16:06.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:06.560 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-05T12:16:06.560 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-05T12:16:06.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.j 2024-09-05T12:16:06.782 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:06 smithi059 systemd[1]: Stopping Ceph mds.j for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:07.208 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j[69371]: 2024-09-05T12:16:06.936+0000 7fabd4800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:07.208 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:06 smithi059 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j[69371]: 2024-09-05T12:16:06.936+0000 7fabd4800640 -1 mds.j *** got signal Terminated *** 2024-09-05T12:16:09.052 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:08 smithi059 podman[118015]: 2024-09-05 12:16:08.78728164 +0000 UTC m=+1.943755908 container died ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:16:09.351 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:09 smithi059 podman[118015]: 2024-09-05 12:16:09.199823183 +0000 UTC m=+2.356297450 container cleanup ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T12:16:09.352 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:09 smithi059 bash[118015]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j 2024-09-05T12:16:09.602 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:09 smithi059 podman[118026]: 2024-09-05 12:16:09.351083314 +0000 UTC m=+0.552581602 container remove ede2494b26fa607cb31858a9c6c60706d6bbeaee1a78ea9c856b0890c622addd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-j, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-05T12:16:09.603 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:09 smithi059 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.j.service: Deactivated successfully. 2024-09-05T12:16:09.603 INFO:journalctl@ceph.mds.j.smithi059.stdout:Sep 05 12:16:09 smithi059 systemd[1]: Stopped Ceph mds.j for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T12:16:09.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.j.service' 2024-09-05T12:16:09.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:09.676 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-05T12:16:09.677 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-05T12:16:09.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.b 2024-09-05T12:16:09.959 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:09 smithi089 systemd[1]: Stopping Ceph mds.b for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:10.434 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: 2024-09-05T12:16:09.996+0000 7f70a8d96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:10.434 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:09 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b[64015]: 2024-09-05T12:16:09.996+0000 7f70a8d96640 -1 mds.b *** got signal Terminated *** 2024-09-05T12:16:14.322 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:14 smithi089 podman[70428]: 2024-09-05 12:16:14.008873125 +0000 UTC m=+4.141123659 container died 9d21701c70f7500676d5bf9ef56ed6c8691f5a57f401842d9149089cb604d94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T12:16:14.684 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:14 smithi089 podman[70428]: 2024-09-05 12:16:14.404016937 +0000 UTC m=+4.536267468 container cleanup 9d21701c70f7500676d5bf9ef56ed6c8691f5a57f401842d9149089cb604d94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:14.684 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 05 12:16:14 smithi089 bash[70428]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-b 2024-09-05T12:16:14.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.b.service' 2024-09-05T12:16:14.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:14.922 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-05T12:16:14.922 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-05T12:16:14.922 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.e 2024-09-05T12:16:15.243 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:15 smithi089 systemd[1]: Stopping Ceph mds.e for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:15.684 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e[63394]: 2024-09-05T12:16:15.241+0000 7f0a69eb0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:15.684 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:15 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e[63394]: 2024-09-05T12:16:15.241+0000 7f0a69eb0640 -1 mds.e *** got signal Terminated *** 2024-09-05T12:16:19.534 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:19 smithi089 podman[70561]: 2024-09-05 12:16:19.247876263 +0000 UTC m=+4.110988663 container died f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T12:16:19.909 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:19 smithi089 podman[70561]: 2024-09-05 12:16:19.624155562 +0000 UTC m=+4.487267961 container cleanup f73f702ee2a39225c15cab932d1dee31a3aa07dc87c15a0f383a68fa8d9b87b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:16:19.909 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 05 12:16:19 smithi089 bash[70561]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-e 2024-09-05T12:16:20.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.e.service' 2024-09-05T12:16:20.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:20.083 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-05T12:16:20.083 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-05T12:16:20.083 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.h 2024-09-05T12:16:20.434 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:20 smithi089 systemd[1]: Stopping Ceph mds.h for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:20.434 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: 2024-09-05T12:16:20.386+0000 7f64900bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:20.434 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:20 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h[63714]: 2024-09-05T12:16:20.386+0000 7f64900bc640 -1 mds.h *** got signal Terminated *** 2024-09-05T12:16:22.979 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:22 smithi089 podman[70683]: 2024-09-05 12:16:22.708009343 +0000 UTC m=+2.409032849 container died 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-05T12:16:23.330 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:23 smithi089 podman[70683]: 2024-09-05 12:16:23.078074126 +0000 UTC m=+2.779097629 container cleanup 1b01f8ddc93871d85ab6f7ff88a9926921cae5bdf22b8649075697af25728ec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T12:16:23.330 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 05 12:16:23 smithi089 bash[70683]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-h 2024-09-05T12:16:23.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.h.service' 2024-09-05T12:16:23.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:23.536 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-05T12:16:23.536 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-05T12:16:23.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.k 2024-09-05T12:16:23.857 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:23 smithi089 systemd[1]: Stopping Ceph mds.k for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:24.184 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k[63081]: 2024-09-05T12:16:23.856+0000 7f5ed4af6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:24.184 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:23 smithi089 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k[63081]: 2024-09-05T12:16:23.856+0000 7f5ed4af6640 -1 mds.k *** got signal Terminated *** 2024-09-05T12:16:25.141 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:24 smithi089 podman[70807]: 2024-09-05 12:16:24.850566658 +0000 UTC m=+1.098777065 container died e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:25.434 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:25 smithi089 podman[70807]: 2024-09-05 12:16:25.214983226 +0000 UTC m=+1.463193630 container cleanup e7d671bed9dd8fc4ae74c2c91c07df58c40c82b888411d3729d9302312f697d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-05T12:16:25.434 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 05 12:16:25 smithi089 bash[70807]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-k 2024-09-05T12:16:25.633 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.k.service' 2024-09-05T12:16:25.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:25.666 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-05T12:16:25.666 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-05T12:16:25.666 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.c 2024-09-05T12:16:25.965 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:25 smithi189 systemd[1]: Stopping Ceph mds.c for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:26.338 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c[64534]: 2024-09-05T12:16:25.963+0000 7f8c2196a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:26.338 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:25 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c[64534]: 2024-09-05T12:16:25.963+0000 7f8c2196a640 -1 mds.c *** got signal Terminated *** 2024-09-05T12:16:29.088 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:28 smithi189 podman[70639]: 2024-09-05 12:16:28.773868986 +0000 UTC m=+2.901075576 container died b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:16:29.515 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:29 smithi189 podman[70639]: 2024-09-05 12:16:29.288558766 +0000 UTC m=+3.415765356 container cleanup b69f493c97a9fd9f101bf4e585bb1773599226707529a0221a0fdfc246817549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T12:16:29.515 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 05 12:16:29 smithi189 bash[70639]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-c 2024-09-05T12:16:29.736 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.c.service' 2024-09-05T12:16:29.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:29.773 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-05T12:16:29.773 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-05T12:16:29.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.f 2024-09-05T12:16:30.088 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:29 smithi189 systemd[1]: Stopping Ceph mds.f for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:30.588 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f[64230]: 2024-09-05T12:16:30.117+0000 7f93bc281640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:30.588 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:30 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f[64230]: 2024-09-05T12:16:30.117+0000 7f93bc281640 -1 mds.f *** got signal Terminated *** 2024-09-05T12:16:34.493 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:34 smithi189 podman[70772]: 2024-09-05 12:16:34.131375263 +0000 UTC m=+4.122508406 container died 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:34.838 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:34 smithi189 podman[70772]: 2024-09-05 12:16:34.566945667 +0000 UTC m=+4.558078811 container cleanup 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-05T12:16:34.838 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:34 smithi189 bash[70772]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f 2024-09-05T12:16:34.838 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 05 12:16:34 smithi189 podman[70786]: 2024-09-05 12:16:34.73467535 +0000 UTC m=+0.600669652 container remove 0f28ac9dc9b26d7edb5ea0e11f101c068609bee8e1bcde5da6c36e25dd395f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:34.961 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.f.service' 2024-09-05T12:16:34.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:34.994 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-05T12:16:34.995 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-05T12:16:34.995 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.i 2024-09-05T12:16:35.313 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:35 smithi189 systemd[1]: Stopping Ceph mds.i for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:35.588 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:35 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i[64836]: 2024-09-05T12:16:35.311+0000 7f5fccbd4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:35.588 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:35 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i[64836]: 2024-09-05T12:16:35.311+0000 7f5fccbd4640 -1 mds.i *** got signal Terminated *** 2024-09-05T12:16:38.588 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:38 smithi189 podman[70893]: 2024-09-05 12:16:38.291192746 +0000 UTC m=+3.078986960 container died 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:16:38.838 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:38 smithi189 podman[70893]: 2024-09-05 12:16:38.68733039 +0000 UTC m=+3.475124609 container cleanup 4e17eda66df8b87463ef4985ece65e747915bee11acafb93384b6deee0936306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2) 2024-09-05T12:16:38.838 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 05 12:16:38 smithi189 bash[70893]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-i 2024-09-05T12:16:39.091 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.i.service' 2024-09-05T12:16:39.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:39.124 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-05T12:16:39.124 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-05T12:16:39.124 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.l 2024-09-05T12:16:39.338 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:39 smithi189 systemd[1]: Stopping Ceph mds.l for 53d560d4-6b7c-11ef-bcd6-c7b262605968... 2024-09-05T12:16:39.838 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l[63918]: 2024-09-05T12:16:39.540+0000 7f44b2bd6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:39.838 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:39 smithi189 ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l[63918]: 2024-09-05T12:16:39.540+0000 7f44b2bd6640 -1 mds.l *** got signal Terminated *** 2024-09-05T12:16:42.317 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 podman[71015]: 2024-09-05 12:16:42.052357505 +0000 UTC m=+2.711767218 container died 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-05T12:16:42.588 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 podman[71015]: 2024-09-05 12:16:42.415862068 +0000 UTC m=+3.075271777 container cleanup 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T12:16:42.588 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 bash[71015]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l 2024-09-05T12:16:42.831 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 podman[71027]: 2024-09-05 12:16:42.691841812 +0000 UTC m=+0.632488978 container remove 7881afaa67bf8f5f113f2ae1048bcbc037b7c0afb17a328de708f3a536796ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:16:42.831 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-05T12:16:42.832 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 systemd[1]: Stopped Ceph mds.l for 53d560d4-6b7c-11ef-bcd6-c7b262605968. 2024-09-05T12:16:42.832 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 05 12:16:42 smithi189 systemd[1]: ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.l.service: Consumed 1.149s CPU time. 2024-09-05T12:16:42.833 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53d560d4-6b7c-11ef-bcd6-c7b262605968@mds.l.service' 2024-09-05T12:16:42.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:16:42.864 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-05T12:16:42.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:16:43.148 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:16:47.142 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:16:47.342 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:16:49.781 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:16:49.980 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:16:52.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:16:52.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:16:52.438 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:16:52.464 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-05T12:16:52.467 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi059/crash 2024-09-05T12:16:52.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:16:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:16:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:16:52.548 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi089/crash 2024-09-05T12:16:52.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:16:52.662 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:16:52.662 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:16:52.664 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi189/crash 2024-09-05T12:16:52.665 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:16:52.747 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:16:52.747 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:16:52.748 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-05T12:16:52.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-05T12:16:52.831 INFO:tasks.cephadm:Compressing logs... 2024-09-05T12:16:52.831 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:16:52.835 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:16:52.838 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:: No such file or directory 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.i.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log: /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.c.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.11.log 2024-09-05T12:16:52.865 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.2.log 2024-09-05T12:16:52.871 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.10.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.1.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.7.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.b.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.k.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log: /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.4.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:16:52.872 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.11.log: 83.2% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.5.log 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.l.log 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr: 90.4% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T12:16:52.873 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.i.log.gz 2024-09-05T12:16:52.874 INFO:teuthology.orchestra.run.smithi189.stderr: 88.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:16:52.874 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.8.log 2024-09-05T12:16:52.874 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.z.log 2024-09-05T12:16:52.874 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.c.log 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.y.log 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log: 83.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr: 94.5% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.k.log.gz 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:16:52.876 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.y.log: --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.h.log 2024-09-05T12:16:52.878 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.z.log: 91.0% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:16:52.878 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.l.log.gz 2024-09-05T12:16:52.878 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.f.log 2024-09-05T12:16:52.878 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:16:52.879 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.f.log: 93.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.z.log.gz 2024-09-05T12:16:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log: 88.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:16:52.880 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.b.log 2024-09-05T12:16:52.881 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:16:52.881 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:16:52.882 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.c.log.gz 2024-09-05T12:16:52.882 INFO:teuthology.orchestra.run.smithi089.stderr: 93.2% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.y.log.gz 2024-09-05T12:16:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.e.log 2024-09-05T12:16:52.884 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:16:52.884 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.f.log.gz 2024-09-05T12:16:52.886 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-05T12:16:52.886 INFO:teuthology.orchestra.run.smithi059.stderr:: No such file or directory 2024-09-05T12:16:52.887 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.0.log 2024-09-05T12:16:52.887 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.9.log 2024-09-05T12:16:52.888 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:16:52.889 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.9.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-05T12:16:52.889 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:16:52.890 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.a.log 2024-09-05T12:16:52.891 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.e.log.gz 2024-09-05T12:16:52.891 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:16:52.893 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.a.log 2024-09-05T12:16:52.893 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:16:52.894 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.g.log 2024-09-05T12:16:52.894 INFO:teuthology.orchestra.run.smithi059.stderr: 90.2% 82.9% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:16:52.895 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T12:16:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.6.log 2024-09-05T12:16:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:16:52.899 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.6.log: /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.d.log 2024-09-05T12:16:52.899 INFO:teuthology.orchestra.run.smithi059.stderr: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.a.log.gz 2024-09-05T12:16:52.900 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.x.log 2024-09-05T12:16:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.g.log.gz 2024-09-05T12:16:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.x.log: 91.0% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:16:52.906 INFO:teuthology.orchestra.run.smithi059.stderr: 94.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.d.log.gz 2024-09-05T12:16:52.908 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:16:52.909 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.j.log 2024-09-05T12:16:52.909 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.3.log 2024-09-05T12:16:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:16:52.918 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.j.log.gz 2024-09-05T12:16:52.926 INFO:teuthology.orchestra.run.smithi059.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:16:53.176 INFO:teuthology.orchestra.run.smithi089.stderr: 94.2% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.h.log.gz 2024-09-05T12:16:53.275 INFO:teuthology.orchestra.run.smithi059.stderr: 90.2% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mgr.x.log.gz 2024-09-05T12:16:53.469 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.c.log.gz 2024-09-05T12:16:53.500 INFO:teuthology.orchestra.run.smithi089.stderr: 91.6% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.b.log.gz 2024-09-05T12:16:53.861 INFO:teuthology.orchestra.run.smithi059.stderr: 92.8% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.6.log.gz 2024-09-05T12:16:54.015 INFO:teuthology.orchestra.run.smithi189.stderr: 92.0% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.11.log.gz 2024-09-05T12:16:54.076 INFO:teuthology.orchestra.run.smithi059.stderr: 92.8% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.0.log.gz 2024-09-05T12:16:54.139 INFO:teuthology.orchestra.run.smithi189.stderr: 93.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.8.log.gz 2024-09-05T12:16:54.231 INFO:teuthology.orchestra.run.smithi089.stderr: 92.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.1.log.gz 2024-09-05T12:16:54.251 INFO:teuthology.orchestra.run.smithi059.stderr: 93.4% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.9.log.gz 2024-09-05T12:16:54.321 INFO:teuthology.orchestra.run.smithi059.stderr: 93.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.3.log.gz 2024-09-05T12:16:54.325 INFO:teuthology.orchestra.run.smithi059.stderr: 90.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mon.a.log.gz 2024-09-05T12:16:54.328 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-09-05T12:16:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m1.464s 2024-09-05T12:16:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m6.258s 2024-09-05T12:16:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m0.609s 2024-09-05T12:16:54.633 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-mds.b.log.gz 2024-09-05T12:16:54.655 INFO:teuthology.orchestra.run.smithi189.stderr: 91.7% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.2.log.gz 2024-09-05T12:16:54.679 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.7.log.gz 2024-09-05T12:16:54.857 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.4.log.gz 2024-09-05T12:16:54.885 INFO:teuthology.orchestra.run.smithi089.stderr: 92.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.10.log.gz 2024-09-05T12:16:54.887 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-05T12:16:54.887 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m2.034s 2024-09-05T12:16:54.887 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m9.581s 2024-09-05T12:16:54.888 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.344s 2024-09-05T12:16:55.090 INFO:teuthology.orchestra.run.smithi189.stderr: 92.3% -- replaced with /var/log/ceph/53d560d4-6b7c-11ef-bcd6-c7b262605968/ceph-osd.5.log.gz 2024-09-05T12:16:55.092 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-05T12:16:55.092 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m2.239s 2024-09-05T12:16:55.093 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m6.864s 2024-09-05T12:16:55.093 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.242s 2024-09-05T12:16:55.093 INFO:tasks.cephadm:Archiving logs... 2024-09-05T12:16:55.094 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi059/log 2024-09-05T12:16:55.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:16:55.999 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi089/log 2024-09-05T12:16:56.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:16:57.353 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi189/log 2024-09-05T12:16:57.354 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:16:58.472 INFO:tasks.cephadm:Removing cluster... 2024-09-05T12:16:58.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force 2024-09-05T12:16:58.758 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:16:59.213 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force 2024-09-05T12:16:59.401 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:16:59.673 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53d560d4-6b7c-11ef-bcd6-c7b262605968 --force 2024-09-05T12:16:59.876 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 53d560d4-6b7c-11ef-bcd6-c7b262605968 2024-09-05T12:17:00.143 INFO:tasks.cephadm:Removing cephadm ... 2024-09-05T12:17:00.143 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:17:00.182 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:17:00.199 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:17:00.214 INFO:tasks.cephadm:Teardown complete 2024-09-05T12:17:00.214 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T12:17:00.225 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T12:17:00.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:17:00.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:17:00.241 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:17:00.357 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T12:17:00.369 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T12:17:00.369 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:17:00.374 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:17:00.376 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:17:00.389 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:17:00.390 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +400us[ +399us] +/- 55ms 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -514us[ -515us] +/- 68ms 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 129 +84us[ +89us] +/- 63ms 2024-09-05T12:17:00.393 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 65 +369us[ +372us] +/- 55ms 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -552us[ -552us] +/- 68ms 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +31us[ +31us] +/- 65ms 2024-09-05T12:17:00.395 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:17:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:17:00.410 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:17:00.410 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-09-05T12:17:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +359us[ +355us] +/- 55ms 2024-09-05T12:17:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -524us[ -524us] +/- 69ms 2024-09-05T12:17:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +33us[ +33us] +/- 65ms 2024-09-05T12:17:00.411 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:17:00.412 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T12:17:00.422 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T12:17:00.423 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T12:17:00.435 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:17:00.591 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:17:00.661 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:17:00.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:17:00.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:00.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:17:00.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:00.819 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:17:00.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:00.847 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T12:17:00.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T12:17:00.868 INFO:teuthology.task.internal:Duration was 2290.489585 seconds 2024-09-05T12:17:00.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T12:17:00.879 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T12:17:00.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:17:00.884 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:17:00.886 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:17:00.923 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:17:00.926 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:17:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:17:01.267 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T12:17:01.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-09-05T12:17:01.268 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:17:01.317 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-05T12:17:01.317 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:17:01.342 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-05T12:17:01.342 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:17:01.364 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T12:17:01.365 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:17:01.369 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:17:01.385 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:17:01.431 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T12:17:01.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:17:01.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:17:01.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:17:02.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T12:17:02.916 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T12:17:02.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:17:02.973 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:17:03.003 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:17:03.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T12:17:03.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.054 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.078 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-05T12:17:03.101 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-05T12:17:03.118 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-09-05T12:17:03.147 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:03.181 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:03.197 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:17:03.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:17:03.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T12:17:03.226 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T12:17:03.227 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi059 2024-09-05T12:17:03.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:17:03.332 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi089 2024-09-05T12:17:03.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:17:03.375 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890407/remote/smithi189 2024-09-05T12:17:03.375 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:17:03.415 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T12:17:03.415 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:17:03.419 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:17:03.422 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:17:03.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T12:17:03.479 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T12:17:03.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T12:17:03.490 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T12:17:03.490 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:17:03.496 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:17:03.498 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:17:03.512 INFO:teuthology.orchestra.run.smithi089.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:17 /home/ubuntu/cephtest 2024-09-05T12:17:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:17 /home/ubuntu/cephtest 2024-09-05T12:17:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:17 /home/ubuntu/cephtest 2024-09-05T12:17:03.529 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T12:17:03.540 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T12:17:03.566 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2290.4895853996277 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-05T12:17:03.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T12:17:03.633 INFO:teuthology.run:pass